CN114549854A - Method, system and equipment for acquiring overall space contour of food - Google Patents

Method, system and equipment for acquiring overall space contour of food Download PDF

Info

Publication number
CN114549854A
CN114549854A CN202210189618.1A CN202210189618A CN114549854A CN 114549854 A CN114549854 A CN 114549854A CN 202210189618 A CN202210189618 A CN 202210189618A CN 114549854 A CN114549854 A CN 114549854A
Authority
CN
China
Prior art keywords
container
meal
boundary line
point cloud
food
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202210189618.1A
Other languages
Chinese (zh)
Inventor
王慧
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shanghai Jiaotong University School of Medicine
Original Assignee
Shanghai Jiaotong University School of Medicine
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shanghai Jiaotong University School of Medicine filed Critical Shanghai Jiaotong University School of Medicine
Priority to CN202210189618.1A priority Critical patent/CN114549854A/en
Publication of CN114549854A publication Critical patent/CN114549854A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
    • G06T17/20—Finite element generation, e.g. wire-frame surface description, tesselation
    • G06T17/205—Re-meshing
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformations in the plane of the image
    • G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
    • G06T3/4084—Scaling of whole images or parts thereof, e.g. expanding or contracting in the transform domain, e.g. fast Fourier transform [FFT] domain scaling
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/187—Segmentation; Edge detection involving region growing; involving region merging; involving connected component labelling
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/60—Analysis of geometric attributes
    • G06T7/62—Analysis of geometric attributes of area, perimeter, diameter or volume
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10028—Range image; Depth image; 3D point clouds
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
    • Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
    • Y02P90/30—Computing systems specially adapted for manufacturing

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Geometry (AREA)
  • Computer Graphics (AREA)
  • Software Systems (AREA)
  • Image Analysis (AREA)
  • Image Processing (AREA)

Abstract

The invention provides a method, a system and equipment for acquiring the overall space contour of food, comprising the following steps: magnifying a mesh surface corresponding to the meal surface to a real size; obtaining the part of the grid surface of the inner surface of the container, which is positioned at the lower part of the boundary line, according to the boundary line between the meal and the container; and combining the mesh surface of the diet surface with the actual size and the part of the mesh surface of the inner surface of the container, which is positioned at the lower part of the boundary line, into a closed space curved surface as the whole space contour of the food. The method can acquire the three-dimensional contour of the food in the three-dimensional space, and is favorable for accurately calculating the volume of the food. The invention can accurately distinguish the boundary line between the food and the container. The method can be widely applied to the scene of identifying and evaluating the dietary nutrients.

Description

Method, system and equipment for acquiring overall space contour of food
Technical Field
The invention relates to the field of food volume calculation, in particular to a method, a system and equipment for acquiring the overall spatial contour of food.
Background
Patent document CN105320955A discloses a meal quality management detection module and a meal quality management detection method using the same. The meal quality management detection module comprises two-dimensional image capturing equipment, three-dimensional image capturing equipment and an analysis unit. The two-dimensional image capturing device is used for obtaining a two-dimensional image of a meal box filled with meal preparation contents, and the two-dimensional image of the meal box has the two-dimensional image of the meal preparation contents. The three-dimensional image capturing device is used for acquiring depth information of dish variety styles of catering contents. The analyzing unit is used for analyzing the two-dimensional image of the meal box to identify the type of the meal box, analyzing the two-dimensional image of the meal distribution content to obtain the texture feature and the color feature of the two-dimensional image of the meal distribution content, analyzing the variety pattern of the meal distribution content according to the texture feature and the color feature, and calculating the weight of the meal distribution content according to the depth information of the two-dimensional image of the meal distribution content and the dish variety pattern. The three-dimensional image capturing device can acquire depth information of dish variety styles of catering contents M1-M5 of the lunch box. The analysis unit can analyze the image of the catering content and the depth information of the dish variety and the style so as to calculate the quantity of the catering content and further judge whether the quantity of the catering content meets the dish specification or not.
However, the patent document CN105320955A does not address how to obtain the outline of the food.
Disclosure of Invention
Aiming at the defects in the prior art, the invention aims to provide a method, a system and equipment for acquiring the overall spatial profile of food.
The invention provides a method for acquiring the overall space contour of food, which comprises the following steps:
step S1: magnifying a mesh surface corresponding to the meal surface to a real size;
step S2: obtaining the part of the grid surface of the inner surface of the container, which is positioned at the lower part of the boundary line, according to the boundary line between the meal and the container;
step S3: and combining the mesh surface of the diet surface with the actual size and the part of the mesh surface of the inner surface of the container, which is positioned at the lower part of the boundary line, into a closed space curved surface as the whole space contour of the food.
Preferably, in the step S1, the mesh plane corresponding to the meal surface is enlarged to the actual size according to the known distance between the bearing plane and the depth camera, the known three-dimensional model of the height, diameter and other dimensions of the container, and the scaling.
Preferably, in the step S2, in order to obtain the boundary line, the characteristic information of the container is obtained according to the meal image recognition; then removing the point cloud of the container indicated by the characteristic information according to the characteristic information of the container, and obtaining a grid surface of the inner surface of the container, which is indicated by the characteristic information and contains the actual size of the meal, of the container; and obtaining the boundary line between the inner surface of the container and the meal according to the point cloud of the container and the point cloud of the meal.
Preferably, on the basis of the point cloud of the container and the point cloud of the meal, a plurality of coils coaxial with the circular container are arranged in the three-dimensional information image, the coils are in contact with the point cloud of the container and the point cloud of the meal, the number of the points of the container and the number of the points of the meal on each coil are counted, and the coils with the number equal to the number of the points of the container and the number equal to the number of the points of the meal are taken as the boundary line.
According to the invention, the system for acquiring the overall spatial profile of the food comprises:
module M1: magnifying a mesh surface corresponding to the meal surface to a real size;
module M2: obtaining the part of the grid surface of the inner surface of the container, which is positioned at the lower part of the boundary line, according to the boundary line between the meal and the container;
module M3: and combining the mesh surface of the diet surface with the actual size and the part of the mesh surface of the inner surface of the container, which is positioned at the lower part of the boundary line, into a closed space curved surface as the whole space contour of the food.
Preferably, in said module M1, the mesh plane corresponding to the meal surface is scaled up to the actual size according to a known distance between the bearing plane and the depth camera, a known three-dimensional model of the dimensions of the height, diameter, etc. of the container.
Preferably, in the module M2, in order to obtain the boundary line, characteristic information of the container is obtained according to the meal image recognition; then removing the point cloud of the container indicated by the characteristic information according to the characteristic information of the container, and obtaining a grid surface of the inner surface of the container, which is indicated by the characteristic information and contains the actual size of the meal, of the container; and obtaining the boundary line between the inner surface of the container and the meal according to the point cloud of the container and the point cloud of the meal.
Preferably, on the basis of the point cloud of the container and the point cloud of the meal, a plurality of coils coaxial with the circular container are arranged in the three-dimensional information image, the coils are in contact with the point cloud of the container and the point cloud of the meal, the number of the points of the container and the number of the points of the meal on each coil are counted, and the coils with the number equal to the number of the points of the container and the number equal to the number of the points of the meal are taken as the boundary line.
According to the present invention, there is provided a computer-readable storage medium storing a computer program which, when executed by a processor, implements the steps of the method for obtaining a global spatial profile of a food.
The intelligent equipment of the dietary nutrition assessment terminal comprises a controller, a camera and a depth camera, wherein the camera and the depth camera are used for collecting dietary images under the control of the controller;
the controller comprises the food overall space contour acquisition system or the computer-readable storage medium storing the computer program.
Compared with the prior art, the invention has the following beneficial effects:
1. the method can acquire the three-dimensional contour of the food in the three-dimensional space, and is favorable for accurately calculating the volume of the food.
2. The invention can accurately distinguish the boundary line between the food and the container.
3. The method can be widely applied to the scene of identifying and evaluating the dietary nutrients.
Drawings
Other features, objects and advantages of the invention will become more apparent upon reading of the detailed description of non-limiting embodiments with reference to the following drawings:
FIG. 1 is a schematic overall flow chart of the present invention.
FIG. 2 is a flow chart illustrating boundary line recognition according to the present invention.
Fig. 3 is a schematic diagram of a two-dimensional information image of a meal and a container top view angle acquired by a camera according to the present invention.
FIG. 4 is a schematic diagram of a three-dimensional information image of a meal and a container acquired by a depth camera according to the present invention.
Fig. 5 is a schematic diagram of the dietary profile acquisition of the present invention.
Detailed Description
The present invention will be described in detail with reference to specific examples. The following examples will assist those skilled in the art in further understanding the invention, but are not intended to limit the invention in any way. It should be noted that it would be obvious to those skilled in the art that various changes and modifications can be made without departing from the spirit of the invention. All falling within the scope of the present invention.
The invention provides a method for acquiring the overall space contour of food, which comprises the following steps:
step S1: magnifying a mesh surface corresponding to the meal surface to a real size; step S2: obtaining the part of the grid surface of the inner surface of the container, which is positioned at the lower part of the boundary line, according to the boundary line between the meal and the container; step S3: and combining the mesh surface of the diet surface with the actual size and the part of the mesh surface of the inner surface of the container, which is positioned at the lower part of the boundary line, into a closed space curved surface as the whole space contour of the food.
In particular, from a known three-dimensional model of the distance between the carrier plane and the depth camera, a known height, diameter, etc. of the container, a scaling may be obtained, as described above, to enlarge the mesh plane corresponding to the meal surface to the actual size. In order to obtain the boundary line, characteristic information of the container is obtained according to the meal image identification, wherein the characteristic information can be a two-dimensional code for example, and the two-dimensional code is used for indicating only one container or only one container category with the same specification; then removing the point cloud of the container indicated by the characteristic information according to the characteristic information of the container, and obtaining a grid surface of the inner surface of the container, which is indicated by the characteristic information and contains the actual size of the meal, of the container; and obtaining the boundary line between the inner surface of the container and the meal, namely the boundary line between the inner surface of the container and the grid surface of the meal surface according to the point cloud of the container and the point cloud of the meal. In a preferred embodiment, a plurality of coils coaxial with the circular container are provided in the three-dimensional information image based on the point cloud of the container and the point cloud of the meal, the coils are in contact with the point cloud of the container and the point cloud of the meal, the number of the points of the container and the number of the points of the meal on each coil are counted, and the coils corresponding to the same number of the points are used as the boundary line. Combining the mesh surface of the diet surface with the actual size and the part of the mesh surface of the inner surface of the container, which is positioned at the lower part of the boundary line, into a closed space curved surface; as shown in fig. 5, fig. 5 shows the mesh plane 100 of the full-size meal surface, the mesh plane 200 of the container inner surface, the portion 300 of the container inner surface where the mesh plane is located at the lower part of the boundary line, the boundary line 400, and the container 500. And taking the volume of the curved inner space of the closed space as the volume of the meal.
Wherein the meal image comprises: the two-dimensional information image of the top view angle photographed by the camera and the three-dimensional information image photographed by the depth camera are respectively the two-dimensional information image shown in fig. 3 and the three-dimensional information image shown in fig. 4. And identifying dishes and containers of the meal through the trained neural network according to the two-dimensional information image. During the training process, images of combinations of different dishes and containers are prepared as samples, so that the neural network can identify the corresponding dishes and containers. The dish of the meal is the name of the meal, namely the name of the dish, so as to distinguish different meals. For example, the protein content of fried green pepper is less than that of the shredded pork under the same volume, so that different meals need to be distinguished to obtain the nutrient content of the meal according to the volume of the meal by using known information.
The present invention also provides a food overall spatial profile acquiring system, which can be realized by those skilled in the art by executing the step flow of the food overall spatial profile acquiring method, that is, the food overall spatial profile acquiring method can be understood as a preferred embodiment of the food overall spatial profile acquiring system.
According to the invention, the system for acquiring the overall spatial profile of the food comprises:
module M1: magnifying a mesh surface corresponding to the meal surface to a real size;
module M2: obtaining the part of the grid surface of the inner surface of the container, which is positioned at the lower part of the boundary line, according to the boundary line between the meal and the container;
module M3: and combining the mesh surface of the diet surface with the actual size and the part of the mesh surface of the inner surface of the container, which is positioned at the lower part of the boundary line, into a closed space curved surface as the whole space contour of the food.
In said module M1, the mesh plane corresponding to the meal surface is scaled up to the actual size according to the known distance between the carrier plane and the depth camera, the known three-dimensional model of the dimensions of height, diameter, etc. of the container.
In the module M2, in order to obtain the boundary line, characteristic information of the container is obtained according to the meal image recognition; then removing the point cloud of the container indicated by the characteristic information according to the characteristic information of the container, and obtaining a grid surface of the inner surface of the container, which is indicated by the characteristic information and contains the actual size of the meal, of the container; and obtaining the boundary line between the inner surface of the container and the meal according to the point cloud of the container and the point cloud of the meal.
Based on the point cloud of the container and the point cloud of the meal which are distinguished, a plurality of coils coaxial with the circular container are arranged in the three-dimensional information image, the coils are in contact with the point cloud of the container and the point cloud of the meal, the number of the points of the container and the number of the points of the meal on each coil are counted, and the coils with the same number are used as the boundary line.
According to the present invention, there is provided a computer-readable storage medium storing a computer program which, when executed by a processor, implements the steps of the method for obtaining a global spatial profile of a food.
The intelligent equipment of the dietary nutrition assessment terminal comprises a controller, a camera and a depth camera, wherein the camera and the depth camera are used for collecting dietary images under the control of the controller;
the controller comprises the food overall space contour acquisition system or the computer-readable storage medium storing the computer program.
Those skilled in the art will appreciate that, in addition to implementing the systems, apparatus, and various modules thereof provided by the present invention in purely computer readable program code, the same procedures can be implemented entirely by logically programming method steps such that the systems, apparatus, and various modules thereof are provided in the form of logic gates, switches, application specific integrated circuits, programmable logic controllers, embedded microcontrollers and the like. Therefore, the system, the device and the modules thereof provided by the present invention can be considered as a hardware component, and the modules included in the system, the device and the modules thereof for implementing various programs can also be considered as structures in the hardware component; modules for performing various functions may also be considered to be both software programs for performing the methods and structures within hardware components.
The foregoing description of specific embodiments of the present invention has been presented. It is to be understood that the present invention is not limited to the specific embodiments described above, and that various changes or modifications may be made by one skilled in the art within the scope of the appended claims without departing from the spirit of the invention. The embodiments and features of the embodiments of the present application may be combined with each other arbitrarily without conflict.

Claims (10)

1. A method for acquiring the overall spatial profile of food is characterized by comprising the following steps:
step S1: magnifying a mesh surface corresponding to the meal surface to a real size;
step S2: obtaining the part of the grid surface of the inner surface of the container, which is positioned at the lower part of the boundary line, according to the boundary line between the meal and the container;
step S3: and combining the mesh surface of the diet surface with the actual size and the part of the mesh surface of the inner surface of the container, which is positioned at the lower part of the boundary line, into a closed space curved surface as the whole space contour of the food.
2. The method for obtaining the overall spatial profile of food according to claim 1, wherein in the step S1, the mesh plane corresponding to the meal surface is scaled up to the actual size according to the known distance between the loading plane and the depth camera, the known three-dimensional model of the container with the same size as the height and diameter of the container.
3. The method for obtaining the overall spatial profile of food according to claim 2, wherein in step S2, in order to obtain the boundary line, the characteristic information of the container is obtained according to the meal image recognition; then removing the point cloud of the container indicated by the characteristic information according to the characteristic information of the container, and obtaining a grid surface of the inner surface of the container, which is indicated by the characteristic information and contains the actual size of the meal, of the container; and obtaining the boundary line between the inner surface of the container and the meal according to the point cloud of the container and the point cloud of the meal.
4. The method according to claim 3, wherein a plurality of coils coaxial with the circular container are provided in the three-dimensional information image based on the point cloud from which the container and the point cloud from which the meal have been distinguished, the coils are in contact with the point cloud from the container and the point cloud from the meal, the number of both the container point and the meal point on each coil is counted, and the coil corresponding to the same number of both is used as the boundary line.
5. A system for obtaining a food global spatial profile, comprising:
module M1: magnifying a mesh surface corresponding to the meal surface to a real size;
module M2: obtaining the part of the grid surface of the inner surface of the container, which is positioned at the lower part of the boundary line, according to the boundary line between the meal and the container;
module M3: and combining the mesh surface of the diet surface with the actual size and the part of the mesh surface of the inner surface of the container, which is positioned at the lower part of the boundary line, into a closed space curved surface as the whole space contour of the food.
6. The system for obtaining the overall spatial profile of food according to claim 5, wherein in the module M1, the mesh plane corresponding to the meal surface is scaled up to the actual size according to the known distance between the bearing plane and the depth camera, the known three-dimensional model of the height, diameter, etc. of the container.
7. The system for obtaining the overall spatial profile of food according to claim 6, wherein in the module M2, in order to obtain the boundary line, the characteristic information of the container is obtained according to the meal image recognition; then removing the point cloud of the container indicated by the characteristic information according to the characteristic information of the container, and obtaining a grid surface of the inner surface of the container, which is indicated by the characteristic information and contains the actual size of the meal, of the container; and obtaining the boundary line between the inner surface of the container and the meal according to the point cloud of the container and the point cloud of the meal.
8. The system according to claim 7, wherein a plurality of coils coaxial with the circular container are provided in the three-dimensional information image based on the point cloud of the container and the point cloud of the meal, the coils are in contact with the point cloud of the container and the point cloud of the meal, the number of both the container point and the meal point on each coil is counted, and the coil corresponding to the same number of both is used as the boundary line.
9. A computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the steps of the method for obtaining a global spatial profile of a food according to any one of claims 1 to 4.
10. A meal nutrition assessment terminal intelligent device is characterized by comprising a controller, a camera and a depth camera, wherein the camera and the depth camera are used for collecting meal images under the control of the controller;
the controller comprises the food overall spatial profile acquisition system of any one of claims 5 to 8, or comprises the computer-readable storage medium of claim 9 having a computer program stored thereon.
CN202210189618.1A 2022-02-28 2022-02-28 Method, system and equipment for acquiring overall space contour of food Pending CN114549854A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210189618.1A CN114549854A (en) 2022-02-28 2022-02-28 Method, system and equipment for acquiring overall space contour of food

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210189618.1A CN114549854A (en) 2022-02-28 2022-02-28 Method, system and equipment for acquiring overall space contour of food

Publications (1)

Publication Number Publication Date
CN114549854A true CN114549854A (en) 2022-05-27

Family

ID=81660996

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210189618.1A Pending CN114549854A (en) 2022-02-28 2022-02-28 Method, system and equipment for acquiring overall space contour of food

Country Status (1)

Country Link
CN (1) CN114549854A (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115868814A (en) * 2023-03-02 2023-03-31 济南野风酥食品有限公司 Intelligent pancake machine regulation and control method and system with visual perception function

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115868814A (en) * 2023-03-02 2023-03-31 济南野风酥食品有限公司 Intelligent pancake machine regulation and control method and system with visual perception function
CN115868814B (en) * 2023-03-02 2023-05-09 济南野风酥食品有限公司 Intelligent regulation and control method and system for pancake machine with visual perception

Similar Documents

Publication Publication Date Title
US11055905B2 (en) Visually augmenting images of three-dimensional containers with virtual elements
CN110019896A (en) A kind of image search method, device and electronic equipment
CN108710916B (en) Picture classification method and device
CN105389581B (en) A kind of rice germ plumule integrity degree intelligent identifying system and its recognition methods
CN105654066A (en) Vehicle identification method and device
CN110363803B (en) Target detection method and system combining depth map slice and neural network
CN111161295B (en) Dish image background stripping method
CN109410209B (en) Method for detecting exogenous foreign matters in nuts based on deep learning classification
EP3465610B1 (en) Biological object detection
CN108596187A (en) Commodity degree of purity detection method and showcase
Russ Image analysis of foods
Dubey Fruit defect detection based on speeded up robust feature technique
Chen et al. A practical solution for ripe tomato recognition and localisation
CN110097537A (en) A kind of meat quantitative analysis evaluation method based on three-D grain feature
CN114549854A (en) Method, system and equipment for acquiring overall space contour of food
CN115131783A (en) User diet nutrient component information autonomous perception method based on machine vision
CN114566254A (en) Non-contact computer vision intelligent diet nutrition assessment method, system and equipment
CN115752683A (en) Weight estimation method, system and terminal based on depth camera
CN114549787A (en) Method, system and equipment for calculating volume of solid-liquid mixed food
CN111325084A (en) Dish information identification method and terminal based on YOLO neural network
CN116012393A (en) Carton point cloud segmentation method, device and processing equipment
CN116630538A (en) Dining table scene-oriented simulation data set generation method and dining table scene-oriented simulation data set generation device
CN105320955A (en) Meal quality management detection module and meal quality management detection method applying same
Megalingam et al. Color and Contour Based Identification of Stem of Coconut Bunch
Dibet Garcia Gonzalez et al. Automated vision system for cutting fixed-weight or fixed-length frozen fish portions

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination