CN114565662A - Method, system and equipment for calculating solid food volume - Google Patents

Method, system and equipment for calculating solid food volume Download PDF

Info

Publication number
CN114565662A
CN114565662A CN202210192163.9A CN202210192163A CN114565662A CN 114565662 A CN114565662 A CN 114565662A CN 202210192163 A CN202210192163 A CN 202210192163A CN 114565662 A CN114565662 A CN 114565662A
Authority
CN
China
Prior art keywords
meal
container
volume
point cloud
type
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202210192163.9A
Other languages
Chinese (zh)
Inventor
王慧
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shanghai Jiaotong University School of Medicine
Original Assignee
Shanghai Jiaotong University School of Medicine
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shanghai Jiaotong University School of Medicine filed Critical Shanghai Jiaotong University School of Medicine
Priority to CN202210192163.9A priority Critical patent/CN114565662A/en
Publication of CN114565662A publication Critical patent/CN114565662A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/60—Analysis of geometric attributes
    • G06T7/62—Analysis of geometric attributes of area, perimeter, diameter or volume
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10028—Range image; Depth image; 3D point clouds
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30108—Industrial image inspection
    • G06T2207/30128—Food products

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • General Engineering & Computer Science (AREA)
  • Computing Systems (AREA)
  • Software Systems (AREA)
  • Molecular Biology (AREA)
  • Computational Linguistics (AREA)
  • Biophysics (AREA)
  • Biomedical Technology (AREA)
  • Mathematical Physics (AREA)
  • General Health & Medical Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Evolutionary Biology (AREA)
  • Geometry (AREA)
  • Medical Treatment And Welfare Office Work (AREA)

Abstract

The invention provides a method, a system and equipment for calculating the volume of solid food, comprising the following steps: acquiring a meal image, and identifying dishes of a meal according to the meal image, wherein the meal image comprises a container and the meal contained in the container; removing the point cloud of the container aiming at the meal image to obtain the point cloud of the meal; converting the point cloud of the meal to obtain a corresponding grid, and calculating the whole volume of the meal according to a grid surface corresponding to the surface of the meal; the volume of the meal is obtained by excluding the whole volume from the gaps between the foodstuffs. The method provides a corresponding volume calculation scheme for the solid food with gaps among food materials, and compared with a general calculation scheme, the calculation result is more accurate. The method can overcome the influence on the calculation of the size of the container caused by the shooting distance in the calculation process. The invention can accurately identify the boundary line between the meal and the container, and provides a good precondition for the calculation of the meal outline.

Description

Method, system and equipment for calculating solid food volume
Technical Field
The invention relates to the field of food volume calculation, in particular to a method, a system and equipment for calculating solid food volume.
Background
Patent documents CN105580052A and CN111127545A in the prior art disclose different schemes for calculating the volume of food. The meal photographing reference object consists of an inner frame, an outer frame and an inner and outer frame connecting part, so that a user can conveniently use the meal photographing reference object as the meal photographing reference object for measuring and evaluating meals; taking pictures of the reference object and food in the same frame from different angles to record meals; the diet images obtained by using the above diet photographing reference system were measured to estimate the volume of the food, and then the weight of the food was obtained from the relationship of the volume-weight of the food (density of the food). There are also methods for estimating the volume of food, such as meals, on a board using a mobile device. The system uses a camera and a light pattern projector. The images of the food with and without the projected light pattern thereon enable the calculation of a stereoscopic shape and volume, while the image segmentation and discrimination step estimates one or more food types in the images.
However, nowadays, more and more social attention is paid to reasonable diet, which is closely related to the nutrition of people during eating and the health of life. The reasonable nutrition brought by the reasonable diet can meet the requirements of human growth, development and various physiological and physical activities, namely the reasonable diet is the basis for ensuring health. And a certain proportion of food belongs to solid food, for example, food such as fried dish, and the prior art can not solve the technical problem of how to get rid of the clearance between the food.
Disclosure of Invention
In view of the defects in the prior art, the invention aims to provide a method, a system and a device for calculating the volume of solid food.
According to the invention, the calculation method of the solid food volume comprises the following steps:
a dietary point cloud obtaining step: acquiring a meal image, and identifying dishes of a meal according to the meal image, wherein the meal image comprises a container and the meal contained in the container; removing the point cloud of the container aiming at the diet image to obtain the point cloud of the diet;
calculating the whole volume: converting the point cloud of the meal to obtain a corresponding grid, and calculating the whole volume of the meal according to a grid surface corresponding to the surface of the meal;
void volume exclusion step: the volume of the meal is obtained by excluding the whole volume from the gaps between the foodstuffs.
Preferably, the method comprises the following steps:
and (3) volume correction: determining the type of the meal in the first hierarchical classification according to the identified meal, and recording the type as a first type; determining the type of the meal in a second hierarchical classification according to the identified first type of the meal, and recording the type as a second type; modifying the volume of the meal according to the first type and the second type.
Preferably, said calculating the overall volume of the meal according to the grid plane corresponding to the meal surface comprises:
step S1: magnifying the mesh surface corresponding to the meal surface to a real size;
step S2: obtaining the part of the grid surface of the inner surface of the container, which is positioned at the lower part of the boundary line, according to the boundary line between the meal and the container;
step S3: combining the mesh surface of the diet surface with the actual size and the part of the mesh surface of the inner surface of the container, which is positioned at the lower part of the boundary line, into a closed space curved surface;
step S4: and taking the volume of the curved inner space of the closed space as the volume of the meal.
Preferably, in the step of acquiring the meal point cloud, characteristic information of the container is identified and obtained according to the meal image; removing the point cloud of the container indicated by the characteristic information according to the characteristic information of the container, and obtaining a grid surface of the inner surface of the container, which is indicated by the characteristic information and contains the actual size of the meal, in the container; and obtaining the boundary line between the inner surface of the container and the meal according to the point cloud of the container and the point cloud of the meal.
According to the present invention, there is provided a solid food volume calculation system comprising:
a meal point cloud acquisition module: acquiring a meal image, and identifying dishes of a meal according to the meal image, wherein the meal image comprises a container and the meal contained in the container; removing the point cloud of the container aiming at the diet image to obtain the point cloud of the diet;
an overall volume calculation module: converting the point cloud of the meal to obtain a corresponding grid, and calculating the whole volume of the meal according to a grid surface corresponding to the surface of the meal;
void volume exclusion module: the volume of the meal is obtained by excluding the whole volume from the gaps between the foodstuffs.
Preferably, the method comprises the following steps:
a volume correction module: determining the type of the meal in the first hierarchical classification according to the identified meal, and recording the type as a first type; determining the type of the meal in a second hierarchical classification according to the identified first type of the meal, and recording the type as a second type; modifying the volume of the meal according to the first type and the second type.
Preferably, said calculating the overall volume of the meal according to the grid plane corresponding to the meal surface comprises:
module M1: enlarging the mesh surface corresponding to the meal surface to a real size;
module M2: obtaining the part of the grid surface of the inner surface of the container, which is positioned at the lower part of the boundary line, according to the boundary line between the meal and the container;
module M3: combining the mesh surface of the diet surface with the actual size and the part of the mesh surface of the inner surface of the container, which is positioned at the lower part of the boundary line, into a closed space curved surface;
module M4: and taking the volume of the curved inner space of the closed space as the volume of the meal.
Preferably, in the dietary point cloud obtaining module, characteristic information of the container is obtained according to the dietary image identification; removing the point cloud of the container indicated by the characteristic information according to the characteristic information of the container, and obtaining a grid surface of the inner surface of the container, which is indicated by the characteristic information and contains the actual size of the meal, in the container; and obtaining the boundary line between the inner surface of the container and the meal according to the point cloud of the container and the point cloud of the meal.
According to the present invention, there is provided a computer readable storage medium having stored thereon a computer program which, when being executed by a processor, carries out the steps of the method of calculating a volume of solid food.
The intelligent equipment of the dietary nutrition assessment terminal comprises a controller, a camera and a depth camera, wherein the camera and the depth camera are used for collecting dietary images under the control of the controller;
the controller comprises a calculation system of the solid food volume or comprises the computer readable storage medium storing the computer program.
Compared with the prior art, the invention has the following beneficial effects:
1. the method provides a corresponding volume calculation scheme for the solid food with gaps among food materials, and compared with a general calculation scheme, the calculation result is more accurate.
2. The method can overcome the influence on the calculation of the size of the container caused by the shooting distance in the calculation process.
3. The invention can accurately identify the boundary line between the meal and the container, and provides a good precondition for the calculation of the meal outline.
Drawings
Other features, objects and advantages of the invention will become more apparent upon reading of the detailed description of non-limiting embodiments with reference to the following drawings:
FIG. 1 is a schematic overall flow diagram of the present invention.
Fig. 2 is a flow chart of the present invention for correcting meal volume.
Fig. 3 is a schematic diagram of a two-dimensional information image of a meal and a container top view angle acquired by a camera according to the present invention.
FIG. 4 is a schematic diagram of a three-dimensional information image of a meal and a container acquired by a depth camera according to the present invention.
Fig. 5 is a schematic diagram of the calculation of meal volume according to the invention.
Detailed Description
The present invention will be described in detail with reference to specific examples. The following examples will assist those skilled in the art in further understanding the invention, but are not intended to limit the invention in any way. It should be noted that it would be obvious to those skilled in the art that various changes and modifications can be made without departing from the spirit of the invention. All falling within the scope of the present invention.
According to the invention, the calculation method of the solid food volume comprises the following steps:
a dietary point cloud obtaining step: acquiring a meal image, and identifying dishes of a meal according to the meal image, wherein the meal image comprises a container and the meal contained in the container; removing the point cloud of the container aiming at the diet image to obtain the point cloud of the diet;
and calculating the whole volume: converting the point cloud of the meal to obtain a corresponding grid, and calculating the whole volume of the meal according to a grid surface corresponding to the surface of the meal;
void volume exclusion step: excluding the overall volume from the gaps between the foodstuffs to obtain the volume of the meal; for dishes with solid attributes, calculating the overall volume of the meal according to a grid surface corresponding to the surface of the meal, and removing gaps among the foods from the overall volume to obtain the volume of the meal; for example, in a dish spiced salt bar, the bar is long, the bar and the bar are not regularly arranged in sequence but are overlapped and stacked, a large gap is formed between the bar and the bar, if the gap is also counted into the volume of a meal and participates in the calculation and evaluation of nutrients, the result of the evaluation is inaccurate, and therefore the gap needs to be eliminated;
and (3) volume correction: determining the type of the meal in the first hierarchical classification according to the identified meal, and recording the type as a first type; determining the type of the meal in a second hierarchical classification according to the identified first type of the meal, and recording the type as a second type; modifying the volume of the meal according to the first type and the second type. For example, different second types under the same first type correspond to different correction coefficients. The correction factor is multiplied by the volume of the meal.
In particular, the first hierarchical classification distinguishes meals with main ingredients, the second hierarchical classification distinguishes meals with ingredients, and the second hierarchical classification is a sub-classification of the first hierarchical classification. Shooting a meal image through a depth camera, wherein the meal image contains meals and containers containing the meals; identifying the diet image to obtain a container, and further identifying to obtain a diet based on the container; identifying the type of the meal in the first hierarchical classification through the trained first neural network; the training samples of the first neural network comprise diet image samples for distinguishing meals by different main materials; identifying the type of the meal in the second hierarchical classification through the trained second neural network; wherein each meal of the first type has a plurality of different sub-classifications of the meal of the second type. According to the invention, two types of samples are prepared to train the first neural network and the second neural network respectively, so that the interference of ingredients on identification is reduced when the first hierarchical classification is identified, and the ingredients are more specifically distinguished when the second hierarchical classification is identified.
More specifically, the major ingredients of the cuisine are generally the same and the ingredients are generally different from one another in different places. For example, the main materials of the eggs fried by tomatoes are tomatoes and eggs, but the ingredients of the eggs fried by the Xinjiang tomatoes are onion granules and more edible oil is used, and the ingredients of the eggs fried by the Shanghai tomatoes are onion sections and are less and light. For example, the main materials of the fried noodles are all noodles, but the ingredients of the Xinjiang fried noodles are onion and shredded beef, a small amount of light soy sauce, and the ingredients of the Shanghai fried noodles are mung bean sprouts and shredded pork, a large amount of light soy sauce and dark soy sauce. In addition to different nutrient contents, different food materials of different cuisine have different shapes, so that gaps among the food materials are different. It is necessary to subdivide the food.
For example, the fried noodles are fried in the same way, the fried noodles in the northwest region are wide, the fried noodles in the south of the Yangtze river are thin, and the difference of the two types of noodles causes the difference of the gaps between the noodles.
For example, the egg-fried rice in Shanghai has no pepper, but the egg-fried rice in Hunan has pepper, so that the gap between the two is different.
In a preferred example, the meal image includes: the two-dimensional information image of the top view angle photographed by the camera and the three-dimensional information image photographed by the depth camera are respectively the two-dimensional information image shown in fig. 3 and the three-dimensional information image shown in fig. 4. And identifying dishes and containers of the meal through the trained neural network according to the two-dimensional information image. During the training process, images of combinations of different dishes and containers are prepared as samples, so that the neural network can identify the corresponding dishes and containers. The dish of the meal is the name of the meal, namely the name of the dish, so as to distinguish different meals. For example, the protein content of fried green pepper is less than that of the shredded pork under the same volume, so that different meals need to be distinguished to obtain the nutrient content of the meal according to the volume of the meal by using known information. The attributes of the meal comprise solid, solid-liquid mixed and liquid, so that the calculation of the meal volume is more practical.
For example, the dish can be tomato fried egg, green pepper fried shredded pork, potato stewed beef, kidney bean stewed small steak, bean curd fungus mushroom soup, preserved egg lean meat porridge. In the dishes, the attributes of the fried eggs of the tomatoes and the fried shredded meat of the green peppers are classified into solid foods, the solid foods are mainly solid food materials, the soup is less, and the surface of the soup is invisible under the overlooking view because of being shielded by the solid food materials; classifying attributes of stewed beef and stewed small rows with potatoes into solid-liquid mixed food, wherein solid food materials are mainly used in the solid-liquid mixed food, more soup is available, the surface of the soup is visible under a overlooking visual angle, and the surface area of the soup is smaller than that of the solid food materials; attributes of the bean curd fungus mushroom soup and the preserved egg and lean meat porridge are classified into liquid foods, wherein the liquid foods mainly comprise fluid foods and solid foods, the fluid surfaces are visible under a overlooking visual angle, and the fluid surface areas are larger than the solid food material surface areas. For meals such as solid-type fried eggs and fried shredded meat with green pepper, gaps exist among different food materials and among the same food materials, and the calculation accuracy can be improved by excluding the gaps.
The container containing the meal is placed on the bearing plane, the distance between the bearing plane and the camera is known, the distance between the bearing plane and the depth camera is also known, and the size information such as the diameter of the container and the like and the three-dimensional model information such as the surface shape and the like are also known, so that after the two-dimensional information image is obtained, the scaling converted into the actual size of the meal can be obtained based on the size of the meal in the two-dimensional information image through the ratio between the diameter size of the container in the two-dimensional information image and the actual diameter size of the container. In a preferred embodiment, the distance between the bearing plane and the camera is a fixed value, and the distance between the bearing plane and the depth camera is a fixed value, and the bearing plane, the camera and the depth camera are assembled according to the fixed values when being assembled. The two-dimensional information image is a color image, and a meal, a container and a bearing plane can be seen. The three-dimensional information image can show meals, containers and bearing planes. Correspondingly, the learning sample of the deep learning neural network for identifying the meal and the container also contains the meal, the container and the bearing plane. The three-dimensional information image records point cloud data, and point clouds of the container and the diet are obtained according to the three-dimensional information image. Specifically, under a overlooking view angle, the point cloud corresponding to an area on the surface of the container which is not covered by the meal and the point cloud corresponding to the surface of the meal. The shape of the meal bottom is limited by the inner surface of the container for containing meals, the shape contour of the inner surface can be used as the shape contour of the meal bottom, and the shape of the meal top can be changed by the operation of a dish holder, for example, even if the meals of the same dish are provided, if the dish amount is different, the heights of the meal tops are different, and for example, under the same dish amount, the dish holder is used for laying the meals evenly or forming the meal tops into a pyramid shape in a stacking mode, the shapes of the meal tops are also different. Therefore, the acquisition of a point cloud corresponding to the top of the meal by the depth camera to reflect its shape facilitates accurate assessment of subsequent nutrients.
Removing the point cloud of the container according to the diet image and the known information to obtain the point cloud of the diet; wherein the known information comprises a three-dimensional model of the container. In order to obtain the shape of the top of the meal, the point cloud of the meal needs to be distinguished from the point cloud of the container. The information of the container is known, and since the distance between the depth camera and the container and its carrier tile is also known, the size of the container in the three-dimensional information image and the size of the known three-dimensional model of the container can be adjusted to a uniform size scale. And overlapping the point cloud in the three-dimensional information image and the container three-dimensional model under the condition of uniform size scaling, wherein the outer edge of the container, such as the opening edge of the plate, is higher than the bearing plane, so that the outer edge of the container in the three-dimensional information image is obtained by identification, the outer edge of the container three-dimensional model is overlapped with the outer edge of the container in the three-dimensional information image, the point cloud overlapped with the container three-dimensional model is identified as the point cloud of the container, and the point cloud in the container point cloud is identified as the point cloud of diet. In a variant, based on the known color of the container, the point cloud of the container and the point cloud of the meal may also be identified by color. In order to facilitate the identification of the container, identification information, such as a two-dimensional code, may be provided on the surface of the container.
Still more particularly, said calculating the overall volume of the meal from a grid plane corresponding to the meal surface comprises:
step S1: enlarging the mesh surface corresponding to the meal surface to a real size; from the known three-dimensional model of the distance between the carrier plane and the depth camera, the known height, diameter, etc. dimensions of the container, a scaling can be obtained, as described above, to enlarge the mesh plane corresponding to the meal surface to the actual dimensions. Step S2: and obtaining the part of the grid surface of the inner surface of the container, which is positioned at the lower part of the boundary line, according to the boundary line. Specifically, in order to obtain the boundary line, characteristic information of the container is obtained according to the meal image identification, wherein the characteristic information can be a two-dimensional code for example, and the two-dimensional code is used for indicating only one container or indicating only one container type with the same specification; then removing the point cloud of the container indicated by the characteristic information according to the characteristic information of the container, and obtaining a grid surface of the inner surface of the container, which is indicated by the characteristic information and contains the actual size of the meal, of the container; and obtaining the boundary line between the inner surface of the container and the meal, namely the boundary line between the inner surface of the container and the grid surface of the meal surface according to the point cloud of the container and the point cloud of the meal. In a preferred embodiment, based on the point cloud of the container and the point cloud of the meal, a plurality of coils coaxial with the circular container are arranged in the three-dimensional information image, the coils are in contact with the point cloud of the container and the point cloud of the meal, the number of the points of the container and the number of the points of the meal on each coil are counted, and the coils corresponding to the same number of the points of the container and the number of the points of the meal on each coil are taken as the boundary line. Step S3: combining the mesh surface of the diet surface with the actual size and the part of the mesh surface of the inner surface of the container, which is positioned at the lower part of the boundary line, into a closed space curved surface; as shown in fig. 5, fig. 5 shows the mesh surface 100 of the full-size meal surface, the mesh surface 200 of the container inner surface, the portion 300 of the container inner surface that is located below the boundary line, the boundary line 400, and the container 500. Step S4: and taking the volume of the curved inner space of the closed space as the volume of the meal. In the step of acquiring the meal point cloud, identifying and obtaining the characteristic information of the container according to the meal image; removing the point cloud of the container indicated by the characteristic information according to the characteristic information of the container, and obtaining a grid surface of the inner surface of the container, which is indicated by the characteristic information and contains the actual size of the meal, in the container; and obtaining the boundary line between the inner surface of the container and the meal according to the point cloud of the container and the point cloud of the meal.
The method provides a corresponding volume calculation scheme for the solid food with gaps among food materials, and compared with a general calculation scheme, the calculation result is more accurate. The method can overcome the influence on the calculation of the size of the container caused by the shooting distance in the calculation process. The invention can accurately identify the boundary line between the meal and the container, and provides a good precondition for the calculation of the meal outline.
According to the present invention, there is provided a solid food volume calculation system comprising:
a meal point cloud acquisition module: acquiring a meal image, and identifying dishes of a meal according to the meal image, wherein the meal image comprises a container and the meal contained in the container; removing the point cloud of the container aiming at the diet image to obtain the point cloud of the diet;
an integral volume calculation module: converting the point cloud of the meal to obtain a corresponding grid, and calculating the whole volume of the meal according to a grid surface corresponding to the surface of the meal;
void volume exclusion module: the volume of the meal is obtained by excluding the whole volume from the gaps between the foodstuffs.
Preferably, the method comprises the following steps:
a volume correction module: determining the type of the meal in the first hierarchical classification according to the identified meal, and recording the type as a first type; determining the type of the meal in a second hierarchical classification according to the identified first type of the meal, and recording the type as a second type; modifying the volume of the meal according to the first type and the second type.
Preferably, said calculating the overall volume of the meal according to the grid plane corresponding to the meal surface comprises:
module M1: enlarging the mesh surface corresponding to the meal surface to a real size;
module M2: obtaining the part of the grid surface of the inner surface of the container, which is positioned at the lower part of the boundary line, according to the boundary line between the meal and the container;
module M3: combining the mesh surface of the diet surface with the actual size and the part of the mesh surface of the inner surface of the container, which is positioned at the lower part of the boundary line, into a closed space curved surface;
module M4: and taking the volume of the curved inner space of the closed space as the volume of the meal.
Preferably, in the dietary point cloud obtaining module, characteristic information of the container is obtained according to the dietary image identification; removing the point cloud of the container indicated by the characteristic information according to the characteristic information of the container, and obtaining a grid surface of the inner surface of the container, which is indicated by the characteristic information and contains the actual size of the meal, in the container; and obtaining the boundary line between the inner surface of the container and the meal according to the point cloud of the container and the point cloud of the meal.
According to the present invention, there is provided a computer readable storage medium having stored thereon a computer program which, when being executed by a processor, carries out the steps of the method of calculating a volume of solid food.
The intelligent equipment of the dietary nutrition assessment terminal comprises a controller, a camera and a depth camera, wherein the camera and the depth camera are used for collecting dietary images under the control of the controller;
the controller comprises a calculation system of the solid food volume or comprises the computer readable storage medium storing the computer program.
Those skilled in the art will appreciate that, in addition to implementing the systems, apparatus, and various modules thereof provided by the present invention in purely computer readable program code, the same procedures can be implemented entirely by logically programming method steps such that the systems, apparatus, and various modules thereof are provided in the form of logic gates, switches, application specific integrated circuits, programmable logic controllers, embedded microcontrollers and the like. Therefore, the system, the device and the modules thereof provided by the present invention can be considered as a hardware component, and the modules included in the system, the device and the modules thereof for implementing various programs can also be considered as structures in the hardware component; modules for performing various functions may also be considered to be both software programs for performing the methods and structures within hardware components.
The foregoing description of specific embodiments of the present invention has been presented. It is to be understood that the present invention is not limited to the specific embodiments described above, and that various changes or modifications may be made by one skilled in the art within the scope of the appended claims without departing from the spirit of the invention. The embodiments and features of the embodiments of the present application may be combined with each other arbitrarily without conflict.

Claims (10)

1. A method of calculating a volume of solid food, comprising:
a dietary point cloud obtaining step: acquiring a meal image, and identifying dishes of a meal according to the meal image, wherein the meal image comprises a container and the meal contained in the container; removing the point cloud of the container aiming at the diet image to obtain the point cloud of the diet;
calculating the whole volume: converting the point cloud of the meal to obtain a corresponding grid, and calculating the whole volume of the meal according to a grid surface corresponding to the surface of the meal;
void volume exclusion step: the volume of the meal is obtained by excluding the whole volume from the gaps between the foodstuffs.
2. The method of calculating the volume of solid food according to claim 1, comprising:
and (3) volume correction: determining the type of the meal in the first hierarchical classification according to the identified meal, and recording the type as a first type; determining the type of the meal in a second hierarchical classification according to the identified first type of the meal, and recording the type as a second type; modifying the volume of the meal according to the first type and the second type;
the first hierarchical classification distinguishes meals with main ingredients, the second hierarchical classification distinguishes meals with ingredients, and the second hierarchical classification is a sub-classification of the first hierarchical classification.
3. The method of calculating solid food volume according to claim 1, wherein said calculating the overall volume of the meal from the grid plane corresponding to the meal surface comprises:
step S1: enlarging the mesh surface corresponding to the meal surface to a real size;
step S2: obtaining the part of the grid surface of the inner surface of the container, which is positioned at the lower part of the boundary line, according to the boundary line between the meal and the container;
step S3: combining the mesh surface of the diet surface with the actual size and the part of the mesh surface of the inner surface of the container, which is positioned at the lower part of the boundary line, into a closed space curved surface;
step S4: and taking the volume of the curved inner space of the closed space as the volume of the meal.
4. The method for calculating the volume of solid food according to claim 1, wherein in the step of acquiring the meal point cloud, characteristic information of a container is identified and obtained according to the meal image; removing the point cloud of the container indicated by the characteristic information according to the characteristic information of the container, and obtaining a grid surface of the inner surface of the container with the actual size for containing the meal in the container indicated by the characteristic information; and obtaining the boundary line between the inner surface of the container and the meal according to the point cloud of the container and the point cloud of the meal.
5. A system for calculating a volume of solid food, comprising:
a meal point cloud acquisition module: acquiring a meal image, and identifying dishes of a meal according to the meal image, wherein the meal image comprises a container and the meal contained in the container; removing the point cloud of the container aiming at the diet image to obtain the point cloud of the diet;
an overall volume calculation module: converting the point cloud of the meal to obtain a corresponding grid, and calculating the whole volume of the meal according to a grid surface corresponding to the surface of the meal;
void volume exclusion module: the volume of the meal is obtained by excluding the whole volume from the gaps between the foodstuffs.
6. The solid food volume calculation system of claim 5, comprising:
a volume correction module: determining the type of the meal in the first hierarchical classification according to the identified meal, and recording the type as a first type; determining the type of the meal in a second hierarchical classification according to the identified first type of the meal, and recording the type as a second type; modifying the volume of the meal according to the first type and the second type;
the first hierarchical classification distinguishes meals with main ingredients, the second hierarchical classification distinguishes meals with ingredients, and the second hierarchical classification is a sub-classification of the first hierarchical classification.
7. The system for calculating solid food volume according to claim 5, wherein said calculating the overall volume of the meal according to the grid plane corresponding to the meal surface comprises:
module M1: enlarging the mesh surface corresponding to the meal surface to a real size;
module M2: obtaining the part of the grid surface of the inner surface of the container, which is positioned at the lower part of the boundary line, according to the boundary line between the meal and the container;
module M3: combining the mesh surface of the diet surface with the actual size and the part of the mesh surface of the inner surface of the container, which is positioned at the lower part of the boundary line, into a closed space curved surface;
module M4: and taking the volume of the curved inner space of the closed space as the volume of the meal.
8. The system for calculating the volume of solid food according to claim 5, wherein in the meal point cloud obtaining module, characteristic information of a container is obtained according to the meal image identification; removing the point cloud of the container indicated by the characteristic information according to the characteristic information of the container, and obtaining a grid surface of the inner surface of the container, which is indicated by the characteristic information and contains the actual size of the meal, in the container; and obtaining the boundary line between the inner surface of the container and the meal according to the point cloud of the container and the point cloud of the meal.
9. A computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the steps of the method of calculating a solid food volume of any one of claims 1 to 4.
10. A meal nutrition assessment terminal intelligent device is characterized by comprising a controller, a camera and a depth camera, wherein the camera and the depth camera are used for collecting meal images under the control of the controller;
the controller comprises a solid food volume calculation system of any one of claims 5 to 8, or a computer readable storage medium of claim 9 having a computer program stored thereon.
CN202210192163.9A 2022-02-28 2022-02-28 Method, system and equipment for calculating solid food volume Pending CN114565662A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210192163.9A CN114565662A (en) 2022-02-28 2022-02-28 Method, system and equipment for calculating solid food volume

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210192163.9A CN114565662A (en) 2022-02-28 2022-02-28 Method, system and equipment for calculating solid food volume

Publications (1)

Publication Number Publication Date
CN114565662A true CN114565662A (en) 2022-05-31

Family

ID=81716109

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210192163.9A Pending CN114565662A (en) 2022-02-28 2022-02-28 Method, system and equipment for calculating solid food volume

Country Status (1)

Country Link
CN (1) CN114565662A (en)

Similar Documents

Publication Publication Date Title
Zhang et al. Deep learning in food category recognition
Jia et al. Accuracy of food portion size estimation from digital pictures acquired by a chest-worn camera
He et al. DietCam: multiview food recognition using a multikernel SVM
CN107873101B (en) Imaging system for object recognition and evaluation
Pouladzadeh et al. Measuring calorie and nutrition from food image
US9734426B2 (en) Automated food recognition and nutritional estimation with a personal mobile electronic device
Kitamura et al. Food log by analyzing food images
Kitamura et al. Foodlog: Capture, analysis and retrieval of personal food images via web
Pouladzadeh et al. Using graph cut segmentation for food calorie measurement
US9424495B1 (en) Digital food imaging analysis: system and method to analyze food consumption
Bruno et al. A survey on automated food monitoring and dietary management systems
Kitamura et al. Image processing based approach to food balance analysis for personal food logging
Akpro Hippocrate et al. Food weight estimation using smartphone and cutlery
JP2008204105A (en) Automatic food intake measuring system and automatic dietary intake measuring method
CN112053428A (en) Method and device for identifying nutritional information contained in food
CN110020609A (en) A kind of method, system and storage medium for analyzing food
McClure Computer vision sorting of potatoes
Pouladzadeh et al. Intelligent SVM based food intake measurement system
Lin et al. A study of automatic judgment of food color and cooking conditions with artificial intelligence technology
Konstantakopoulos et al. 3D reconstruction and volume estimation of food using stereo vision techniques
Beltran et al. Reliability and validity of food portion size estimation from images using manual flexible digital virtual meshes
CN114565662A (en) Method, system and equipment for calculating solid food volume
CN114549787A (en) Method, system and equipment for calculating volume of solid-liquid mixed food
CN114549854A (en) Method, system and equipment for acquiring overall space contour of food
Shermila et al. Estimation of protein from the images of health drink powders

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination