JP2018049584A - Meal size estimation program, meal size estimation method, and meal size estimation apparatus - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a health management server capable of creating an advice message further suited to a client, while performing standardization and improving efficiency in creating an advice message of a specialist.SOLUTION: A meal size estimation program according to the present invention causes a computer to be functioned as: a storage section for storing an index image dataset in which an index image and size information are associated, a material image dataset in which material images and names of materials are associated, and a nutrition value dataset in which materials and nutrition values of the materials are associated; an acquisition section for acquiring a diet image of a user; a specification section for specifying index images and material images included in the diet image; an estimation section for estimating an amount of the materials associated with the specified material images on the basis of the size information associated with the specified index images; and a correction section for correcting a nutrition value associated with the materials on the basis of the estimated amount of the materials and calculating a nutrition value after the correction.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a meal amount estimation program, a meal amount estimation method, and a meal amount estimation apparatus.

  In recent years, health guidance services that receive expert guidance have become widespread in response to increasing health awareness such as diet and prevention of metabolic syndrome. In addition, with the shift to IT, there are increasing cases of receiving health guidance services on-line without direct contact between experts and clients.

As such a health guidance service, a technique has been proposed in which a menu that takes into account calories consumed is provided, and calories ingested are easily obtained and recorded without any calorie notation (for example, Patent Document 1). reference).
Techniques for providing are disclosed.

JP 2006-252150 A

  However, the invention described in Patent Document 1 cannot calculate the true meal amount.

  Then, an object of this invention is to provide the technique which can estimate a meal amount.

According to the present invention, as the first meal amount estimation program,
An index image data set in which the index image and the size information are associated; a material image data set in which the material image and the material name are associated; and a nutrition value dataset in which the material and the nutrition value of the material are associated. A storage unit for storing,
An acquisition unit for acquiring a meal image of the user;
A specifying unit for specifying the index image and the material image included in the meal image;
An estimation unit that estimates an amount of the material associated with the identified material image based on the size information associated with the identified index image;
Based on the estimated amount of the material, a meal amount estimation program for correcting the nutrition value associated with the material and causing the computer to function as a correction unit that calculates the corrected nutrition value is obtained.

According to the present invention, as the second meal amount estimation program, the meal amount estimation program according to the first aspect,
The index image data set further includes error information associated with the size information,
The identifying unit identifies an index image with less error when there are a plurality of the index images included in the meal image;
Meal amount estimation program is obtained.

According to the present invention, as the third meal amount estimation program, the meal amount estimation program according to the first or second,
Causing the computer to further function as an index receiving unit that receives an operation for selecting an object existing in the acquired meal image and an operation for inputting the size information of the object, from a user;
The estimation unit estimates the amount of the material associated with the identified material image based on the input size information of the selected object.
Meal amount estimation program is obtained.

According to the present invention, as the fourth meal amount estimation program, the meal amount estimation method according to any one of the first to third,
The acquisition unit acquires from the user a first meal image and a second meal image that is an image after ingestion of the first meal image;
A difference for obtaining a difference between a first corrected nutritional value calculated based on the first meal image by the correction unit and a second corrected nutritional value calculated based on the second meal image. A meal amount estimation program for causing a computer to function as the calculation unit is obtained.

According to the present invention, as the fifth meal amount estimation program, the meal amount estimation program according to the fourth aspect,
The acquisition unit specifies and acquires the first meal image and the second meal image based on time information given to the meal image of the user.
Meal amount estimation program is obtained.

According to the present invention,
An index image data set in which the index image and the size information are associated; a material image data set in which the material image and the material name are associated; and a nutrition value dataset in which the material and the nutrition value of the material are associated. The step of memorizing
Obtaining a meal image of the user;
Identifying the index image and the material image included in the meal image;
Estimating the amount of the material associated with the identified material image based on the size information associated with the identified index image;
Correcting the nutrition value associated with the material based on the estimated amount of the material, and calculating a corrected nutrition value.

According to the present invention,
An index image data set in which the index image and the size information are associated; a material image data set in which the material image and the material name are associated; and a nutrition value dataset in which the material and the nutrition value of the material are associated. Means for storing
Means for obtaining a user's meal image;
Means for identifying the index image and the material image included in the meal image;
Means for estimating the amount of the material associated with the identified material image based on the size information associated with the identified index image;
Based on the estimated amount of the material, there is obtained a meal amount estimation device comprising: means for correcting the nutrition value associated with the material and calculating a corrected nutrition value.

  According to the present invention, it is possible to estimate the actual amount of meal ingested by the user.

FIG. 1 is a diagram illustrating an example of a user terminal screen according to an embodiment of a meal amount estimation program. FIG. 2 is a block diagram showing a functional configuration of the user terminal of FIG. FIG. 3 is a diagram illustrating a flow of the meal image analysis process. It is a female figure with the state when each material is recognized from the meal image by this Embodiment. It is a schematic diagram which shows the example of the parameter | index of the meal amount estimation program by this Embodiment. It is a conceptual diagram of the method of estimating the quantity of a raw material using the parameter | index of FIG. In the meal amount estimation program by this Embodiment, it is a conceptual diagram of the data table in case it has the nutritional value according to the quantity of material amount previously. It is a conceptual diagram of a data table in which an index, size information, and an error range are associated with each other. It is a conceptual diagram of the data table with which the raw material and the nutritional value were linked | related. It is a figure which shows the state which has recognized each raw material contained in the meal image of the state after a meal used for the 2nd Embodiment of this invention.

(First embodiment)
<Overview>
Hereinafter, a first embodiment of the present invention will be described with reference to the drawings. As shown in FIG. 1, the meal amount estimation program of the present invention is based on a meal photograph (meal image) taken by a user using a camera of the user terminal 10 or the like (a white rice, Fish, miso soup, etc.) for estimating the nutritional value. More specifically, the amount of each material (usually, large, half, etc.) is estimated based on an index with small individual differences, which contributes to more accurate nutritional value calculation.

<Configuration>
The meal amount estimation program of the present embodiment is implemented using the information processing apparatus (user terminal) 10. As shown in FIG. 2, the user terminal 10 includes a processing unit 11, a main memory 12, an external memory 13, an interface 14, a communication unit 15, a camera unit 16, and the like, which are electrically connected to each other via a bus. Connected. The processing unit 11 loads the program module stored in the external memory 13 and various data stored in the camera unit 16 into the main memory 12 and sequentially executes the program to execute processing by the meal amount estimation program. Control. In particular, the processing unit 11 calculates a material, an amount, and a nutritional value based on a meal photograph acquired via the camera unit 16, and estimates a nutritional value of the material actually ingested by the user. The interface 14 receives user operations such as a touch panel and a touch keyboard.

  Note that all or part of the information processing of the user terminal 10 may be distributed to one or more processes using other terminals (such as a server and a cloud computer) connected via a network. In this case, the information processing apparatus is configured by the user terminal 10 and other terminals as a whole.

<Operation>
The meal amount estimation according to the present embodiment is divided into two stages, a learning step and an identification step, and the identification step can be further divided into three steps: a meal content identification step, a material amount identification step, and a nutritional value calculation step. .

  In the learning step, a material discriminator for estimating whether or not a specific material is included in the photograph is learned for each material. As a flow of the learning process, it is determined whether or not a specific material is included in the meal image. Then, a material classifier is learned by a Support Vector Machine (SVM) using the obtained feature value using the presence or absence of the material name as a label. This process is performed for each material, and a material classifier for each material is constructed.

  The SVM according to the present embodiment is a pattern recognition technique based on supervised learning for two-class discrimination, and checks whether a material to be identified is included in a material list extracted from a large number of meal images. From the result of the examination, whether or not the material is included is used as a material label. Using the feature amount and label calculated from the meal image as learning data, a classifier for each material included in the meal image is constructed by SVM.

  In the identification step, it is determined whether or not each material is included in the meal image using the classifier constructed by learning, the feature amount is calculated from the meal image, and each material in the meal image is calculated using the identifier. Steps to identify meal content to be identified, steps to extract the presence or absence of an index (details will be described later) contained in each material, and if there is an index, refer to the material amount identifying step to calculate the size of the whole material, and the nutrition value table Then, the process proceeds in the order of the nutritional value calculation step of specifying the nutritional value of the specified material and correcting the nutritional value in consideration of the size of the material.

  Hereinafter, the identification step will be described in more detail. As illustrated in FIG. 3, the user terminal 10 reads a meal photograph (hereinafter, referred to as a “cooking image”) taken by the camera unit 16 (step S <b> 301). The feature amount is calculated from the read meal image and each material is specified. At this time, as shown in FIG. 4, when a plurality of materials are present in one meal image, an area is automatically (or manually) specified by a bounding box for each material. Also good.

  In FIG. 4, a box 102 that identifies rice, a box 104 that identifies boiled vegetables, a box 106 that identifies fish, a box 108 that identifies miso soup, and 110 that identifies elbow sauce are displayed. At this time, the name of the specified material is displayed in the vicinity of the box, and if it is incorrect, it may be corrected by the user.

  Subsequently, an index image is specified from each material of the meal image (step S303). As the index in the present embodiment, an index having a small individual difference can be applied. In this embodiment, “rice grain” is used. When the index image is specified, the size of each material is specified using the index image (step S304).

  In the present embodiment, as shown in FIG. 6, the size of rice (raw material) is calculated using a standard rice grain diameter of 5 mm. In the present embodiment, the diameter (maximum length) of the material size is classified in advance for materials that can be conventionally sized, such as “normal”, “large”, and “small” like rice. Yes. Specifically, as shown in FIG. 7, when the diameter is 100 to 130 mm, it is recognized as a normal size rice (ordinary serving), and when the diameter is 131 to 150 mm, it is recognized as a large serving rice (large serving). However, when the diameter is 80 to 99 mm, it is recognized as a small-sized rice (small bowl).

  The data table shown in FIG. 7 classifies the size of the material based on the diameter size, but it is also possible to calculate the area of the material using an index and recognize the size of the material according to the area. Good.

  In the present embodiment, the size of all materials is calculated using rice grains as an index, and this is used as the amount of the material. However, when there is no rice in the meal image, other sizes included in the meal image are included. An index may be used. For example, an index existing in a meal image may be specified with reference to an index table as shown in FIG.

  The index table shown in FIG. 8 includes an index, an index size, and an error. In the present embodiment, when there are a plurality of index images registered in the index table, an index with a small error is selected as an index. For example, when both udon noodles and chopsticks are included in the meal image, the size of the material is calculated using the width (6 mm) of the udon with less error.

  Further, when there is no index registered in the index table in the meal image, any object in the meal image by the user (preferably an approximate size such as the size of the food or the size of the dish is preferable) And the actual size may be registered. For example, it is good also as image | photographing together with auxiliary indicators, such as a finger | toe, a coin, and a scale. In this case, the size information of the auxiliary index may be registered in advance or afterwards.

  Next, the nutrition value of each identified material is identified by reading the nutrition information stored in the nutrition value database (step S305). In the nutrition information, various nutrients such as calorific value, singularization amount, lipid, protein, vitamin group, mineral and the like are registered for each material. The recognized value of the recognized material is obtained by referring to the database.

  Subsequently, the nutritional value of the material is corrected based on the amount of the identified material. For example, the nutritional value is corrected by associating standard size information (that is, information on the size / area of the material used when the nutritional value is acquired) with the nutritional value database in advance. Then, a predetermined coefficient is calculated based on the comparison with the size, and each value in the nutritional value database is multiplied by the coefficient. For example, referring to a data set in which standard size information (120 mm) and standard nutritional value at the size are associated with each other for fish (carp) fillets, according to actual size information (90 mm) of the fillets in the meal image The nutritional value corrected by multiplying the standard nutritional value by a predetermined coefficient may be adopted as the actual nutritional value.

  The correction method is not limited to this, and it may be compared with a standard area. In this way, the corrected nutritional value and the total sum (corrected total nutritional value) of each material are displayed on the screen of the user terminal 10 (see FIG. 1).

  According to the first embodiment of the present invention described above, the size of the index varies depending on the distance between the meal and the camera when the meal image is taken (that is, the perspective of the meal image). The amount of material contained can be estimated appropriately.

  The meal image may be acquired by the camera unit or the like of the user terminal, or the user terminal or the cloud / electrically connected to the user terminal (when taken by another camera or the like). The meal image stored in the local storage may be acquired directly or via a network or the like.

(Second Embodiment)
The meal amount estimation program according to the second embodiment of the present invention relates to a program for estimating an actual intake amount from two meal images before and after a meal. Since the learning step and the identification step in the present embodiment are almost the same as those in the first embodiment described above, only different parts will be described and description of common parts will be omitted. In the present embodiment, the nutritional value described above is calculated for each of the pre-meal meal image and the post-meal meal image, and the nutrition estimated based on the post-meal meal image from the nutritional value estimated based on the pre-meal meal image Estimate the actual nutritional value by subtracting the value.

  In the post-meal meal image identification step, as shown in FIG. 10, only the uneaten material is specified. That is, a bounding box 104 ′ for recognizing the remaining boiled vegetables, a bounding box 106 ′ for recognizing the remaining fish, and a bounding box 110 ′ for recognizing the remaining elbows are displayed.

  In the present embodiment, a meal image before a meal and a meal image after a meal may be automatically selected based on a time stamp that is stored in association with the image when the image is taken by the camera unit. As a result, the user can easily know the nutritional value ingested simply by taking pictures of the beginning and end of eating.

DESCRIPTION OF SYMBOLS 10 User terminal 11 Processing part 12 Main memory 13 External memory 14 Interface 15 Communication part 16 Camera part

Claims (7)

An index image data set in which the index image and the size information are associated; a material image data set in which the material image and the material name are associated; and a nutrition value dataset in which the material and the nutrition value of the material are associated. A storage unit for storing,
An acquisition unit for acquiring a meal image of the user;
A specifying unit for specifying the index image and the material image included in the meal image;
An estimation unit that estimates an amount of the material associated with the identified material image based on the size information associated with the identified index image;
A meal amount estimation program for causing a computer to function as a correction unit that corrects the nutrition value associated with the material based on the estimated amount of the material and calculates a corrected nutrition value. The meal amount estimation program according to claim 1,
The index image data set further includes error information associated with the size information,
The identifying unit identifies an index image with less error when there are a plurality of the index images included in the meal image;
Meal amount estimation program. The meal amount estimation program according to claim 1 or 2,
Causing the computer to further function as an index receiving unit that receives an operation for selecting an object existing in the acquired meal image and an operation for inputting the size information of the object from a user;
The estimation unit estimates the amount of the material associated with the identified material image based on the input size information of the selected object.
Meal amount estimation program. A method for estimating the amount of meal according to any one of claims 1 to 3,
The acquisition unit acquires from the user a first meal image and a second meal image that is an image after ingestion of the first meal image;
A difference for obtaining a difference between a first corrected nutritional value calculated based on the first meal image by the correction unit and a second corrected nutritional value calculated based on the second meal image. A meal amount estimation program that causes a computer to function as a calculation unit. A meal amount estimation program according to claim 4,
The acquisition unit specifies and acquires the first meal image and the second meal image based on time information given to the meal image of the user.
Meal amount estimation program. An index image data set in which the index image and the size information are associated; a material image data set in which the material image and the material name are associated; and a nutrition value dataset in which the material and the nutrition value of the material are associated. The step of memorizing
Obtaining a meal image of the user;
Identifying the index image and the material image included in the meal image;
Estimating the amount of the material associated with the identified material image based on the size information associated with the identified index image;
Correcting the nutrition value associated with the material based on the estimated amount of the material, and calculating a corrected nutrition value; An index image data set in which the index image and the size information are associated; a material image data set in which the material image and the material name are associated; and a nutrition value dataset in which the material and the nutrition value of the material are associated. Means for storing
Means for obtaining a user's meal image;
Means for identifying the index image and the material image included in the meal image;
Means for estimating the amount of the material associated with the identified material image based on the size information associated with the identified index image;
A meal amount estimation apparatus comprising: means for correcting the nutrition value associated with the material based on the estimated amount of the material and calculating a corrected nutrition value.

Images