WO2016031246A1

WO2016031246A1 - Nutrient quantity calculating device and refrigerator provided with same

Info

Publication number: WO2016031246A1
Application number: PCT/JP2015/004290
Authority: WO
Inventors: 平石　智一; 豊嶋　昌志; 浩史佐藤; 英蓮安; 綾子齊藤; 秀子蓮池; 一歩宮沢; 一生永利; 昌哉弦巻; 小川　誠; 陽平佐藤; 雅則久保田
Original assignee: ハイアールアジア株式会社
Priority date: 2014-08-26
Filing date: 2015-08-26
Publication date: 2016-03-03
Also published as: CN107077709A; JPWO2016031246A1; JP6577365B2; CN107077709B

Abstract

The invention provides a nutrient quantity calculating device which uses a simple operation to calculate the quantity of a nutrient in a foodstuff, and a refrigerator equipped with the same. The nutrient quantity calculating device 10 is provided with a measuring instrument 12 which weighs and captures an image of a foodstuff 16, and a mobile terminal 24 which, for example, estimates the type of the foodstuff on the basis of information input from the measuring instrument 12. The measuring instrument 12 is primarily provided with: a weighing unit 14; an image-capturing unit 18 and an illuminating unit 20 disposed above the weighing unit 14; and a control unit 22 connected to the weighing unit 14. By means of such a configuration, the nutrient quantity calculating device 10 is capable of calculating the quantity of a nutrient contained in the foodstuff 16 placed on the measuring instrument 12, and presenting the calculation result to a user.

Description

Nutrient amount calculation device and refrigerator equipped with the same

The present invention relates to a nutrient amount calculation device that calculates the amount of nutrients of ingredients used by a user for cooking, and a refrigerator equipped with the nutrient amount calculation device.

Calculating the amount of ingredients and food nutrients consumed by the user is important for health management and physical condition maintenance. Since the amount of nutrients in a dish is determined by the type and amount of ingredients used in cooking, it is possible to calculate the amount of nutrients in a dish based on information on ingredients. Specifically, the total nutrient amount of the food can be calculated by measuring the weight of the food and multiplying the weight by the amount of nutrient per unit amount of the food. However, it is complicated to perform this calculation work every time the user prepares a dish.

Patent Document 1 listed below describes a calorie calculation device that can automatically calculate the calories of food ingredients. Specifically, referring to FIG. 1 and [0011]-[0021] and the like, the calorie calculation device 1 here includes a measurement unit 20, a weight detection unit 30, and a control unit 70. Yes. Further, the measurement unit 20 measures moisture contained in the analysis object S, and the weight detection unit 30 measures the weight of the analysis object S. The control unit 70 calculates the calorie of the analysis object S using the measurement result of the measurement unit and the measurement result of the weight detection unit. Thereby, the effect which can measure the calorie of analysis subject S which is food, for example is acquired.

JP 2014-126559 A

However, the invention described in Patent Document 1 has room for improvement from the viewpoint of simplifying the calorie calculation method. Specifically, in order to operate the measurement unit in order to calculate calories, it is necessary for the user to operate an operation unit provided separately. In addition, a plurality of ingredients are used in normal cooking, but such a calorie calculating device requires an operation for calculating the calories for each ingredient.

Furthermore, in the calorie calculation device described in Patent Document 1, the calorie of the analysis object S is calculated based on the moisture ratio of the analysis object S. However, although it is possible to analyze the surface state of the object to be analyzed by the analysis method using infrared rays or the like, it is difficult to analyze the internal state, and thus it is difficult to specify the type. Therefore, the case where it is difficult to analyze the calorie of analysis object S appropriately is anticipated.

Furthermore, in Patent Document 1, the total calories are calculated from the weights of proteins, carbohydrates, and the like contained in foodstuffs. In addition to these, if the amount of nutrients such as inorganic substances such as vitamins can be estimated, it is convenient for the user. The property can be further improved.

An object of the present invention is to provide a nutrient amount calculation device for estimating the amount of nutrients of ingredients used by a user for cooking with a simple operation, and a refrigerator including the nutrient amount calculation device.

The nutrient amount calculation apparatus according to the present invention is based on the food material photographing means for obtaining food image data by photographing food before cooking, the food measuring means for obtaining food weight data by measuring the food, and the food image data. A food material type estimation means for estimating the food material type; a nutrient content calculation means for calculating the nutrient content contained in the food material based on the food material type and the food weight data; The photographing means photographs the food when the variation in the food weight data measured by the food measuring means is less than a certain value.

Furthermore, in the nutrient amount calculation apparatus according to the present invention, the food metering unit obtains the food weight data by measuring the food material at regular intervals, and the food photographing unit measures the food material at a plurality of times measured immediately before. The food is photographed when the standard deviation of the weight data is less than a certain value.

Furthermore, in the nutrient amount calculation apparatus of the present invention, the food photographing unit captures the foods sequentially placed on the food metering unit by photographing the food when the food weight data is different from the previous photographing. A picture is taken every time the food is placed.

Furthermore, the nutrient amount calculation apparatus of the present invention is characterized in that the food photographing means photographs the food after a certain period of time after the weight of the food placed on the food measuring means is determined.

Furthermore, in the nutrient amount calculation apparatus of the present invention, the food material type estimation unit calculates an image feature value from the food image data, and from the food material list in which the image feature value and the food material type are listed in association with each other. The food material having a close image feature amount is selected, and the nutrient amount calculation means multiplies the nutrient amount per unit amount of the selected food material by the food material weight data to obtain the nutrient amount of the food material. It is characterized by calculating.

Furthermore, the nutrient amount calculation apparatus of the present invention is characterized in that the type of food and the image feature amount selected by the food type estimation means are added to the food list.

Furthermore, in the nutrient amount calculation apparatus according to the present invention, the food photographing unit is configured to photograph the foods sequentially placed on the food metering unit each time the foods are placed, so that a plurality of the food image data is obtained. The food material type estimation means identifies the image portion of the newly added food material by taking the difference between the latest food image data and the food image data photographed last time, The image feature amount is calculated from an image portion.

Furthermore, the nutrient amount calculation apparatus of the present invention is characterized in that the food material type estimation means and the nutrient amount calculation means are realized as a function of a mobile terminal.

Furthermore, the refrigerator of the present invention is characterized by including the nutrient amount calculation device described above.

The nutrient amount calculation apparatus according to the present invention is based on the food material photographing means for obtaining food image data by photographing food before cooking, the food measuring means for obtaining food weight data by measuring the food, and the food image data. A food material type estimation means for estimating the food material type; a nutrient content calculation means for calculating the nutrient content contained in the food material based on the food material type and the food weight data; The photographing means photographs the food when the variation in the food weight data measured by the food measuring means is less than a certain value. Therefore, it is possible to photograph the food placed on the food weighing means more clearly by photographing the food after the fluctuation of the weight measured by the food weighing means becomes less than a certain value, and the food image The accuracy of estimation using data is improved. Further, since the food photographing means photographs the food based on the output of the food measuring means, it is possible to photograph the food without the user performing a special operation for photographing.

Furthermore, in the nutrient amount calculation apparatus according to the present invention, the food metering unit obtains the food weight data by measuring the food material at regular intervals, and the food photographing unit measures the food material at a plurality of times measured immediately before. The food is photographed when the standard deviation of the weight data is less than a certain value. Therefore, it is possible to photograph the food in a more stable state using the food photographing unit.

Furthermore, in the nutrient amount calculation apparatus of the present invention, the food photographing unit captures the foods sequentially placed on the food metering unit by photographing the food when the food weight data is different from the previous photographing. A picture is taken every time the food is placed. Therefore, the image data of the food can be sequentially photographed without the user performing a special operation for photographing.

Furthermore, the nutrient amount calculation apparatus of the present invention is characterized in that the food photographing means photographs the food after a certain period of time after the weight of the food placed on the food measuring means is determined. Accordingly, it is possible to prevent a user's hand operating the food material from being erroneously reflected in the food material image data.

Furthermore, in the nutrient amount calculation apparatus of the present invention, the food material type estimation unit calculates an image feature value from the food image data, and from the food material list in which the image feature value and the food material type are listed in association with each other. The food material having a close image feature amount is selected, and the nutrient amount calculation means multiplies the nutrient amount per unit amount of the selected food material by the food material weight data to obtain the nutrient amount of the food material. It is characterized by calculating. Accordingly, since the type of the food is specified using the image feature amount calculated from the color of the image, the type of the food can be easily specified without the user inputting the name of the food. .

Furthermore, the nutrient amount calculation apparatus of the present invention is characterized in that the type of food and the image feature amount selected by the food type estimation means are added to the food list. Accordingly, since the number of food lists used from the next search increases, it is possible to improve the estimation accuracy of the food material type based on the subsequent image feature amount.

Furthermore, in the nutrient amount calculation apparatus according to the present invention, the food photographing unit is configured to photograph the foods sequentially placed on the food metering unit each time the foods are placed, so that a plurality of the food image data is obtained. The food material type estimation means identifies the image portion of the newly added food material by taking the difference between the latest food image data and the food image data photographed last time, The image feature amount is calculated from an image portion. Accordingly, since the image feature amount of the food is calculated using image data of only the portion where the food is photographed, the image feature amount is calculated more accurately and the accuracy of continuously estimating the type of the food is improved. I can do it.

Furthermore, the nutrient amount calculation apparatus of the present invention is characterized in that the food material type estimation means and the nutrient amount calculation means are realized as a function of a mobile terminal. Therefore, by connecting to a telephone communication line via a portable terminal and using the information stored in the server, it is possible to improve the accuracy of estimating the kind of food.

Furthermore, the refrigerator of the present invention is characterized by including the nutrient amount calculation device described above. Therefore, the convenience of the user who uses a refrigerator can be improved by giving the refrigerator in which a foodstuff is stored the function which calculates the nutrient amount of a foodstuff.

It is a schematic diagram which shows the nutrient amount calculation apparatus concerning the 1st Embodiment of this invention. It is a flowchart which shows the method of calculating a nutrient amount using the nutrient amount calculation apparatus concerning the 1st Embodiment of this invention. It is a flowchart which shows the method of calculating a nutrient amount using the nutrient amount calculation apparatus concerning the 1st Embodiment of this invention. It is a figure which shows the method of estimating the kind of foodstuff from image data in the method of calculating a nutrient amount using the nutrient amount calculation apparatus concerning the 1st Embodiment of this invention. It is a figure which shows the nutrient amount calculation apparatus concerning the 2nd Embodiment of this invention, (A) is a block diagram which shows the structure of a nutrient amount calculation apparatus, (B) is a block which shows the data structure used FIG. (A) is a perspective view which shows the structure of the nutrient amount calculation apparatus concerning the 2nd Embodiment of this invention, (B) is a perspective view which shows a refrigerator. It is a flowchart which shows the method of calculating the nutrient amount of a dish with the nutrient amount calculation apparatus concerning the 2nd Embodiment of this invention. It is a flowchart which shows the method of calculating the nutrient amount of a dish with the nutrient amount calculation apparatus concerning the 2nd Embodiment of this invention.

<First Embodiment>
Hereinafter, the nutrient amount calculation apparatus 10 according to the present embodiment will be described.

FIG. 1 illustrates the configuration of the nutrient amount calculation apparatus 10. The nutrient amount calculation device 10 measures the food 16 before cooking and measures the measurement, and a portable terminal that estimates the type of the food and calculates the nutrient amount based on information input from the measurement device 12. 24. The main function of the nutrient amount calculation device 10 is to calculate the amount of nutrients contained in the food material 16 placed on the measuring instrument 12, and to present the calculation result to the user. Therefore, the user can easily know the nutrient amount of the food 16 by placing the food 16 before cooking on the measuring instrument 12. In this embodiment, nutrients include, for example, minerals such as calories and salt, vitamins, proteins, carbohydrates, fats and the like.

Further, the nutrient amount calculation device 10 of this embodiment may include a server 28 in addition to the measuring device 12 and the portable terminal 24 described above. In this case, the portable terminal 24 or the measuring instrument 12 is connected to the server 28 via a communication network such as the Internet. The food image data obtained by photographing the food 16, the food ID indicating the type (food name) of the food 16, and the food weight data indicating the weight of the food 16 are transmitted from the portable terminal 24 to the server 28. In addition, the server 28 accumulates and analyzes these data transmitted from the mobile terminals 24 of a large number of users, and feeds back the analysis information based on the analysis results to the mobile terminal 24, so that the food that has taken the food 16 is captured. It is possible to improve the accuracy of estimating the type of the food 16 from the image data.

The measuring instrument 12 includes a weighing unit 14 (foodstuff weighing unit), an imaging unit 18 (foodstuff photographing unit) and an illumination unit 20 disposed above the weighing unit 14, and a control unit 22 connected to the weighing unit 14. The main features.

The weighing unit 14 is a so-called electronic balance, and transmits an electric signal indicating the weight of the food 16 placed on the upper surface thereof to the control unit 22. In order to clarify the outer edge of the foodstuff 16 to be placed, a color different from the color of a general foodstuff is adopted as the color of the upper surface of the measuring unit 14. In this embodiment, since the nutrient amount of the food 16 is calculated based on the food weight data measured by the weighing unit 14, the nutrient amount can be accurately calculated as compared with the case of calculating from only the food image data. .

The imaging unit 18 is composed of an imaging device such as a CCD, for example, and is disposed above the weighing unit 14. The imaging unit 18 obtains food image data by photographing the weighing unit 14 on which the food 16 is placed from above. The obtained food image data is transmitted to the control unit 22. When the imaging unit 18 images the food 16, the relative positions of the imaging unit 18 and the weighing unit 14 are fixed. Accordingly, it is possible to stably photograph the food 16 placed on the upper surface of the weighing unit 14. In this embodiment, the timing at which the food image data is acquired by the imaging unit 18 is determined based on the output of the weighing unit 14, and such matters will be described later with reference to FIG. 2 and the like.

The illumination unit 20 includes, for example, an LED, and is disposed above the measuring unit 14 and in the vicinity of the imaging unit 18. The illumination unit 20 has a function of emitting light toward the food 16 when the imaging unit 18 photographs the food 16. Since the relative position of the illumination unit 20 with respect to the imaging unit 18 and the weighing unit 14 is fixed, the conditions for photographing the food 16 are made the same, and the type of the food 16 is selected using the obtained food image data. The accuracy at the time of estimation can be improved.

The control unit 22 has a predetermined control program installed therein, and has the function of controlling the operations of the imaging unit 18 and the illumination unit 20 as well as the above-described food image data and food weight data being input. The control unit 22 also has a function of communicating with the mobile terminal 24 arranged in the vicinity of the measuring instrument 12. The control unit 22 and the portable terminal 24 may be wired or wirelessly connected. For wireless connection, for example, Wi-Fi standard data communication can be employed.

The mobile terminal 24 is, for example, a smartphone owned by the user, and an application for controlling the measuring instrument 12 is installed. The portable terminal 24 includes a display unit 26 that is a touch panel, for example, and can display the food image data captured by the imaging unit 18 on the display unit 26. Further, by operating the display unit 26, the type of the food material 16 can be specified as will be described later. The portable terminal 24 stores programs that are nutrient amount calculation means and food material type estimation means in advance.

In this embodiment, the user operates the portable terminal 24 separate from the measuring device 12 to estimate the nutrient amount of the food material 16, but the functions of both may be integrated. That is, an operation unit such as a touch panel may be provided in the measuring instrument 12, and the user may select the food 16 by operating the operation unit.

Next, a method for calculating the nutrient amount of the food material 16 using the nutrient amount calculation device 10 described above will be described with reference to FIG. 1 with reference to FIG. Here, each step from step S11 to step S13 and step S16 to step S21 described below is performed by the measuring instrument 12 described above. On the other hand, Step S14 to Step S15 and Step S22 to Step S36 are performed by the portable terminal 24.

First, in order to calculate the nutrient amount of the food material 16, the user turns on the measuring instrument 12 (steps S10 and S11). Next, initial weight data, which is information about the weight, is obtained from the weighing unit 14 and recorded in the control unit 22 as an initial value (0 g) (step S12). Thereafter, the measuring unit 14 in a state where the food material 16 is not placed on the upper surface is photographed by the imaging unit 18 (step S13). The photographed initial food image data is stored in the control unit 22 as an initial value.

Here, in the case where the illuminance around the measuring instrument 12 is insufficient, when the measuring unit 14 is imaged by the imaging unit 18 in order to capture a clear image of the upper surface of the measuring unit 14, the measuring unit is used by the illumination unit 20. 14 may be irradiated. In such a case, in order to make the photographing conditions uniform, irradiation by the illumination unit 20 is also performed when photographing the food image data of the food 16 in the steps described later.

Next, after starting a dedicated application on the mobile terminal 24 which is a smartphone, the measuring instrument 12 and the mobile terminal 24 are connected using a wireless connection based on the Wi-Fi standard (step S14). Then, the application acquires the initial food weight data and the initial food image data obtained in the above steps from the measuring instrument 12 (step S15).

Thereafter, the food 16 to be cooked is placed on the upper surface of the measuring unit 14 of the measuring instrument 12 (step S16). The food weight data measured by the weighing unit 14 is sequentially transmitted to the control unit 22, but immediately after the food 16 is placed on the weighing unit 14, the value of the food weight data measured by the weighing unit 14 is stable. Not done. Therefore, in this embodiment, the process waits until the value of the food weight data becomes a constant value (step S17). Further, in this embodiment, in order to prevent the accuracy of estimating the food 16 from being reduced as a result of the user's hand handling the food being reflected in the food image data, the weighing unit 14 measures in step S17. After the food weight data is stabilized, photographing is further waited for a predetermined time (step S18).

When the above-described fixed time has elapsed, the food weight data of the food 16 measured by the measuring unit 14 at that time is recorded in the control unit 22 and the food image 16 is obtained by photographing the food 16 from above with the imaging unit 18 ( Steps S19 and S20). Thereafter, the obtained food weight data and food image data are transmitted from the measuring device 12 to the portable terminal 24 (step S21).

In the following steps, the portable terminal 24 calculates the type of the food 16 and the amount of nutrients based on the transmitted food image data and food weight data.

Specifically, first, a difference from the value of the food weight data received last time is taken (step S22). That is, when the first food 16 is measured, the state in which the food 16 is not placed, that is, 0 g is subtracted from the value of the food weight data when the first food 16 is placed. When measuring the second food 16, the value of the food weight data when the first food 16 is placed is subtracted from the value of the food weight data when the second food 16 is placed. .

This matter will be described in detail with reference to FIG. FIG. 4 schematically shows a situation of image processing or the like when three food ingredients A, B, and C are sequentially weighed and photographed. The weight Xg first measured by placing the food A on the measuring unit 14 is used as it is as the food weight data of the food A in the subsequent processing. Next, when the food B is placed on the weighing unit 14, Yg is measured, but (YX) g is used as the food weight data of the food B. Further, when the food C is placed on the measuring unit 14, Zg is measured, but (ZYX) g is used as the food weight data of the food B.

Next, the updated image portion is used as an image of the newly added food material 16 by taking the difference from the previously captured food image data (steps S23 and S24).

This matter will be described with reference to FIG. For example, first, the food image data 30 in which the food A is placed on the weighing unit 14 is acquired, and the image portion of the food A is obtained from the difference from the food image data (not shown here) on which the food is not placed. Is generated. After that, when the ingredient B is further placed and the ingredient image data 36 is acquired, the difference between the ingredient image data 30 and the ingredient image data 36 is taken to obtain the ingredient image data from which the image portion of the ingredient B is separated. 38 is generated. Furthermore, when the foodstuff image C is further placed and the foodstuff image data 42 is acquired, the foodstuff image data 44 in which the image portion of the foodstuff C is separated by taking the difference between the foodstuff image data 36 and the foodstuff image data 42. Is generated.

Next, an image feature amount is calculated (step S25). Specifically, based on the color and roughness of the image portion generated as described above, an image feature amount obtained by quantifying the feature of the image portion is calculated. And the thing with an image feature-value near is searched from "the foodstuff list selected in the past" and prior learning data (step S26). In the “past ingredient list selected in the past”, the ingredient ID indicating the type of the ingredient 16 selected in the past and the image feature amount are associated and listed. The pre-learning data is input in advance when the application of the mobile terminal 24 is shipped, and is a list in which the food ID and the image feature amount are associated with each other. In the following description, the “food list selected in the past” and the pre-learning data may be collectively referred to as a food list. And the image feature amount calculated in step S25 and the food material having the similar image feature amount are searched from the food material list (step S26).

In step S27, a plurality of candidate ingredients whose image feature amounts are close to the ingredients to be searched from the ingredient list are displayed on the display unit 26 of the portable terminal 24. Here, when displaying a candidate foodstuff on the display part 26, a foodstuff name may be displayed and the image imitating a foodstuff may be displayed.

Here, in principle, it is also possible to use the food with the closest image feature quantity from the food list as the food to be searched. In this embodiment, as will be described later, by selecting a plurality of ingredients having similar image feature amounts from the ingredient list and selecting a search target ingredient from the plurality of ingredients, the accuracy of estimating the ingredient 16 is improved. .

If the search target food is present in the food displayed on the display unit 26 (YES in step S28), the user selects the correct food by operating the display unit 26, which is a touch panel, etc. ( Step S29). At this time, as shown in the lower part of FIG. 4, the

food image data

34, 40, 46 indicating the selected food may be displayed on the display unit 26.

Also, the food material ID corresponding to the selected food material 16 is associated with the image feature amount of the food material 16 and added to the “food material list selected in the past” (step S30). As a result, the current search result can be used in the next candidate search for ingredients, so that the accuracy of the candidate search can be improved. That is, the “food list selected in the past” can be learned according to the lifestyle of the user.

On the other hand, when there is no correct ingredient in the ingredient candidate displayed in step S28 (NO in step S28), the user inputs the ingredient 16 into the text on the portable terminal 24, and the ingredient having the same name as the inputted ingredient name. Is retrieved from the ingredient master (step S31). Here, the food master is a list in which the food ID is associated with the nutrient amount of the food. Search results based on the input text are displayed on the display unit 26 of the portable terminal 24 (step S32).

Thereafter, the user selects a correct food from the displayed foods (step S33). The food material ID indicating the selected food material 16 and its image feature amount are recorded in association with each other in the same manner as in step S30 described above (step S34).

Next, the nutrient amount of the food 16 is calculated using the type and weight of the food 16 specified in the above step (step S35). Specifically, the amount of nutrient contained in the ingredient 16 is calculated by multiplying the amount of nutrient per unit quantity associated with the ingredient ID contained in the aforementioned ingredient master by the weight of the ingredient 16. As described above, the nutrient amount in this embodiment includes minerals such as calories and salt, vitamins, proteins, carbohydrates, fats and the like. Further, the accumulated nutrient amount is displayed on the display unit 26 of the portable terminal 24 (step S36). Thus, the user can know the amount of all nutrients contained in the food material 16 placed on the measuring instrument 12.

Each step from step S16 to step S36 described above is performed on each food material 16 prepared by the user (step S37).

Next, with reference to FIG. 3, step S <b> 16 to step S <b> 21 for photographing the food material 16 after it is placed on the weighing unit 14 will be described in detail below.

First, after the food 16 is placed on the weighing unit 14, the control unit 22 of the measuring instrument 12 acquires food weight data indicating the weight of the food 16 from the weighing unit 14 (step S50). The acquisition of the food weight data is performed continuously at regular intervals, for example, continuously at intervals of 0.1 seconds. In this embodiment, by applying the following steps to the food weight data acquired continuously, the food image data is acquired in a stable state, and the estimated accuracy of the food type using the food image data Can be improved.

Next, in order to enable the arithmetic processing described later, the food material weight data, which is analog data, is converted into digital data (step S51). Next, in the t-th loop, the average value w (t) of the five previous food weight data is obtained (step S52). For example, if the food weight data is measured at the fifth time, the average value of the food weight data from the first time to the fifth time is calculated. Further, the standard deviation α (t) of the last five food weight data is calculated using the previous five food weight data (step S53).

If the standard deviation α (t) calculated in the above step is smaller than a predetermined value (for example, 0.5) (YES in step S54), the value of the food weight data measured in the last five times is converged. Judge that there is. That is, the control unit 22 can determine that the time variation of the food weight data is sufficiently small, the posture of the food 16 is stable, and the food image data can be captured clearly.

Next, in step S55, it is determined whether or not the time variation of the food material weight data is in an unstable state (for example, more than 3 g), and the food material weight data is different from the food material weight data at the previous photographing. By confirming that the time variation of the food material weight data is in an unstable state, the food material weight data is obtained under the situation where the food material 16 is not added due to a drift phenomenon during long-time operation of the measuring instrument 12. It is possible to prevent the imaging unit 18 from taking a picture when the value slightly changes (for example, 3 g or less). In addition, by confirming that the food weight data is different from the food weight data at the time of previous shooting, the food 16 described later can be shot only when the food 16 is newly placed on the measuring unit 14.

When the time variation of the food material weight data is in an unstable state and the food material weight data is different from the previous photographing (YES in step S55), the measured food material weight data is determined (step S57). Further, the control unit 22 takes a picture of the food 16 with the imaging unit 18 after a predetermined time (for example, 0.5 seconds) has elapsed in order to retract the user's hand operating the food 16 from the field of view of the imaging unit 18. Obtain food image data. Further, the measuring unit 14 determines that the stable state has been reached. On the other hand, if the food weight data is not in an unstable state or if the food weight data is the same as that at the previous photographing (NO in step S55), the process returns to step S51 without determining the food weight data.

If the standard deviation α (t) is 0.5 or more (NO in step S54), the food weight data measured at the tth time (for example, the fifth time) and the t−1th time (for example, the fourth time) ) Is checked whether the absolute value of the difference from the measured food weight data is equal to or less than a predetermined value (for example, 3 grams) (step S56). If the above value is equal to or greater than the predetermined value (YES in step S56), there is a possibility that a new food 16 may be placed on the weighing unit 14, and the time variation of the food weight data has shifted to an unstable state. Judgment is made, and the process returns to step S51. On the other hand, if the above value is less than the predetermined value (NO in step S56), it is determined that the time variation of the food weight data remains in a stable state, and the process returns to the above step S51.

According to the above-described embodiment, since the imaging unit 18 captures the food 16 based on the output of the weighing unit 14 by placing the food 16 on the weighing unit 14, the user performs a special operation for shooting. The food image data can be easily acquired without performing the process.

In addition, when the deviation value of the food weight data measured by the weighing unit 14 is equal to or less than a predetermined value, the imaging unit 18 photographs the food 16, so that the food 16 that is stationary on the upper surface of the weighing unit 14 can be photographed. Therefore, it is possible to improve the accuracy of obtaining clear food image data and estimating the type of food 16.

Further, in this embodiment, the food image data is acquired by the imaging unit 18 based on the output of the measuring unit 14, and the nutrient amount of the food 16 is calculated on the mobile terminal 24 side using the food image data. Here, the food image data in this embodiment is a still image. If there is no function for determining when to shoot a still image based on the output of the weighing unit 14, it is necessary to analyze the moving image to calculate the nutrient content of the food, but this is required for the analysis of the moving image. The amount of information processing is large compared to still images. Therefore, in this embodiment, the information processing amount on the portable terminal 24 side can be remarkably reduced and the power consumption amount can be reduced.

In this embodiment, since the food image data is image data obtained by photographing the food 16 that is stationary on the upper surface of the measuring unit 14, it is the most accurate image data for estimating the type of the food 16. Therefore, even if it compares with the case where the amount of nutrients of a foodstuff is calculated by analyzing a moving picture, the accuracy which estimates the kind of foodstuff 16 is not inferior. In other words, in this embodiment, imaging of the food image data most suitable for the analysis of the food material 16 is performed using the output of the weighing unit 14 as a trigger, and the amount of nutrients of the food material 16 is accurately controlled while keeping the amount of data to be analyzed low. It is a good calculation.

Here, with reference to FIG. 4, in the above-described form, the nutrient amount is calculated while adding the food material 16, but the food material 16 may be deleted along the way. In that case, the control part 22 can detect that the foodstuff 16 was deleted because the foodstuff weight data measured by the measurement part 14 reduce | decrease. Moreover, the control part 22 identifies the kind of the deleted foodstuff 16 by comparing the foodstuff image data image | photographed with the imaging part 18 with the last thing.

Further, the nutrient amount calculation device 10 described above may be incorporated as a function of the refrigerator. By adding a function of calculating the nutrient amount of the food 16 to the refrigerator in which a large number of food 16 is stored, the added value of the refrigerator can be improved.

This embodiment is particularly effective when the server 28 calculates the nutrient amount of the food material 16. As described above, the food image data is a still image and has a much smaller amount of information than a moving image. However, the food image data has high accuracy because it is obtained by taking an image of the output of the weighing unit 14 as a trigger. Therefore, it is possible to calculate the nutrient amount of the food 16 with high accuracy while reducing the amount of information transmitted to the server 28 through the communication line.
<Second Embodiment>
Hereinafter, a nutrient amount calculation apparatus 110 according to an embodiment of the present invention will be described in detail with reference to the drawings. In the nutrient amount calculation apparatus 110 according to the present embodiment, the nutrient amount is calculated from the food image data obtained by photographing the food 140 before cooking, and the type of dish and the cooking image data 130 obtained by cooking these foods 140. And correspondingly recorded. Thereby, it is possible to calculate the amount of nutrients only by taking an image of the cooked food.

FIG. 5A is a block diagram illustrating a schematic configuration of the nutrient amount calculation apparatus 110, and FIG. 5B is a block diagram illustrating nutrient amount calculation data 124 that is referred to in order to calculate the nutrient amount.

With reference to FIG. 5 (A), the nutrient amount calculation device 110 of this embodiment includes a food photographing unit 112, a food measuring unit 114, a dish photographing unit 116, a dish measuring unit 118, and a nutrient amount calculating unit 120. Storage means 122. The general function of the nutrient amount calculation device 110 is to simply calculate the nutrient amount of the cooked food from the weight data or image data of the food 140 used for cooking or the food itself.

The food material photographing means 112 is a means for photographing the food material 140 to be cooked in color. Specifically, the food material photographing means 112 is constituted by a photographing element such as a CCD. The food imaging unit 112 generates the food image data 126 by shooting the food 140, and the food image data 126 is transmitted to the nutrient amount calculation unit 120. Here, the food photographing unit 112 may photograph one of the ingredients 140 used for cooking one by one, or may photograph a plurality of the same kind of materials.

The food material measuring means 114 is a means for measuring the weight of the food material 140 to be cooked. The food material weighing unit 114 measures the food material 140 to obtain food material weight data 128 indicating the weight of the food material 140, and the food material weight data 128 is transmitted to the nutrient amount calculating unit 120. Here, the food measuring unit 114 may measure the foods 140 one by one, or may measure a plurality of the same kind of foods 140 simultaneously.

The dish photographing unit 116 is a unit that photographs a dish made by cooking the above-described food material 140. The dish image data 130 obtained by photographing the dish by the dish photographing unit 116 is transmitted to the nutrient amount calculating unit 120.

The dish weighing means 118 is a means for weighing cooked dishes. The dish weight data 132 obtained by the dish weighing unit 118 weighing the dish is transmitted to the nutrient amount calculating unit 120.

Here, the above-described food photographing unit 112 and the dish photographing unit 116 may be provided individually or a single photographing unit may be used. Further, the food weighing means 114 and the food weighing means 118 may be provided individually or a single photographing means may be used.

The nutrient amount calculation means 120 calculates the amount of nutrients of the cooked food from each data transmitted from each means described above. As the nutrient amount calculation means 120, for example, a CPU is employed. The nutrient amount calculation means 120 may also calculate the amount of nutrients such as salt as will be described later in addition to the amount of nutrients in the dish.

Specifically, when calculating the amount of nutrients for cooking from the ingredients 140, the type of the ingredients 140 is estimated from the ingredients image data 126, and the estimated nutrient amount per unit amount of the ingredients 140 is multiplied by the ingredient weight data 128. For example, the nutrient amount of each food 140 is calculated. Then, the nutrient amount of the dish scheduled to be cooked is calculated by adding the nutrient amounts of all the ingredients 140 used for cooking.

Further, when calculating the nutrient amount from the data related to the cooked dish, the type of dish is estimated from the dish image data 130, and the estimated nutrient amount per unit amount of the dish is multiplied by the dish weight data 132. The nutrient content of the dish is calculated.

A specific method for calculating the amount of nutrients in a dish will be described later with reference to a flowchart shown in FIG.

The storage means 122 is means for storing each image data, each weight data, etc. obtained by each means described above. Specifically, a hard disk or a semiconductor storage device is employed as the storage unit 122. The nutrient amount calculation apparatus 110 does not necessarily include the storage unit 122, and a server connected via a network or the like may be used as the storage unit, and the above-described data may be stored in the server.

With reference to FIG. 5 (B), the nutrient amount calculation data 124 used for calculating the nutrient amount described above will be described. The nutrient amount calculation data 124 includes ingredient image data 126, ingredient weight data 128, dish image data 130, dish weight data 132, ingredient database 125, dish database 127, ingredient nutrient quantity database 129, and dish nutrient quantity database 131. .

These data will be explained next. The food image data 126 is image data of a still image obtained by photographing with the above-described food photographing means 112. The food weight data 128 is data indicating the weight of the food 140 obtained by weighing with the food weighing means 114. The dish image data 130 is image data of a still image obtained by photographing a dish made by cooking the ingredients 140 with the dish photographing unit 116. The dish weight data 132 is data indicating the weight of the dish measured by weighing the prepared dish with the dish weighing unit 118.

The food material database 125 is a database composed of data associating features extracted from the food image data 126 with the types of the food materials 140. For example, the color and surface roughness extracted from the food image data 126 and the food material 140 are stored. The database to associate. The dish database is a database that associates the dish image data 130 with the kind of dish, and includes, for example, data that associates the color of the surface of the dish with the kind of dish.

The ingredient nutrient amount database 129 is a database that associates the ingredient 140 with the nutrient amount per unit quantity of the ingredient 140. The cooking nutrient amount database 131 is a database that associates a dish with the nutrient amount per unit amount of the dish. As these data, data publicly disclosed by public institutions may be used, or data accumulated and revised by the user using the nutrient amount calculation device 110 of this embodiment may be used. Both data may be used in combination.

Here, two or more of the above databases may be integrated. For example, the ingredient database 125 and the dish database 127 may be integrated into one database. In addition, the food nutrient amount database 129 and the cooking nutrient amount database 131 may be integrated into a single database.

The configuration of the nutrient amount calculation apparatus 110 embodied with reference to FIG. 6 will be described. 6A is a perspective view showing a specific configuration of the nutrient amount calculation apparatus 110, and FIG. 6B is a perspective view showing a refrigerator 142 in which the nutrient quantity calculation apparatus 110 is incorporated.

Referring to FIG. 6A, the nutrient amount calculation device 110 includes a pedestal 134 on which the food 140 is placed, a movable support 136 whose end is fixed to the pedestal 134, and a support 136. And an imaging unit 138 provided.

The pedestal 134 is a plate-like member having a flat surface on which the food material 140 is placed, and has a built-in measuring module for measuring the food material 140. Further, the nutrient amount calculation means 120 that receives various data from the measurement module and the imaging unit 138 and calculates the nutrient amount may be incorporated in the base 134.

The support portion 136 is a rod-like member disposed near the end of the pedestal 134, and the lower end thereof is rotatably connected to the pedestal 134. Further, since a concave region corresponding to the shape of the support portion 136 is provided on the upper surface of the pedestal 134, the tilted support portion 136 is accommodated in the concave region.

Near the upper end of the support part 136, an imaging part 138 made of, for example, a CCD is provided. The imaging unit 138 is installed at a position where the food 140 placed on the upper surface of the pedestal 134 is photographed in a state where the support unit 136 is raised. The food image data 126 obtained by photographing by the imaging unit 138 is transmitted to a processing unit built in the pedestal 134.

Here, the nutrient amount calculation data 124 described with reference to FIG. 5B may be recorded on a recording medium such as a hard disk built in the nutrient amount calculation device 110, or a recording device arranged outside. May be recorded. Alternatively, a part of the nutrient amount calculation data 124 may be recorded in a recording device built in the nutrient amount calculation device 110, and another part of the nutrient amount calculation data 124 may be recorded in a recording device arranged outside. Here, all or part of the nutrient amount calculation data 124 is recorded in the server 156. The nutrient amount calculation device 110 and the server 156 are connected via a communication network 158 such as the Internet network. A method of using the nutrient amount calculation apparatus 110 described above will be described later with reference to FIG.

FIG. 6B illustrates a refrigerator 142 as one application example of the nutrient amount calculation device 110 described above. The refrigerator 142 includes a plurality of storages such as a refrigerator and a freezer, and the front opening of each storage is closed by

doors

144, 146, 148, and 150 so as to be opened and closed. For example, the door 144 rotates and opens in the left-right direction with either left or right end as a fulcrum, and the

doors

146, 148, 150 open and close in the front-rear direction.

The small door 152 is a door in which a part of the door 146 is rotatable in the front-rear direction, and rotates and opens with the lower end as a fulcrum. The upper surface of the small door 152 in the opened state functions as the pedestal 134 shown in FIG. An imaging unit 154 that captures the food 140 is disposed near the opening of the door 146.

When the user calculates the nutrient amount of a dish using the refrigerator 142, first, the small door 152 is opened, and the ingredients 140 and the dish are placed on the upper surface of the small door 152 in the opened state. Then, the food material 140 or the dish is weighed by a weighing module built in the small door 152. In addition, the food 140 or the dish is photographed by the imaging unit 154. As a result, the food 140 and the food are measured and photographed.

By incorporating the nutrient amount calculating device 110 into the refrigerator 142, the food material 140 used for cooking is generally stored in the refrigerator in many cases, so that the food material 140 can be immediately measured and the measurement and photography can be further simplified. Can be performed.

Based on FIG. 7 and FIG. 8, the calculation method of the nutrient amount of the dish using the nutrient amount calculation device 110 will be described with reference to each of the above-described drawings. FIG. 7 shows a nutrient amount calculation method at the first stage of cooking a specific dish, and FIG. 8 shows a nutrient amount calculation method for the second and subsequent times of cooking the specific dish. Here, each data described below is appropriately stored in the nutrient amount calculation device 110, the server 156, and the like shown in FIG.

Referring to FIG. 7, the nutrient amount calculation method in the first stage of cooking a specific dish will be described.

First, the nutrient amount calculation device 110 shown in FIG. 6A is activated (step S111). Specifically, the support 136 is raised with respect to the pedestal 134 and the start switch is turned on.

And the foodstuff 140 before cooking is mounted on the upper surface of the base 134 (step S112). FIG. 6A illustrates a case where the fish before cooking is employed as the food material 140. Here, only one food 140 is placed on the upper surface of the pedestal 134, but a plurality of the same kind of foods 140 may be placed on the upper surface of the pedestal 134. Thereby, it becomes possible to measure and photograph the food 140 easily.

Next, the food material 140 placed on the upper surface of the pedestal 134 is photographed by the imaging unit 138 (step S113). Specifically, the food 140 is imaged by the imaging unit 138 in a state where light is irradiated to the food 140 by a light emitting unit (not shown) as necessary. As a result, food image data 126 obtained by photographing the food 140 is obtained. The acquired food image data 126 is stored in a storage means such as a hard disk. Here, the food material 140 to be photographed may be an unprocessed product or a processed product. For example, if the food material 140 is a banana, the state before the skin is peeled off or the state after the skin is peeled off may be used. When the banana before facing the skin is photographed, the food 140 is estimated from this image, and the nutrient amount is calculated by excluding the portion corresponding to the skin. On the other hand, when the peeled banana is photographed, the food material 140 is estimated using the food image data 126, and the nutrient amount is calculated assuming that the whole is used as the food material 140.

Next, the type of food is estimated from the food image data 126 (step S114). Various methods are conceivable as a method for estimating the food material 140 from the food image data 126. Here, as an example, a method for estimating the food material 140 by focusing on the color and roughness of the surface of the food material 140 will be described.

Specifically, referring to FIG. 6A, a portion where foodstuff 140 is photographed is extracted from the above-described food image data 126, and data relating to the color and surface roughness of this portion is extracted. On the other hand, in the food database 125 shown in FIG. 5B, the color, surface roughness, type, and the like of the surface of the food are tabulated for each food. Therefore, the color and surface roughness of the food material 140 extracted from the food image data 126 are compared with the color and surface roughness recorded for each food material in the food material database 125, and the food material having the closest approximation of these values is “ Estimated ingredients ”.

Next, the user determines whether or not the food material estimated in the previous step is correct (step S115). Specifically, the estimated food image and name are displayed on a display device or the like associated with the nutrient amount calculation device 110. And if a user judges that the estimated foodstuff is correct, it will transfer to the next step by performing the operation to that effect (YES of step S115). On the other hand, if the displayed food is not correct (NO in step S115), other foods with similar colors and roughness are displayed (presented) to the user (step S116). As a result, if the newly displayed food is correct, the process proceeds to the next step (YES in step S117), and if it is not correct, another food is estimated and displayed (NO in step S117). Note that steps S115 and S117 for making this determination may be performed by a user operating a switch or a touch panel provided in the nutrient amount calculation apparatus 110 itself. Furthermore, the estimated food image or name may be displayed on a portable information terminal such as a smartphone in which a specific application is installed, and the user may operate it.

Here, when the type of the food material 140 is specified in step S115 and step S117, the combination of the food material image data 126 and the food material 140 in these steps is associated and stored. That is, the food material database 125 shown in FIG. 5B is revised. From the next time, this combination is used in step S114, thereby improving the accuracy of the estimation.

In general, the nutrient amount per unit amount is determined by the type of the food material 140, but the nutrient amount can be calculated by specifying the type of the food material 140 placed on the nutrient amount calculation device 110 by the above steps. It becomes. The type data specifying the type of the food material 140 is stored and used in a later step.

Further, in the above-described step S115 to step S117, if the estimated food 140 is not correct, another food 140 is presented, but if this estimation is not correct more than a predetermined number of times (for example, 5 times or more), the user The ingredients 140 may be input manually. Thereby, the step which a user selects the foodstuff 140 can be simplified.

Next, the food material 140 placed on the upper surface of the pedestal 134 of the nutrient amount calculation device 110 is weighed (step S118). Specifically, the weight of the food 140 is measured by a measuring module built in the pedestal 134. The food weight data 128 obtained by this measurement is stored in a storage device provided in the nutrient amount calculation device 110.

Next, in the calculation means (nutrient quantity calculation means 120) such as a CPU provided in the nutrient quantity calculation device 110, the estimation result of the kind of food 140 and the food weight data 128 described above are used for the cooking to be cooked. The total nutrient amount (first estimated cooking nutrient amount) is calculated (step S119).

Specifically, the amount of nutrients per unit amount is converted into data in the food nutrient amount database 129 (FIG. 5B). Therefore, for each ingredient 140, the ingredient weight data 128 is multiplied by the nutrient amount per unit quantity, and these are added to calculate the total nutrient quantity (first estimated dish nutrient quantity) of the cooked food.

The operations from step S112 to step S119 described above are performed for each food 140. As an example, when cooking curry, operations from step S112 to step S117 are performed for each material such as carrot, onion, meat, and potato. Thereby, the kind of each foodstuff 140 is specified, and those measurement and integration are also performed. In addition, the accumulated food 140 and nutrient amount are notified to the user, for example, by being displayed on a display provided in the nutrient amount calculation device 110. The calculated ingredients 140 and nutrient amounts may be accumulated for each nutrient amount, stored in association with the dish, and used as a nutrient amount database in a later step.

In step S120, a dish is prepared from the above-described ingredients 140. For example, the curry is cooked by frying or boiling the above-mentioned ingredients such as carrots.

Next, the dish made by cooking is placed on the pedestal 134 shown in FIG. 6A and weighed (step S121). In this step, since the dish is placed on the pedestal 134 together with a container such as a pot used for cooking, by storing the weight of the container in advance, the weight of the container is subtracted from the total weight, so that only the dish Can weigh. By this step, dish weight data 132 of the cooked dish is obtained. Furthermore, the dish image data 130 is obtained by photographing the dish placed on the pedestal 134 using the imaging unit 138.

Here, a step of selecting a specific cooking method may be performed between step S120 and step S121 described above. Specifically, the user inputs a cooking method such as boiling, frying, steaming, frying, etc. to the nutrient amount calculation device 110 via an input means such as a touch panel. In general, even if the ingredients 140 used are of the same type and amount, the amount of nutrients in the dish differs depending on the cooking method. For example, when fried, the oil used for cooking is added to the dish, so the amount of nutrients in the cooked dish is higher than when steamed. Thus, for example, when “stir fry” is input as a cooking method, the accuracy of the calculated nutrient amount can be improved by recalculating the nutrient amount of the dish in consideration of the amount of oil used. I can do it.

Furthermore, the dish image data 130 is obtained by photographing the dish placed on the upper surface of the pedestal 134 with the imaging unit 138. Here, the top surface of the dish is taken.

Next, the cooked food is estimated based on the dish image data 130 obtained in the previous step (step S123). This estimation method may be the same as step S113 described above.

Specifically, referring to FIG. 6 (A), a portion where a dish is photographed is extracted from the above-described food image data 126, and data relating to the color and surface roughness of this portion is extracted. On the other hand, in the dish database 127 shown in FIG. 5B, the color, surface roughness, type, and the like of the top surface of each dish are tabulated for each dish. Therefore, the color and surface roughness of the dish extracted from the dish image data 130 are compared with the color and surface roughness recorded in the dish database 127, and the dish having the closest approximation of these values is “estimated dish”. "

Next, the user determines whether or not the food estimated in the previous step is correct (step S124). Specifically, the estimated image and name of the dish are displayed on a display device or the like attached to the nutrient amount calculation device 110. When the user determines that the estimated food is correct, the calculation of the nutrient amount is completed by performing an operation to that effect (step S127). Note that, by specifying and weighing the type of cooked dish, the type of dish (for example, curry) estimated from the dish image data 130 is associated with the nutrient amount per unit amount. Data indicating this matter is registered in the cooking nutrient amount database 131 shown in FIG. 5B, and this data is used for the next cooking.

On the other hand, if the displayed dish is not correct (NO in step S124), other dishes having similar colors and roughness are displayed (presented) to the user (step S125). As a result, if the estimated dish is correct, the process proceeds to the next step (YES in step S126), and if it is not correct, another dish is estimated and displayed (NO in step S126).

In steps S124 to S126 described above, if the estimated dish is not correct, other dishes are presented. If this estimation is not correct more than a predetermined number of times (for example, five times or more), the user can The type of dish may be input manually. Thereby, the step which a user selects can be simplified.

Here, after the type of dish is specified, the combination of the dish image data 130 and the dish in these steps is assumed to be correct and stored as the dish database 127. Specifically, for each dish, the type of dish and the nutrient amount per unit amount are stored in association with each other. Then, from the next dish, this combination is used in step S123, so that the accuracy of the estimation is improved.

In the nutrient amount calculation method of the present embodiment described above, first, according to the method shown in steps S113 to S116, the type of the food 140 can be specified by the above-described image analysis without the user inputting the type of the food 140. Convenience has been improved. Similarly, since the type of dish is also specified by the method shown in steps S122 to S125, the input operation is not necessary and convenience is improved.

Referring to FIG. 8, the second and subsequent cooking methods performed by the user regarding the same dish will be described. The cooking method shown here is the same as the method described with reference to FIG. 7, except that the food 140 is not photographed and weighed before cooking.

First, after starting the nutrient amount calculation apparatus 110 shown in FIG. 6A (step S151), cooking is performed using the ingredients 140 (step S152). Here, when a similar user performs cooking, it is assumed that the types and ratios of the ingredients 140 used for cooking are similar, and the imaging and measurement of the ingredients 140 can be omitted for simplicity. Even when the same dish is cooked, if the type and ratio of the ingredients 140 to be used are different, the ingredients 140 may be photographed and measured.

Here, after cooking the dish, as described above, the user may input the type of cooking (sauté, boil), etc., to the nutrient amount calculation device 110. Thereby, it becomes possible to calculate the nutrient amount accurately by taking into account the seasoning used for cooking.

After cooking is completed, the dish is placed on the upper surface of the pedestal 134 of the nutrient amount calculation device 110 shown in FIG. 6A, and the weight is measured to obtain dish weight data 132 (step S153). . Then, the dish image data 130 is obtained by photographing the dish from above using the imaging unit 138 (step S154).

Next, the type of cooked food is estimated based on the dish image data 130, and if this estimation is incorrect, it is corrected (steps S155, S156, S157, S158). The specific method of each step is the same as steps S123, S124, S125, and S126 described with reference to FIG.

Next, the nutrient amount of the cooked dish is calculated based on the data regarding the type of dish and the dish weight data 132 obtained in the above step (step S159). Also, in this step, a nutrient amount database is prepared in which each dish is associated with the nutrient amount per unit amount, and the nutrient amount per unit amount of the corresponding type of dish is added to the weight of the weighed dish. By multiplying, the total nutrient amount is calculated.

Specifically, the nutrient amount per unit amount of curry to be cooked when the target user cooks curry, for example, is recorded in advance by the nutrient amount calculation method shown in FIG. This information is recorded for each type of dish in the dish nutrient database 131 shown in FIG. Therefore, the nutrient amount per unit amount of the target dish recorded in the dish nutrient database 131 is multiplied by the dish weight data 132 weighed in step S153 described above, so that the total nutrient amount (first) 2 (estimated cooking nutrient amount) is calculated (step S160).

According to the nutrient amount calculation method described above, the nutrient amount per unit amount of the dish to be cooked is estimated using the information regarding the ingredients 140 photographed by the first cooking shown in FIG. Therefore, it is possible to calculate the amount of nutrients only by photographing and weighing the dish after cooking.

The above-described embodiment can be modified as follows, for example.

In step S118 shown in FIG. 7, the nutrient amount may be calculated in consideration of the seasoning used and the loss rate of the material. Thereby, it becomes possible to calculate the amount of nutrients more accurately.

Here, the above-described embodiments can be implemented in combination with each other. For example, the weighing method and the photographing method of the food material 16 described in the first embodiment can be applied to the second embodiment.

DESCRIPTION OF SYMBOLS 10 Nutrient amount calculation apparatus 12 Measuring device 14 Weighing part 16, A, B, C Foodstuff 18 Imaging part 20 Illumination part 22 Control part 24 Portable terminal 26 Display part 28 Server 30 Foodstuff image data 32 Foodstuff image data 34 Foodstuff image data 36 Foodstuff Image data 38 Ingredient image data 40 Ingredient image data 42 Ingredient image data 44 Ingredient image data 46 Ingredient image data 110 Ingredient amount calculating device 112 Ingredient photographing means 114 Ingredient measuring means 116 Ingredient measuring means 116 Ingredient photographing means 118 Ingredient measuring means 120 Ingredient amount calculating means 122 Storage Means 124 Nutrient amount calculation data 125 Ingredient database 126 Ingredient image data 127 Cooking database 128 Ingredient weight data 129 Ingredient nutrient amount database 130 Cooking image data 131 Cooking nutrient amount database 132 Cooking weight Data 134 base 136 support 138 imaging unit 140 ingredients 142 Refrigerator 144 door 146 door 148 door 150 door 152 small door 154 imaging unit 156 Server 158 communication network

Claims

Ingredient photographing means for photographing ingredients before cooking and obtaining ingredient image data;
Ingredient weighing means for weighing the ingredients to obtain ingredient weight data;
On the basis of the food image data, the food material type estimation means for estimating the food material type,
A nutrient amount calculating means for calculating a nutrient amount contained in the food based on the type of the food and the food weight data;
The nutrient amount calculation apparatus, wherein the food photographing means photographs the food when a variation in the food weight data measured by the food measuring means becomes less than a certain value.
The food measuring means obtains the food weight data by measuring the food at regular intervals,
The nutrient amount calculation apparatus according to claim 1, wherein the food photographing unit photographs the food when a standard deviation of the food weight data measured a plurality of times immediately before is less than a certain value.
The food material photographing unit photographs the food material when the food material weight data is different from that at the previous photographing time, thereby photographing the food material sequentially placed on the food material measuring means every time the food material is placed. The nutrient amount calculation device according to claim 1 or 2, wherein the nutrient amount calculation device according to claim 1 or 2 is performed.
The said foodstuff imaging | photography means image | photographs the said foodstuff after a fixed period after the weight of the said foodstuff mounted in the said foodstuff measurement means is decided. Nutrient content calculator.
The food material type estimation means calculates an image feature value from the food image data, and selects the food material having the image feature value close from a food material list in which the image feature data and the food material type are associated with each other. Selected,
The said nutrient amount calculation means calculates the nutrient amount of the said foodstuff by multiplying the nutrient amount per unit amount of the selected said foodstuff and the said foodstuff weight data. 4. The nutrient amount calculation apparatus according to any one of 4 above.
6. The nutrient amount calculation apparatus according to claim 5, wherein the type of the food material and the image feature amount selected by the food material type estimation unit are added to the food material list.
The food photographing means obtains a plurality of food image data by photographing the food sequentially placed on the food measuring means, each time the food is placed,
The food material type estimation means identifies the image portion of the newly added food material by taking the difference between the latest food image data and the food image data photographed last time, and from the image portion, the The nutrient amount calculation apparatus according to claim 1, wherein an image feature amount is calculated.
The nutrient amount calculation apparatus according to any one of claims 1 to 7, wherein the food material type estimation unit and the nutrient amount calculation unit are realized as a function of a mobile terminal.
A refrigerator comprising the nutrient amount calculation device according to any one of claims 1 to 8.