JP2022147290A - Eating behavior monitoring device, system, program, and method for providing feedback based on temperatures of food/drink items - Google Patents

Abstract

To provide an eating behavior monitoring device, an eating behavior monitoring system, an eating behavior monitoring program, and an eating behavior monitoring method for providing feedback based on the temperatures of food/drink items.SOLUTION: In an eating behavior monitoring system, an eating behavior monitoring device 1 includes: a food/drink item and environmental temperature obtaining unit 111 which obtains, from temperature measurement means (such as, a thermographic camera 3 and chopsticks 5 having a notification function) with which the temperatures of food/drink items regarding a meal for a person who takes the meal can be measured, food/drink item temperature information which is information regarding the temperatures of the food/drink items; a food/drink item content obtaining unit 113 which obtains food/drink item content information which is information regarding the kinds and/or the amounts of the food/drink items, from a food/drink item detection unit 112 which can detect the kinds and/or the amounts of the food/drink items; and a feedback unit 115 which selects or generates feedback information regarding the eating behavior of the person taking the meal according to the food/drink item temperature information and the food/drink item content information using criteria or models for feedback, based on the temperatures and contents of the food/drink items, which have been predetermined, and provides the feedback information.SELECTED DRAWING: Figure 1

Description

TECHNICAL FIELD The present invention relates to a technique for collecting and analyzing information related to eating behavior, particularly information related to ingested food and drink.

In recent years, due to an increase in the number of elderly people and an increase in health consciousness, efforts to review daily eating behavior and improve eating habits are being actively carried out.

For example, Patent Literature 1 discloses an eating habit management device that enables users to check their own eating habits. This eating habits management device includes a eating habits information transmitting unit having a chopstick detecting unit for detecting that chopsticks are placed on a chopstick rest, and another eating habits information transmitting unit having a mastication information detecting unit installed in a mastication frequency measuring device. and a dietary information processing unit having control means for collecting and processing dietary information from these dietary information transmission units.

Patent document 2 also discloses an eating habits management program that enables users to check their own eating habits. This eating habits management program comprises steps of acquiring image data by the camera unit, recognizing an image of eating utensils from the acquired image data by image recognition processing, and recognizing a user's An information communication terminal, which is a computer equipped with a camera section, is caused to execute dietary management processing having a step of detecting dietary information, which is information on dietary habits, and a step of collecting and processing the detected dietary habits information. It is a thing.

Furthermore, Japanese Patent Application Laid-Open No. 2002-200001 discloses a chopstick motion feature determination device that can determine how chopsticks are moving. Specifically, this device calculates the relative acceleration from the fixed chopstick acceleration and the acting chopstick acceleration, and based on the relative acceleration pattern showing the difference in the relative acceleration for each axis direction, the fixed chopstick and the action when the relative acceleration is calculated. A chopstick motion feature determination unit is provided for determining which of a plurality of predetermined motion features the chopsticks are operating in, thereby measuring the eating behavior from the acceleration of the chopsticks. It is also possible to correct the way you hold chopsticks.

Further, Patent Literature 4 discloses a tableware for promoting mastication provided with a timer, which makes it possible to maintain a constant mastication time during eating. Specifically, in chopsticks as tableware for promoting mastication, a timer is activated when the chopsticks or the like reach a predetermined inclination when bringing food to the mouth. A message indicating that mastication should be continued is displayed on the display unit to prompt the user to hold the mastication time.

Furthermore, Patent Document 5 describes image recognition of a user's eating and drinking from a database that prohibits eating and drinking mainly in relation to medicine, according to the user's health condition. A food intake management method is disclosed that uses technology to detect and notify when a prohibited food is to be consumed.

Further, Patent Document 6 discloses a cooking assistance method that reduces the risk of food poisoning in each meal provided to each recipient of the meal when a plurality of meals are cooked simultaneously and individually by machine. Specifically, this cooking support method selects one serving tray, measures the tableware target temperature, which is the surface temperature of each tableware on this one serving tray, and measures the core temperature of the ingredients in each tableware. a step of confirming that the core temperature of the ingredients in each dish satisfies the food target temperature condition, which is a temperature condition to be met during cooking; When the step of measuring and the surface temperature of each dish on the other serving tray satisfies a predetermined condition based on the dish target temperature for the dish placed at the same position on one serving tray, the surface temperature was measured. and determining that the foodstuff in the tableware satisfies the food target temperature condition.

Further, Patent Literature 7 discloses an information processing device that enables customers to eat and drink food at an optimum temperature in a restaurant. Specifically, this device uses sensor identification information for identifying a temperature sensor capable of detecting the temperature of food and drink provided to customers in a restaurant, and customer identification information for identifying the table or seat of the customer where the food and drink is provided. A storage means for storing information in association with each other is communicable with a store terminal provided in the restaurant or a table terminal provided at the table or seat, and the output value of the temperature sensor is stored as the sensor identification information. and a communication means capable of receiving the above-mentioned output value, based on the change in the received output value, the above-mentioned and a control means for transmitting to the store terminal before the temperature of the food changes to a temperature unsuitable for eating and drinking.

Furthermore, Patent Literature 8 discloses a food or beverage providing method that enables a restaurant to serve the next dish or beverage in accordance with the progress of each customer's meal while reducing labor costs. A method is disclosed. Specifically, in this method, an image capturing step for capturing an image of the food or drink present on the dining table in a restaurant, and image information captured by the image capturing means are sent from the store side to the customer side whether or not there is a request for additional service. image information providing step for providing to the store side as information to be used for determining the timing for inquiring.

In addition, as an embodiment of this food or beverage serving method, in order to support the timing of serving food at an appropriate temperature, an infrared camera is used to measure the temperature of the plate on the dining table, and when the temperature of the plate is the same as room temperature There is also proposed a method of assisting the timing so that the next dish is offered to the waiter or the like when it is determined that the customer has no intention of eating.

JP 2014-097223 A JP 2015-146168 A JP 2015-225277 A JP-A-2000-270993 JP 2018-181385 A JP 2020-010835 A JP 2018-128818 A JP 2004-252497 A

As described above, conventionally, various techniques have been proposed for monitoring eating behavior and, based on the information obtained, realizing a suitable diet and promoting the health of the diners. However, the actual situation is that these prior arts still do not properly grasp the actual contents of the food and drink provided, especially the temperature, and provide accurate feedback for the diners.

For example, with the techniques disclosed in Patent Document 1 and Patent Document 2, it is possible to obtain information about what kind of food and drink is consumed, how long it takes, and how many times it is chewed. temperature information cannot be obtained. In this regard, the techniques disclosed in Patent Document 3 and Patent Document 4 are the same, and it is possible to collect various information, such as the timing of feedback, from the movement of chopsticks during a meal. Acquisition of information on the temperature of food and drink is not assumed in the first place.

Furthermore, in the technology disclosed in Patent Document 5, the type of food and drink can be identified using image recognition technology, and information mainly related to eating with medicine can be reported. It is not possible to obtain information about temperature.

On the other hand, it is true that the technique disclosed in Patent Literature 6 acquires temperature information of food and drink using a thermal imaging camera and generates information for reducing the risk of food poisoning. However, it is not possible to recognize contents other than the temperature of the provided food, for example, its type, and it is not assumed to perform temperature management suitable for the content of food, for example, its type. In fact, in this technology, the processing is carried out using the given target food temperature conditions.

In this respect, the technology disclosed in Patent Document 7 is the same, and although it is possible to detect the temperature of food and drink provided to customers at a restaurant, the content other than the temperature of the food and drink provided is detected. For example, the type is not recognized, and therefore, information in the menu information database set in advance is required when, for example, the notification of eating and drinking promotion considering the menu is performed.

Furthermore, in the technology disclosed in Patent Document 8, the temperature of food and drink provided to customers in a restaurant can be detected. nothing has been done to recognize In addition, therefore, for example, even when the next dish or beverage is automatically cooked, served, or served, it is necessary to use the information from the dish database and the order content recording unit for each table. .

Incidentally, in an actual meal, the temperatures of various types of food and drink usually have an appropriate temperature range for each type. For example, ingestion of hot food whose temperature is within the appropriate temperature range not only contributes to the realization of a delicious and suitable meal, but also contributes to the health promotion of the diner. In other words, it is highly expected to have the effect of warming the cold body and improving metabolism and immunity. However, in the conventional techniques as described in Patent Documents 6 to 8, it is difficult to provide feedback considering the content of the provided food and drink, for example, the appropriate temperature range according to the type, without having knowledge of the content in advance. Unless you secure it, it is impossible.

Therefore, the present invention provides an eating behavior monitoring device, system, program and method that enable the temperature of food and drink provided to be grasped according to the contents thereof and to provide more accurate feedback for the diners. intended to

According to the present invention, food temperature acquisition means for acquiring food temperature information, which is information relating to the temperature of food and drink, from temperature measuring means capable of measuring the temperature of food and drink related to the meal of the diner;
food content acquisition means for acquiring food content information, which is information relating to the type and/or amount of the food and drink, from a recognition means capable of recognizing the type and/or amount of the food and drink;
Selecting or selecting feedback information related to the eating behavior of the diner based on the food temperature information and the food content information using preset feedback criteria or models based on food temperature and content An eating behavior monitoring device is provided having means for generating and providing feedback.

As an embodiment of the eating behavior monitor device according to the present invention, the food content acquisition means acquires the food content information related to at least the type of the food of the diner,
The feedback means predicts information about the future temperature of the type of food and drink based on at least the food and drink content information about the type and the food and drink temperature information, and feeds the predicted future temperature information to the food and drink. Based on this, it is also preferable to select or generate the feedback information that includes information regarding the timing, time and/or order in which the type of food and drink should be taken.

Further, in the above embodiment, the food and drink temperature acquisition means also acquires environmental temperature information, which is information relating to the temperature of the eating environment of the diner, from the temperature measurement means or from another temperature measurement means,
Preferably, the feedback means also predicts information about the future temperature of the type of food and drink based also on the environmental temperature information.

Furthermore, in the above-described embodiment, the eating behavior monitor device provides eating behavior-related information that is information about the past eating behavior of the diner, and is related to at least the temperature when the food of the type was ingested. further comprising eating behavior information generating means for receiving the eating behavior related information from the storing means storing the eating behavior related information including the information and generating eating behavior information summarizing the eating behavior related information;
The feedback means determines the timing, time, and/or at which the type of food and drink included in the feedback information should be taken, based on the information on the temperature at which the type of food and drink is ingested, which is included in the eating behavior information. Alternatively, it is also preferable to adjust the information regarding the order according to the diner concerned.

In addition, in the above embodiment that handles eating behavior information, the eating behavior information also includes information related to the time when the type of food and drink was ingested,
The feedback means, based also on the information on the time at which the type of food and drink is ingested, feeds the information on the timing, time and/or order in which the type of food and drink included in the feedback information should be taken into the meal. It is also preferable to adjust according to the person.

Furthermore, in the above-described embodiment that deals with eating behavior information, the eating behavior information is generated from the output of a pre-installed myoelectric sensor capable of detecting the mastication of the diner who has consumed the type of food and drink. Including information related to chewing behavior at the time,
The feedback means provides information on the timing, time and/or order in which the type of food and drink included in the feedback information should be taken, based on the information on the chewing behavior when the type of food and drink is ingested. , it is also preferable to adjust according to the diner concerned.

Furthermore, in the above-described embodiment that deals with eating behavior information, the eating behavior information is a quality index of the eating behavior of the diner, which is an index related to the appropriateness of the provided food and drink An index related to the degree to which the food and drink was finished while it was still at the appropriate temperature, an index to the degree to which the meal was finished while the food and drink were at the appropriate temperature, and an index to the number of types of the food and drink provided. and/or quality indicators of eating behaviour, including indicators of the adequacy of the dieter's dietary environment,
The feedback means adjusts the information related to the timing, time and/or order in which the type of food and drink included in the feedback information should be ingested to suit the diner based also on the quality index of the eating behavior. is also preferred.

Further, in the eating behavior monitoring device according to the present invention, the temperature measuring means may be a thermography installed in the dining environment of the diner, a thermometer mounted on a utensil used for the meal of the diner, and/or It is also preferable that the thermometer is mounted on the tableware for the meal of the diner.

Furthermore, in the above-described embodiment of predicting information about the future temperature of food and drink, the feedback means is based on the information about the timing, time and/or order in which the selected or generated type of food and drink should be ingested. , generating control information for controlling the irradiation of light to the type of food and drink by a light irradiation means installed in advance capable of irradiating light to the food and drink, and outputting the control information to the light irradiation means or Sending is also preferred.

Furthermore, in the above embodiment of predicting information about the future temperature of food and drink, the feedback means is based on the information about the timing, time and/or order in which the selected or generated type of food and drink should be taken. a light source pre-installed in the utensils used for the meal of the diner, a vibrator pre-installed in the utensils, a light source pre-installed in the tableware used for the meal of the diner, and/or the tableware It is also preferable to generate control information for controlling the notification operation to the diner by a pre-installed vibrator, and to transmit the control information to the light source and/or the vibrator.

Further, in the above-described embodiment that predicts information about the future temperature of food and drink, the feedback means is based on the information about the timing, time and/or order in which the selected or generated type of food and drink should be taken. , generating control information for controlling the operation of cooling and/or heating the food by a cooling device and/or a heating device pre-mounted on tableware related to the meal of the diner, and cooling the control information Transmission to a device and/or a heating device is also preferred.

Further, in the eating behavior monitoring device according to the present invention, the recognition means acquires information related to an image including the food and drink from a pre-installed camera capable of photographing the food and drink, and based on the information related to the image, , it is also preferable to recognize the type and/or amount of the food or drink. Further, here, the main eating behavior monitoring device is an information processing device equipped with a recognition means capable of receiving information related to the image from the above-described camera installed outside, and the camera and the recognition means. A tablet computer, a smart phone with such a camera and recognition means, or an Augmented Reality (AR) type smart glasses with such a camera and recognition means are also preferred.

According to the present invention, a temperature measuring means capable of measuring the temperature of the food and drink related to the meal of the diner,
food temperature acquisition means for acquiring food temperature information, which is information relating to the temperature of the food, from the temperature measurement means;
a recognition means capable of recognizing the type and/or amount of the food and drink;
Food and drink content acquisition means for acquiring food and drink content information, which is information relating to the type and/or amount of the food and drink, from the recognition means;
Selecting or selecting feedback information related to the eating behavior of the diner based on the food temperature information and the food content information using preset feedback criteria or models based on food temperature and content An eating behavior monitoring system is provided having a means for generating and providing feedback.

According to the present invention, there is further provided a program for operating a computer that monitors eating behavior of a diner, comprising:
food temperature acquisition means for acquiring food temperature information, which is information relating to the temperature of food and drink, from temperature measuring means capable of measuring the temperature of food and drink related to the meal of the diner;
food content acquisition means for acquiring food content information, which is information relating to the type and/or amount of the food and drink, from a recognition means capable of recognizing the type and/or amount of the food and drink;
Selecting or selecting feedback information related to the eating behavior of the diner based on the food temperature information and the food content information using preset feedback criteria or models based on food temperature and content An eating behavior monitor program is provided that causes a computer to act as a means of generating and providing feedback.

According to the present invention, there is further provided a method of monitoring eating behavior in a computer for monitoring the eating behavior of a diner, comprising:
Acquiring food temperature information, which is information relating to the temperature of the food and drink, from a temperature measuring means capable of measuring the temperature of the food and drink related to the meal of the diner, and recognizing the type and/or amount of the food and drink obtaining food and drink content information, which is information relating to the type and/or quantity of the food and drink, from a possible recognition means;
Selecting or selecting feedback information related to the eating behavior of the diner based on the food temperature information and the food content information using preset feedback criteria or models based on food temperature and content A method of monitoring eating behavior is provided, comprising the steps of generating and providing.

According to the eating behavior monitoring device, system, program, and method of the present invention, it is possible to grasp the temperature of the provided food according to its contents, and to provide more accurate feedback for the diner.

1 is a schematic diagram showing an embodiment of an eating behavior monitoring system including an eating behavior monitoring device according to the present invention; FIG. It is a graph for demonstrating an example of the temperature prediction process result in the food-and-drinks temperature prediction part which concerns on this invention. It is a table for demonstrating one aspect|mode of the content in the food-and-drink optimal temperature database (DB) which concerns on this invention. It is a table for demonstrating one aspect|mode of the content in food-and-drink combination DB which concerns on this invention. 1 is a flow chart showing an embodiment of an eating behavior monitoring method including calculation of a QoF index according to the present invention;

EMBODIMENT OF THE INVENTION Below, embodiment of this invention is described in detail using drawing.

[Eating behavior monitoring system and device]
FIG. 1 is a schematic diagram showing an embodiment of an eating behavior monitoring system including an eating behavior monitoring device according to the present invention.

The eating behavior monitor system of this embodiment shown in FIG.
(a) a visible light camera 2 capable of photographing food and drink placed on the dining table and generating image (video) information including the food and drink;
(b) a thermography camera 3 capable of photographing food and drink placed on the dining table and generating infrared image (video) information including the food and drink;
(c) a projector 4 capable of projecting feedback information (including a spotlight-like image (video) specifying food and drink) at a specified position among the positions of the food and drink served on the dining table;
(d) Chopsticks 5 with a notification function, which are equipped with thermometers at the tip of the food sandwich and the end on the opposite side, and further include a vibrator for notification and a lighting lamp;
(e) a myoelectric measuring device 6 attached to the head of the diner, capable of measuring the chewing motion of the diner;
(f) Information from the visible light camera 2, thermography camera 3, chopsticks with notification function 5, and myoelectric measurement device 6, for example, by short-range wireless communication such as Bluetooth (registered trademark) or Wi-Fi (registered trademark) Alternatively, it has an eating behavior monitor device 1 capable of providing "feedback information" relating to the eating behavior of the diner by obtaining it through wired communication.

Here, in the present embodiment, the eating behavior monitor device 1 analyzes the image information acquired from the visible light camera 2 of (a) above using a known image recognition technology, and It has a food/drink recognition unit 112 capable of recognizing one or both of the quantity (at least the type in this embodiment).

In addition, in this embodiment, the eating behavior monitor device 1 analyzes the infrared image information acquired from the thermography camera 3 of (a) above by a known thermography analysis technology, and determines the temperature of each food and drink served on the table, It also has a food/drink/environmental temperature acquisition unit 111 capable of generating information related to the ambient temperature. In this embodiment, the food/beverage/environmental temperature acquisition unit 111 acquires information related to the temperature of each food and the environmental temperature of its surroundings from the notification function-equipped chopsticks 5 described in (d) above. You can also

Furthermore, in the present embodiment, the eating behavior monitoring device 1 displays the above (f) "feedback information" on the display (DP) 107, as audio from the speaker 109, or (near-field wireless communication or wired communication). Feedback information can be provided by causing the projector 4 to project feedback information on predetermined food and drink (via communication), and by causing the chopsticks with notification function 5 to drive a vibrator or a lighting lamp.

Specifically, in order to realize the provision of this "feedback information", the eating behavior monitor device 1 has the following features:
(A) From the temperature measuring means (the thermography camera 3 and the chopsticks with notification function 5 in this embodiment) capable of measuring the temperature of the food and drink related to the meal of the diner, information related to the temperature of the food and drink the above-mentioned food and drink temperature acquisition unit 111 that acquires "temperature information";
(B) A food and drink content acquisition unit that acquires "food and drink content information" that is information related to one or both of the type and amount of the food and drink (at least the type in this embodiment) from the food and drink recognition unit 112 described above. 113 and
(C) Based on the "food temperature information" and "food content information", using a preset feedback criterion or model based on food temperature and content, and a feedback unit 115 that selects or generates and provides "feedback information".

In this way, the eating behavior monitor device 1 can acquire the "food temperature information" about food and drink together with the "food and drink content information" about the food and drink, not alone. Considered "feedback information" can be selected or generated. In other words, it is possible to grasp the temperature of the provided food and drink according to its content, and to provide more accurate feedback for the diners.

For example, the eating behavior monitoring device 1 grasps the type and temperature of each food and drink, including the soup, and first selects hot soup whose temperature is within the appropriate temperature range for health or for a suitable meal. After generating "feedback information" indicating that the soup should be taken, the projector 4 projects a spotlight-like image (video) onto the soup as the feedback information, and then the display 107 displays the message "Please have the soup first." A message can be displayed, and furthermore, a voice saying "Please have soup first" can be output from the speaker 109. FIG.

Here, the “type” of the food and drink in the “food and drink content information” in (B) above may be a dish name (menu name) such as miso soup or steak, or a dish name such as noodles or soup. It can also be a genre. For example, the genre may be used for a food or drink that is difficult to identify with a dish name (menu name) in the food and drink recognition unit 112 . Furthermore, the "amount" of food and drink may be the amount in each food and drink, but alternatively or in addition, it is also possible to refer to the number of food and drink, such as the number of plates and the number of dishes.

In addition, in the present embodiment, the eating behavior monitoring device 1 also uses myoelectric signal information as the eating behavior-related information acquired from the myoelectricity measuring device 6 described in (e) above to monitor the QoF (Quality of Feeding behavior). It is also possible to generate "eating behavior information" including behavioral quality) indicators. Here, the QoF index, which will be described in detail later, is an index devised by the inventor of the present application, and serves as an index for evaluating the quality of eating behavior in consideration of the exact temperature of food and drink ingested. ing.

In this embodiment, the eating behavior monitoring device 1 is a tablet computer that can be easily installed on the dining table as appropriate. Alternatively, or in combination with the visible light camera 2, image information may be acquired.

As a modification, the food and drink recognition unit 112 is provided in the visible light camera 2 or in an external information processing device to which the visible light camera 2 is connected. It may be obtained by receiving "food and drink content information".

Further, as a modification, the eating behavior monitor device 1 can be augmented reality (AR) type smart glasses equipped with a camera (capable of photographing food and drink during a meal) and a food and drink recognition unit 112. . In this case, the "feedback information" may be displayed on the display arranged in front of the user by mapping the food and drink on the dining table. In addition, it is also possible to give the function of the electromyography measuring device 6 to this AR type smart glasses. As a result, diners can provide information that is the basis of "eating behavior information" simply by wearing this AR type smart glasses, and it is also possible to enjoy more accurate "feedback information". It becomes.

Furthermore, the present eating behavior monitoring system has the above-described configurations (a) to (f) in this embodiment, but of course all of them are not essential, and at least temperature measuring means capable of measuring the temperature of food and drink. and recognition means capable of recognizing the type and/or amount of the food and drink. Of course, if the recognition means requires image information including the food and drink, a camera capable of photographing the food and drink is also required.

Here, general knowledge about the importance of the temperature of food and drink in meals will be described. As it is said that medicine and food are the same source, a suitable diet is indispensable for maintaining and improving health. The basis of this good diet is to eat hot food while it is warm and cold food while it is cold.

In addition, it is said that the lower the temperature of the food or drink ingested by humans, the less sweet and umami are perceived, while the more bitter and salty are perceived. Therefore, there is an optimum temperature range for feeling delicious food as a balance of these tastes, and in general, the optimum temperature range is 60 to 70 ° C for warm food and 5 to 12 ° C for cold food.

Furthermore, as in hot soups and stews, hot ramen and steamed pork buns, as well as cold cold tofu, cold juices and beer, etc., ingesting them at a predetermined appropriate temperature is an essential condition for feeling deliciousness. There are many things. As explained above, the temperature of food and drink at the time of eating is an important parameter to be monitored in order to achieve a meal that is both delicious and healthy.

[Device configuration, eating behavior monitor program]
Similarly, in the functional block diagram of FIG. , a food and drink optimum temperature database (DB) 105, a food and drink matching DB 106, a touch panel display (TP/DP) 107, a microphone 108, a speaker 109, and a processor memory.

Here, the processor memory stores an eating behavior monitor program according to an embodiment of the present invention, and has a computer function. Take action. Therefore, although the eating behavior monitoring device according to the present invention may be a tablet computer as in the present embodiment, it can also be used in a cloud server, a non-cloud server, a personal computer, etc., on which the eating behavior monitoring program according to the present invention is installed. (PC), smartphones, notebook computers, and even augmented reality (AR) type smart glasses. .

Further, the processor/memory includes a food/drink/environment temperature acquisition unit 111, a food/drink recognition unit 112, a food/drink content acquisition unit 113, and a QoF (Quality of Feeding behavior) calculation unit 114a. a behavior information generation unit 114; have These functional components can be regarded as functions of an eating behavior monitor program stored in a processor memory. In addition, the flow of processing shown by connecting the functional components of the eating behavior monitoring device 1 with arrows in the functional block diagram of FIG. 1 can also be understood as an embodiment of the eating behavior monitoring method according to the present invention.

Similarly, in the functional block diagram of FIG. Environmental temperature information, which is information related to the temperature of the environment, is also obtained. Further, food temperature information and environment temperature information may be obtained from a thermometer mounted on tableware used by the diner, a room thermometer, or the like. In any case, the information related to the temperature of the food and the temperature of the eating environment collected from such temperature measuring means via the communication interface 101 and the communication control unit 121 is, for example, food temperature information. It is also preferable that the food/beverage/environmental temperature acquisition unit 111 appropriately receives such information stored in the storage unit 102 and adjusts/manages it as food/beverage temperature information and environmental temperature information.

Here, the thermography camera 3 can output, for example, the temperature of each pixel in the infrared image within the angle of view every second as time-series temperature information. The food/drink/environment temperature acquisition unit 111 receives the time-series temperature information from the food/drink temperature information storage unit 102, and uses a known thermography analysis technique to obtain
(a) Considering a cluster of pixels having a temperature higher (or lower) than the temperature of the surrounding pixels by a predetermined value or more as one piece of food and drink, the highest (or lowest) value in the pixel group regarded as one piece of food and drink The temperature is the temperature of the food and drink, and the information obtained by linking the temperature with the food and drink identifier and the time is the food and drink temperature information of the food and drink,
(b) The temperature of the pixels around the pixel group regarded as food and drink (for example, the temperature of the dining table surface) can be used as the environmental temperature, and the information linking the environmental temperature and the time can be used as the environmental temperature information.

In addition, according to the above information, the food/drink/environment temperature acquisition unit 111 may, for example,
(c) The number of pixel groups regarded as one food and drink is the number of food and drink types (for example, the number of dishes and the number of plates),
(d) The number of pixels constituting one food item may be the amount of the food item (for example, the amount served on a plate). It may be provided to the content acquisition unit 113 .

In addition, the food/drink/environment temperature acquisition unit 111
(e) information on the maximum or minimum temperature (maximum change from the environmental temperature) acquired from the thermometer at the tip of the chopsticks 5 with notification function to hold the food and drink; as the temperature, the information that links this temperature with an identifier related to the order (peak or trough) and time is taken as food and drink temperature information of food and drink (to be ingested using chopsticks with notification function 5),
(f) The temperature (e.g., the temperature corresponding to the air temperature) obtained from the thermometer at the end of the chopsticks with notification function 5 on the opposite side of the tip of the food is used as the environmental temperature, and this environmental temperature and time are linked. It is also preferable to use the information obtained as environmental temperature information. For example, (ingestion time, (ingestion order identifier), food identifier, food temperature, environmental temperature) may be generated as a single piece of ingestion event information that combines food temperature information and environmental temperature information. In addition, it is possible to recognize that the eating behavior of one mouthful of the diner has occurred at one peak or trough that forms the maximum or minimum in the time-series information related to the temperature acquired from the chopsticks with notification function 5. be.

Furthermore, as a preferred mode of acquiring food temperature information using chopsticks 5 with a notification function, each food (recognized by a food recognition unit 112 described later) is sequentially displayed on the display 107 of the eating behavior monitor device 1, for example. The temperature of each food (at the beginning of intake) is measured by designating and showing the food, and the diner who sees it touches the designated food and drink sequentially before eating with chopsticks 5 with a notification function. It is also preferable to Further, at this time, the temperature measurement may be set to be skipped for foods and drinks that are clearly at room temperature.

For example, a weighted average is calculated between the food temperature information (or environmental temperature information) acquired from the chopsticks with notification function 5 and the food temperature information (or environmental temperature information) acquired from the thermography camera 3. The final food and drink temperature information (or environmental temperature information) may be determined by taking the Further, in this case, the correspondence relationship of the food and drink to be averaged may be determined using, for example, analysis results of image (video) information from the visible light camera 2, which will be described later (for example, order of intake, intake time, etc.). good.

Furthermore, it is also preferable that the thermometer mounted on chopsticks 5 with notification function is a compact temperature sensor with good temperature response. In addition, in order to improve temperature response, the main body of chopsticks with notification function 5 is made of a material with relatively low thermal conductivity, such as titanium or stainless steel, among metals that generally have high thermal conductivity. Although the heat of food and drink is easily transferred to the thermometer, heat conduction may be limited between the thermometer (food and drink) and the gripping portion of the chopsticks 5 . Furthermore, it is also possible to adopt a spoon, a fork, etc. instead of or in combination with the chopsticks 5 as a thermometer-equipped eating utensil. For example, a spoon equipped with a thermometer may also incorporate a known miniature strain sensor to measure the weight of the food being consumed. The measured weight of the food is included in the food content information of the food.

Similarly, in the functional block diagram of FIG. (Video) information is acquired, and based on the image information, one of the type and amount of each food and drink served on the table by a known image recognition technology (for example, using a learned food and drink type identification model) Or both (in this embodiment, at least the type) are recognized. Here, of course, it is also possible to recognize the number of types of food and drink (the number of dishes and the number of plates) served.

Here, the food and drink recognition unit 112 acquires image (video) information including food and drink generated by a camera mounted on the eating behavior monitor device 1 installed on the dining table, and recognizes the type of each food and drink. One or both of (for example, dish name) and quantity (for example, the number of dishes, the number of dishes, and the amount served on each dish) (at least the type in this embodiment) may be recognized.

In addition, the food and drink recognition unit 112 compares the acquired image information with the infrared image information acquired from the thermography camera 3, and based on the positions of the corresponding pixels of both, the food and drink determined from the infrared image information. It is also preferable to specify the food corresponding to (and its temperature) on the image information, and assign the food and drink identifier common to the food and drink temperature information to the recognized food (type).

From the food recognition unit 112, the food content acquisition unit 113 obtains the type (for example, the name of the dish) and the amount (for example, the number of dishes, the number of plates, and the amount served on each plate) of each food with an identifier. Information relating to one or both (in this embodiment, at least the type) is acquired as food and drink content information. The food/drink recognition unit 112 determines not only the type of each food/drink but also the nutritional components of each food/drink (using, for example, a nutritional component database prepared in advance). Food and drink content information may also be obtained that also includes nutritional component information for the product.

Similarly, in the functional block diagram of FIG. Furthermore, myoelectric signal information (for example, information in which the preprocessed myoelectric signal is linked to the measurement time) is received from the myoelectric measurement device 6 via the communication interface 101 and the communication control unit 121, and this information is The information is stored as food-related information (including at least food temperature information) that is information related to the past eating behavior of the diner.

The eating behavior information generating unit 114 receives the eating behavior related information from the eating behavior related information storage unit 104, and generates “eating behavior information” that summarizes the eating behavior related information. Specifically, this "eating behavior information" can be, for example, an eating behavior log that organizes and records the details of the eating behavior for each meal of each diner. In addition, as the contents of this eating behavior,
(a) ambient temperature of the meal;
(b) the time each food was taken, the order of the food taken, the meal time (meal start time and meal end time), the total number of mastications, the mastication pace, the total number of mouthfuls, and (c) the ingested It is also preferable that the type (dish name), temperature, intake amount, time taken for one mouthful of food, and the number of chewing times for each food and drink are recorded.

Here, the information related to mastication in the above (b) and (c) is an invention by the inventor of the present application based on the acquired myoelectric signal information. It may be determined by the biological signal processing method described in JP-A-2019-107067 and JP-A-2018-139630, or by other known mastication measurement methods. In addition, from the eating behavior information, which is a log of eating behavior over a long period of time, information such as the temperature range in which a diner prefers to take food and the relationship between the temperature of the food and drink to be ingested and the environmental temperature is also known. It is possible to obtain

Further, in the present embodiment, the QoF calculation unit 114a of the eating behavior information generation unit 114 calculates "QoF (Quality of Feeding behavior)" as information included in the eating behavior information from the food and drink information as described above. It is also preferred to generate an index. This QoF index is an index devised by the inventor of the present application, and serves as an index for evaluating the quality of eating behavior in consideration of the exact temperature of ingested food and drink. Calculation of this QoF index will be described later in detail using FIG.

<Feedback Means>
Also in the functional block diagram of FIG.
(a) food and drink temperature information taken in from the food and drink/environment temperature acquisition unit 111;
(b) Based on the food content information acquired from the food content acquisition unit 113, it selects or generates and provides feedback information related to the eating behavior of the diner.

Here, the feedback information to be provided may be selected by searching from a feedback database in which various types of feedback information are stored in advance. In this case, the feedback database stores feedback information data arranged for each item related to the temperature and content of the food and drink, and outputs the corresponding feedback information data using a search key related to the temperature and content of the food and drink. It can be thought of as a “model” of possible feedback offerings.

In addition, the feedback information to be provided is based on various feedback information templates arranged for each item related to the temperature and content of the food and drink based on the information related to the temperature and content of the food and drink as a "standard" for reading. It may be generated by reading out and applying relevant information related to feedback to this template. Further, the feedback information provided may be determined from the temperature and content of the food or drink using a trained machine learning "model" that outputs an identifier for the feedback information to be provided.

Further, in the present embodiment, each food to which its own temperature is given in the food temperature information and each food to which its own type and amount are given in the food content information are common as described above. is given an identifier. Therefore, the feedback unit 115 can grasp the type (for example, the name of the dish), the temperature, and the amount of each food served on the table from the food temperature information and the food content information.

(Feedback using predicted temperature information)
As one of preferable feedback processes, the feedback unit 115 predicts information about the future temperature for each food (for each type of food), and outputs predicted temperature information, which is information about the predicted future temperature. may select or generate feedback information including information on at least one of the timing, time and order in which the (type of) food and drink should be taken. Here, at the time of this prediction, it is also preferable to generate predicted temperature information by predicting information related to the future temperature of the food and drink based on the environmental temperature information included in the food and drink temperature information. An aspect of the predicted temperature information generation process performed by the food and drink temperature prediction unit 115a of the feedback unit 115 will be described below.

In general, the temperature T(t) that the food, which was originally at temperature T0, takes after t seconds has elapsed under the environmental temperature Te is given by the following equation (1) T(t)=(T0−Te)×exp(− t/τx) + Te
is represented by Here, τx is the thermal time constant in the set unit time. If the above equation (1) is transformed into a form for obtaining this thermal time constant τx,
(2) τx = -t/ln {(T(t)-Te)/(T0-Te)}
becomes. Therefore, if the initial temperature T0 of the food and drink, the temperature T(t) after t seconds have elapsed, and the environmental temperature Te are known, the thermal time constant τx of the food and drink can be determined using the above equation (2). can. Furthermore, by using the determined thermal time constant τx and the above equation (1), the temperature of the food after a predetermined time (for example, 30 minutes) has elapsed can be calculated to generate predicted temperature information. of.

Note that the thermal time constant τx can be obtained without using the above equation (2), for example, by preparing a database in which thermal time constant values are recorded in advance for each type of food and drink. However, even for the same dish, the actual thermal time constant value is affected by the physical properties of the ingredients, the physical properties of the tableware on which it is served, and so on, and there are many cases in which it deviates from this previously recorded value. Therefore, it is preferable to determine the thermal time constant τx by the method described above.

FIG. 2 is a graph for explaining an example of the temperature prediction processing result in the food and drink temperature prediction unit 115a.

FIG. 2 shows a graph of the predicted temperature information for hot food A and cold food B that have been left at ambient temperature (30°C), that is, the relationship between predicted temperature (°C) and elapsed time (minutes). It is Both predicted temperatures show exponential changes approaching the ambient temperature (30°C). Here, the appropriate temperature range for cold food B is set to 0-15°C, while the appropriate temperature range for hot food A is set to 50-75°C.

Therefore, from the appearance of each temperature change, cold food B is initially within the appropriate temperature range, but hot food A is too hot beyond the appropriate temperature range. It can be seen that the optimum temperature range is exceeded before entering. Therefore, in this case, the feedback unit 115 can select or generate feedback information to the effect that "the cold food B is recommended to be taken before the hot food A", for example.

Regarding the temperature of the food and drink, for example, the temperature of the food and drink measured changes due to the change in heat distribution caused by stirring the food and drink each time it is ingested, and the temperature of the food and drink changes every time it is ingested. When the food is torn and the amount and shape of the food is changed, the thermal time constant of heat dissipation and heat absorption is often changed. Therefore, it is preferable that the food/drink temperature prediction unit 115a again perform the temperature prediction process based on the newly measured temperature of the food/drink without depending on the initial temperature prediction result after the ingestion behavior has occurred. For example, every time a predetermined period of time elapses (at a predetermined cycle), based on the temperature of the food at that time, the prediction of the time during which the temperature of the food is within the appropriate temperature range may be updated.

In addition, the food temperature prediction unit 115a detects "outliers" and "change points" that exceed a predetermined error of the temperature measurement means regarding changes in food temperature even when no intake behavior is detected. It is also preferable to perform the temperature prediction process again from the newly acquired temperature of food and drink.

By the way, as an "outlier", if the "difference" between the predicted value and the actual measurement of the temperature of food and drink exceeds a predetermined threshold in comparison with the dispersion of the difference in the past predetermined period, this A measured value for the "difference" can be an outlier. For example, the mean squared error RMSE is adopted as the variation of the difference in a predetermined period in the past, and if the above "difference" exceeds twice the value of RMSE as a predetermined threshold, A measured value may be an outlier. Furthermore, it is also possible to determine whether it is an "outlier" based on the relationship between the distance to the past k measured values and the latest measured value using the k-nearest neighbor method. is. In any case, both "outliers" and "change points" can be detected by known statistical techniques.

Returning to FIG. 1, specific contents of the feedback processing in the feedback unit 115 will be described below.

For example, when the feedback unit 115 determines that the served food A is too hot beyond its appropriate temperature range, for example, "Do not take the food A until it cools down a little more (for example, until it becomes less than 70 ° C.). You may select or generate feedback information stating that you should refrain from or be careful when ingesting. By providing such feedback information to the diners, it is possible to prevent the occurrence of burns in the oral cavity and the future occurrence of oral cancer due to repeated thermal stimulation of the oral cavity. .

Further, the feedback unit 115 may select or generate feedback information that notifies information on the appropriate temperature range of each food and drink served. As a result, the diner who receives this feedback information can take each food and drink at an appropriate temperature, and can practice delicious and healthy meals.

In this embodiment, the feedback unit 115 can obtain the appropriate temperature range of the food from the appropriate food temperature DB 105 using the food type (for example, dish name, menu name, genre) as a search key. Here, as shown in FIG. 3, the appropriate food temperature DB 105 includes:
(a) For each menu name, a menu/ingredient temperature DB that records and manages the cuisine genre, ingredients, and appropriate temperature range of the menu name in association with each other, and (b) For each cuisine genre, the genre A Genre Appropriate Temperature DB that records and manages appropriate temperature ranges for foods and drinks in association with each other
It is equipped with

By the way, in the case of a menu name such as "meat bun", the internal temperature is considerably higher than the outside, and it is difficult to determine whether it is within the appropriate temperature range by measuring the temperature from the outside. As shown in the "prediction" column of , "difficult to predict" may be registered in the corresponding column of the menu/ingredient temperature DB. Further, when information on both the name of the dish and the genre is reliably obtained in the processing of the food and drink recognition unit 112, the information on the appropriate temperature range in the menu/ingredient appropriate temperature DB is more accurate than the information on the appropriate temperature range in the genre appropriate temperature DB. Therefore, it is also preferable to set the information on the appropriate temperature range of the menu/ingredient temperature DB as the feedback information.

Further, for example, when the feedback unit 115 determines that the difference between the temperature of the originally warm food B served and the ambient temperature is less than a predetermined threshold value (for example, 10°C), the feedback unit 115 continues to consume the cold food. Since this can be stressful, feedback information may be selected or generated, for example, to the effect that "Food B is cold and should be stopped."

Furthermore, the feedback unit 115 may select or generate feedback information to the effect that the food to be taken first among them is suggested based on the type of each food and drink served. For example, feedback information to the effect that ``vegetable salad (rich in dietary fiber) should be eaten first in order to prevent overeating and sudden rises in blood sugar levels and prevent lifestyle-related diseases,'' and In order to increase meal-induced thermogenesis, which is a type of metabolism, and to promote digestion, absorption and metabolism, warm soup should be ingested first" may be selected or generated.

Further, for example, the feedback unit 115 uses the ingredients of each food and drink served as a search key to acquire information on the food and drink combination between the food and drink from the food and drink eating combination DB 106, and based on this food and drink combination information. Feedback information regarding the alignment may be selected or generated. Here, as shown in FIG. 4, the food and drink combination DB 106 stores, for each pair of material 1 and material 2, effects, effects, and preventable diseases by ingesting both materials (food and drink) together. etc. are associated with each other, recorded, and managed. Ingredients for the food and drink used as a search key here can be obtained from the appropriate food and drink temperature DB 105 using the type of the food and drink as a search key.

Here, for example, the feedback unit 115 identifies the food C and the food D that are well-cooked and both are in the appropriate temperature range at that time, among the food and drinks that were served, and "(to enhance the absorption of nutrients) It is also possible to select or generate feedback information to the effect that the food C and the food D should be taken together and before the food E that is still too hot."

Further, for example, the feedback unit 115 provides feedback information related to a warning to the effect that "the consumption of the food F may cause health problems" with respect to the food F that is too hot (or too cold) outside the appropriate temperature range. may be selected or generated. In addition, it is also possible to select or generate feedback information that encourages an ingestion behavior that is preferable from the viewpoint of health, taking into consideration both the temperature of the food and drink and the environmental temperature. For example, it is possible to select or generate feedback information to the effect that "it is cold this morning, so you should eat hot soup first" or feedback information to the effect that "it is hot today, you should eat cold cold tofu first".

Further, for example, the feedback unit 115 predicts a temporal change in the food temperature from the ambient temperature and the food temperature in the food temperature prediction unit 115a described above, and for example, "because the room temperature is currently below 20°C. , warm food G should be finished within 15 minutes." Further, for example, it is possible to use feedback information as information about how many more minutes it will take for the ice cream that has hardened due to being too cold to reach the proper temperature, and information about how many more minutes the ice cream will continue to be at the proper temperature.

(Feedback using eating behavior information)
Next, a mode of selecting/generating feedback information using the “eating behavior information” generated by the eating behavior information generation unit 114 will be described.

Specifically, the feedback unit 115, based on the information regarding the temperature of a certain type of food and drink when the diner ingests the food and drink, included in the eating behavior information, determines the temperature of the type of food and drink included in the feedback information. It is also preferable to tailor the information regarding the timing, time and/or order in which food and drink should be taken to the eater. In addition, when this eating behavior information also includes intake time information related to the time at which the type of food and drink was ingested, the feedback unit 115, based also on this intake time information, determines the type of food and drink included in the feedback information. Information regarding when, when and/or in what order the items should be taken may be tailored to the eater.

Furthermore, if this eating behavior information also includes information related to chewing behavior when the type of food and drink is ingested, which is generated from the output of the electromyography measuring device 6 capable of detecting the chewing of the diner, feedback The unit 115, based on the information related to the mastication behavior when ingesting the type of food and drink, obtains the information related to the timing, time and/or order in which the food and drink of the type included in the feedback information should be taken, It is also preferred to adjust for the dieter.

Here, a specific example of the feedback processing using the eating behavior information described above will be described.

For example, based on the eating behavior information, the feedback unit 115 determines that the person has a tendency to eat food whose appropriate temperature range is a temperature range that includes 75°C, after cooling it to less than 65°C before ingesting it. In this case, the appropriate temperature range for the type of food and drink may be changed to a temperature range of less than 65° C. and registered in the appropriate food temperature DB 105 for the diner. In addition, if such a tendency to eat after cooling down is seen only in summer, for example, the appropriate temperature range for the type of food and drink in summer is changed to a temperature range of less than 65 ° C and registered in the food and drink appropriate temperature DB 105. You may

Further, for example, if the feedback unit 115 determines from the eating behavior information that the average meal time of the diner is below a predetermined threshold value, or if the average chewing frequency per mouth of the diner exceeds the predetermined threshold value, the feedback unit 115 , for example, feedback information may be selected or generated to the effect that ``you should eat slowly and thoroughly chew your food in a relaxed manner''. Further, when it is determined from the eating behavior information that the standard deviation of meal times exceeds a predetermined threshold, for example, feedback information to the effect that "you should eat regularly" may be selected or generated.

In addition, for example, the feedback unit 115 determines from the eating behavior information that the average number of times per day that the diner takes hot food (food having an appropriate temperature range equal to or higher than a predetermined range) is below a predetermined threshold. In this case, for example, it is also preferable to select or generate feedback information to the effect that "hot food and drink should be taken positively."

The selection or generation of feedback information using "eating behavior information" has been described above. will be described.

Specifically, the feedback unit 115, based also on the QoF index of the diner included in the "eating behavior information", provides information regarding the timing, time and/or order of eating and drinking included in the feedback information. may be tailored to the dieter. For example, when selecting or generating feedback information regarding the food P for a diner whose "(explained later) optimal temperature intake index" for the food P is below a predetermined threshold value, the feedback information may include "this food As for the substance P, it is possible to add information to the effect that it should be taken after it has cooled (or warmed up) a little more.

<Calculation of QoF index>
Here, the specific content of the QoF index will be explained. As this QoF index, at least one, preferably all of the following indices can be adopted.

(Optimal temperature provision index)
As an index related to the appropriate temperature of the provided food and drink,
(3) (Proper temperature provision index) = (number of food types served at proper temperature) / (total number of food types (number of dishes))
can be adopted. Here, the optimum temperature provision index takes a value in the range of 0-1.

(Optimal temperature intake index)
As an index related to the degree to which the provided food P has been ingested while it is at the appropriate temperature,
(4) (Appropriate temperature intake index of food P)=(Ingestion time of food P within the appropriate temperature range)/(Total intake time of food P)
may be adopted. Here, the optimum temperature intake index also takes a value in the range of 0 to 1.

(Diet index at appropriate temperature)
As an index related to the degree to which a meal was completed while the food and drink were at the appropriate temperature,
(5) (Dietary index at appropriate temperature) = Σi (Indicator of appropriate temperature intake for food _i )/(Total number of food types (number of dishes))
can be adopted. Here, the optimal temperature meal index also takes a value in the range of 0 to 1. This optimal temperature meal index is the average value of the optimal temperature intake index for each food and drink served in one meal.

(Food and drink type number index)
As an index related to the number of types of food and drink served,
(6) (Food and drink type number index) = (total number of food and drink types (number of dishes)) / (predetermined target total number of types (target number of dishes))
may be adopted. Here, if the total number of food types (the number of food items) exceeds a predetermined target total number of food types (the target number of food items), the food type number indicator may be rounded to 1.

(environmental optimum temperature index)
As an index related to the appropriate temperature of the user's eating environment,
(7) (environmental optimum temperature index) = (time during which the environmental temperature was within the prescribed optimum temperature range)/(total meal time)
can be adopted. Here, the appropriate environmental temperature index also takes a value in the range of 0 to 1. Moreover, it is also preferable that the "predetermined suitable temperature range" of the environmental temperature is set to a value range according to the season.

It is also possible to employ QoF indices other than the indices described above. for example,
(a) information related to the number of people around the dining table, which is determined based on the image information acquired from the visible light camera 2 using a known face image recognition technology;
(b) Whether or not communication occurred during the meal, based on the information on the amount of conversation that occurred during the meal, determined based on the voice information acquired from the microphone using a known voice recognition technology Communication metrics may be determined.

moreover,
(a) Information related to the number of appearances of a keyword (such as “delicious!”) indicating a good meal, determined based on voice information acquired from a microphone using a known voice recognition technology;
(b) Using the technique disclosed in Japanese Patent Application Laid-Open No. 2020-048917, which is an invention by the inventor of the present application, a "smile" during a meal determined based on the myoelectric signal information acquired from the electromyography measuring device 6. Information related to the time and number of occurrences,
(c) It is also possible to determine a satisfaction index regarding whether or not the person is satisfied with the meal, based on the information regarding the amount of intake, which is determined based on the electromyographic signal information also obtained from the electromyographic measurement device 6. is.

The QoF index described above may be displayed as a radar chart for each index described above, for example, on the display 107 immediately after the meal is finished, and presented to the diner. In addition, the average value of the QoF index for each predetermined period (for example, one day, one week, or one month) in the past is presented as, for example, a bar graph or a line graph so that the diners can recognize the changes in the QoF index so far. It is also preferable to Furthermore, for example, information related to such QoF indicators of each diner who is an elderly person is shared with external medical institutions, etc. Especially when the diner is an elderly person living alone, local communities, local governments, and It is also preferable to work with medical institutions and the like to make proposals such as recommending online dinners, for example.

FIG. 5 is a flow chart showing an outline of an embodiment of an eating behavior monitoring method including calculation of a QoF index according to the present invention. Incidentally, of course, the eating behavior monitoring method according to the present invention is not limited to the form shown in the flowchart of FIG.

(S101) Acquire (or update) the type (dish name) and the number of types (number of items) of food and drink served.
(S102) Using the Food/Drink Appropriate Temperature DB 105, search for appropriate temperature range and ingredient names for each type (each dish name).
(S103) Using the food and drink matching DB 106, food and drink combinations are searched from the (retrieved) ingredient names of each type.
(S104) The temperature of each type (food name) and the environmental temperature are measured and managed.
(S105) Using the measurement result in step S104, the future temperature of each type (food name) is predicted.

(S106) Using the information obtained in steps S102 to S105,
(a) Feedback information regarding the timing, time and order of ingestion of each food and drink;
Selecting or generating feedback information that includes at least one of (b) information relating to matching between foods and drinks, and (c) information relating to attention to intake.
(S107) Provide feedback information.

(S108) Measure and monitor the amount of food and drink of each type (dish name).
(S109) Recording and managing the intake behavior (for example, the number of times of chewing).
(S110) Provide feedback information (caution, etc.) about the detected ingestion behavior (if it is determined to be necessary).
Here, for example, when detecting an ingestion behavior of holding a plate in hand, if the temperature of the food and drink served on the plate is above (below) the appropriate temperature range, it is determined that "this dish is Feedback information to the effect that "still too hot (too cold)" may be presented immediately.

(S111) Whether or not the food has been eaten completely (whether or not the amount of all food and drink has become zero), and whether or not the intake has been completed (whether or not an ingestion behavior (e.g., mastication) has been performed for a predetermined period of time) make a judgment.
Here, when one or both of the determinations of "completely eaten" and "completed intake" are made, the process moves to step S112 to finally calculate the QoF index.

(S112) On the other hand, if it is determined in step S111 that both "the meal is not finished" and "the intake is not finished", then Determine whether there is a change.
Here, (for example, due to the addition of a new type of food and drink or the reduction of one or more types of food and drink), "There is a change in the number of types (number of items). However, the number of types (number of items) is greater than or equal to 1", the process proceeds to step S101, which is the first step of the eating behavior monitoring method. Further, when it is determined that "the number of types (the number of items) has not changed", the process proceeds to step S104 in order to update the food temperature information.

(S113) Furthermore, when it is determined that "the number of types (number of items) is zero" (for example, because all the food and drink have been removed), based on the information obtained so far, A QoF index is calculated and presented, further managed, and then the regular eating behavior monitoring method ends.

Returning to the functional block diagram of FIG. 1, the notification means control information generating unit 115b of the feedback unit 115 selects or generates "information related to the timing, time and/or order in which the predetermined type of food and drink should be taken." ”, the predetermined type of food (soup) is irradiated with light by a light irradiation means (projector 4 in FIG. 1) that can irradiate the predetermined type of food (soup in FIG. 1). Control information for controlling light irradiation may be generated, and this control information may be output or transmitted to the light irradiation means (projector 4).

As a result, for example, when the feedback information to the effect that "soup should be eaten first" is selected or generated, the notification means control information generating unit 115b will generate "what to eat first for the soup that should be eaten first". is projected, and this projection is continued until an action of eating soup is detected." can also be implemented.

In this embodiment, the projector 4 is installed and adjusted so that the entire range of the top surface of the dining table can be projected as if the top surface of the dining table is a screen. In addition, in order to map and project the feedback information on the specified position on the dining table, the notification means control information generation unit 115b generates control information including an image to be projected as feedback information and its projection position. is generated and transmitted to the projector 4, and the image is projected at the designated projection position. Here, in order to present the feedback information in a form that is easy for the diners to understand or accept, the information to be projected may be content composed of animation images, character information, or the like. As a preferred modification, the projector 4 irradiates a simple spotlight. An irradiation position control section for controlling the driving of the servo motor may be provided so that irradiation can be performed. In this case, first
(a) The food and drink recognition unit 112 uses the image information (including the served soup) generated by the visible light camera 2 to determine the position information where the soup is placed,
(b) The notification means control information generation unit 115b generates control information including an instruction to illuminate the position related to the determined position information of the soup, and transmits the control information to the irradiation position control unit of the projector 4,
(c) The irradiation position control section of the projector 4 drives the servo motor so that the projector 4 can irradiate the position indicated (in the received control information).

In addition, in the present embodiment, the notification means control information generation unit 115b of the feedback unit 115 is selected or generated as feedback information based on "information regarding the timing, time and/or order in which a predetermined type of food and drink should be taken". ,
(a) a light source pre-mounted in a utensil used for a meal of a diner (in FIG. 1, a lighting lamp for notification mounted on chopsticks 5 with a notification function);
(b) a vibrator pre-mounted on a utensil used for eating by a diner (in FIG. 1, a notification vibrator mounted on chopsticks 5 with a notification function);
and/or
(d) generating control information for controlling the notification operation to the diner by the vibrator pre-mounted on the tableware related to the diner's meal, and transmitting this control information to the light source and/or the vibrator; You may

As a result, for example, when the feedback information to the effect that "the food (that has just been sandwiched between chopsticks) is too hot (its temperature exceeds the appropriate temperature range)" is selected or generated, the notification means control information generation unit 115b Generating control information with content to generate a vibration meaning a warning that it is "too hot", transmitting this to the chopsticks with notification function 5 (that are sandwiching the food and drink that are too hot), and chopsticks with notification function. 5 vibrator may be made to carry out the operation of the content.

Further, the notifying means control information generation unit 115b generates appropriate control information suitable for the feedback information and transmits it to the notifying means as described above, so that various notifications reflecting the feedback information can be implemented. . For example, for a food that should wait for it to cool down (too hot), a countdown indicating the remaining time until "the time when the food is allowed to be taken (the temperature of the food is within the appropriate temperature range)" determined from the predicted temperature information. It is also possible to irradiate (a position adjacent to) the food and drink with a projection light capable of projecting a timer or progress bar.

In addition, when the lighting lamp mounted on chopsticks with notification function 5 can emit light of various colors, chopsticks with notification function 5
(a) When a hot food or drink that is too hot (its temperature exceeds the appropriate temperature range) is sandwiched, the lighting lamp immediately flashes red,
(b) Lights green when hot food or drink within the appropriate temperature range is sandwiched,
(c) When sandwiching food or drink at room temperature (at the same temperature as the ambient temperature), do not turn on the lighting lamp,
(d) When a cold food or drink that is too cold (its temperature is below the optimal temperature range) is sandwiched, it immediately glows purple;
(e) Color presentation that makes it possible to understand at a glance the relationship between the temperature of the food you are about to consume and its appropriate temperature range, such as by lighting a lighting lamp in cyan when a cold food or drink within the appropriate temperature range is sandwiched. It is also preferable to

Alternatively, the spotlight may continue to irradiate the food whose temperature is within the appropriate temperature range, and the spotlight may be turned off when the temperature deviates from the appropriate temperature range. Furthermore, a group of food and drink judged to be good to eat together is illuminated with a spotlight of the same color, for example white, while a food and drink that should be eaten separately is spotlighted with a spotlight of a different color, for example white. and a red spotlight. Moreover, it is also possible to irradiate a stronger spotlight to the food and drink that should be taken next due to the relationship of eating together to indicate the next intake candidate.

Furthermore, when the food (that is sandwiched between the chopsticks) is too hot (its temperature exceeds the appropriate temperature range), the speaker 109 of the eating behavior monitoring device 1 (or a speaker installed outside) The boiled food is too hot. Please be careful." may be output. In addition, regarding foods such as “meat buns,” for which the temperature of the entire food and drink is difficult to estimate from the surface temperature (obtained from the infrared image), the menu/ingredient temperature DB of the food and drink appropriate temperature DB 105 indicates “difficult to predict”. After confirming that the is registered, it is also preferable to make a notification to that effect by voice from the speaker 109, for example.

Similarly, in the functional block diagram of FIG. 1, the food temperature control information generation unit 115c of the feedback unit 115 generates "information regarding the timing, time and/or order in which a predetermined type of food should be taken" selected or generated as feedback information. "Based on this, generate control information for controlling the operation of cooling and / or heating food and drink by a cooling device and / or a heating device pre-mounted in tableware related to a meal of a diner, and this control information may be sent to said cooling device and/or said heating device.

As a result, for example, when feedback information to the effect that "(the food you just put your chopsticks on) is too hot (its temperature exceeds the appropriate temperature range)" is selected or generated, the food temperature control information generation unit 115c , generates control information to the effect that the cooling operation is to be performed, and applies this to the Peltier element mounted on the plate (on which the food and drink are placed) (as a plurality of cooling devices arranged in a lattice pattern inside the plate surface) It may be sent to coolers to cause these Peltier coolers to perform actions to cool the food (reduce the temperature of the food to within the proper temperature range).

Here, as the heating device, for example, an electric heater or an IH heater capable of performing electromagnetic induction heating may be employed. In this case, the food temperature control information generating unit 115c, for example, transmits control information to the effect that a heating operation is to be performed to an IH heater built into a plate on which too cold food is served, and heats the food. It is also possible to carry out the operation to do (raise the temperature of the food and drink to be within the appropriate temperature range).

As described in detail above, according to the present invention, it is possible to acquire food temperature information about food and drink together with food content information about the food and drink, instead of obtaining food and drink temperature information alone. selected or generated feedback information. That is, it is possible to grasp the temperature of the provided food according to its content, and to provide more accurate feedback for the diners.

In addition, by providing such feedback information, for example, diners are encouraged to change their eating behavior to a more suitable eating behavior in the sense that they can eat deliciously, or to encourage a behavioral change to a healthier eating behavior. It is also possible to

In addition, for example, feedback information generated by the present invention in order to promote healthier eating behavior for diners, including children and the elderly, considering the temperature and content of the provided food and drink. It is also possible to present feedback information that is positive for health. That is, according to the present invention, Goal 3 of the Sustainable Development Goals (SDGs) led by the United Nations, "Ensure healthy lives and promote well-being for all at all ages, eliminate poverty in all its forms everywhere" You can also contribute to 'putting an end to

Furthermore, for example, feedback information generated by the present invention in order to encourage eating behaviors in which problems such as leftovers and nutritional imbalance are resolved for diners in each home, and includes the temperature and content of the provided food and drink. It is also possible to present feedback information that is useful for solving the problem. That is, according to the present invention, it is also possible to contribute to Goal 12 of the Sustainable Development Goals (SDGs) led by the United Nations, "ensure sustainable consumption and production patterns."

For the various embodiments of the present invention described above, various changes, modifications and omissions within the scope of the technical ideas and aspects of the present invention can be easily made by those skilled in the art. The foregoing description is exemplary only and is not intended to be limiting. The invention is to be limited only as limited by the claims and the equivalents thereof.

1 Eating behavior monitoring device 101 Communication interface 102 Food temperature information storage unit 103 Food image information storage unit 104 Eating behavior related information storage unit 105 Appropriate food temperature database (DB)
106 Food and drink combination DB
107 touch panel display (TP/DP)
108 Microphone 109 Speaker 111 Food/Environment Temperature Acquisition Unit 112 Food Recognition Unit 113 Food Content Acquisition Unit 114 Eating Behavior Information Generation Unit 114a QoF Calculation Unit 115 Feedback Unit 115a Food Temperature Prediction Unit 115b Notification Means Control Information Generation Unit 115c Food temperature control information generation unit 121 communication control unit 122 input/output control unit 2 visible light camera 3 thermography camera 4 projector 5 chopsticks with ventilation function 6 myoelectric measuring device

Claims (16)

food temperature acquisition means for acquiring food temperature information, which is information relating to the temperature of food and drink, from temperature measuring means capable of measuring the temperature of food and drink related to the meal of the diner;
food content acquisition means for acquiring food content information, which is information relating to the type and/or amount of the food and drink, from a recognition means capable of recognizing the type and/or amount of the food and drink;
Selecting or selecting feedback information related to the eating behavior of the diner based on the food temperature information and the food content information using preset feedback criteria or models based on food temperature and content feedback means for generating and providing. The food and drink content acquisition means acquires the food and drink content information related to at least the type of the food and drink for the diner,
The feedback means predicts information about the future temperature of the type of food and drink based on at least the food and drink content information about the type and the food and drink temperature information, and predicts the predicted future temperature of the food and drink. 2. The eating behavior monitoring device according to claim 1, wherein said feedback information including information relating to the timing, time and/or order in which said type of food and drink should be taken is selected or generated based on. The food and drink temperature acquiring means also acquires environmental temperature information, which is information relating to the temperature of the eating environment of the diner, from the temperature measuring means or another temperature measuring means,
3. The eating behavior monitoring device according to claim 2, wherein said feedback means predicts information on the future temperature of said type of food and drink based also on said environmental temperature information. Eating-behavior-related information that is information related to the past eating behavior of the said diner, and which includes at least information related to the temperature at the time of ingestion of the food of the said type, from the storage means that has stored the eating-behavior-related information of the said food further comprising eating behavior information generating means for receiving behavior-related information and generating eating behavior information summarizing the eating behavior-related information;
The feedback means provides the timing, time, and temperature at which the type of food and drink included in the feedback information should be taken, based on the information on the temperature at which the type of food and drink is ingested, which is included in the eating behavior information. 4. The eating behavior monitoring device according to claim 2 or 3, wherein information relating to the order is adjusted according to the person concerned. The eating behavior information also includes information related to the time when the food and drink of the type was ingested,
The feedback means, based also on the information on the time at which the type of food and drink was ingested, feeds the information on the timing, time and/or order in which the type of food and drink included in the feedback information should be taken. 5. The eating behavior monitoring device according to claim 4, wherein the device is adjusted according to the person eating. The eating behavior information includes information related to mastication behavior when ingesting the type of food and drink, which is generated from the output of a pre-installed myoelectric sensor capable of detecting the mastication of the diner,
The feedback means provides information regarding the timing, time and/or order in which the type of food and drink included in the feedback information is to be taken, based on information regarding mastication behavior when the type of food and drink is ingested. 6. The eating behavior monitoring device according to claim 4 or 5, wherein is adjusted according to the diner. The eating behavior information is a quality index of the eating behavior of the diner, and is an index related to the appropriate temperature of the provided food and drink, and the degree to which the provided food and drink is consumed while it is at the appropriate temperature. the degree to which the meal was completed while the food was at the appropriate temperature; the number of types of food and drink served; Includes quality indicators of eating behavior including such indicators,
The feedback means adjusts the information related to the timing, duration and/or order in which the type of food and drink included in the feedback information is to be ingested in accordance with the eater, based on the quality index of the eating behavior. The eating behavior monitoring device according to any one of claims 4 to 6, characterized in that: The temperature measuring means may be a thermography installed in the eating environment of the diner, a thermometer mounted on a utensil used for the meal of the diner, and/or a tableware for the meal of the diner. 8. The eating behavior monitoring device according to any one of claims 1 to 7, wherein the device is a thermometer. The feedback means is a pre-installed light irradiation means capable of irradiating the food and drink based on the timing, time and/or order to ingest the selected or generated food and drink of the type 9. The method according to any one of claims 2 to 8, wherein control information for controlling irradiation of light to said type of food and drink is generated, and said control information is output or transmitted to said light irradiation means. The eating behavior monitor device described. The feedback means, based on the selected or generated information related to the timing, time and/or order of ingestion of the type of food and drink, the Control for controlling the notification operation to the diner by the vibrator pre-mounted on the utensil, the light source pre-mounted on the tableware related to the meal of the diner, and/or the vibrator pre-mounted on the tableware 10. The eating behavior monitoring device according to any one of claims 2 to 9, wherein information is generated and the control information is transmitted to the light source and/or the vibrator. The feedback means includes a cooling device and/or a cooling device pre-mounted on the tableware for the meal of the diner based on the selected or generated information relating to the timing, time and/or order of ingestion of the type of food and drink. 2. Generate control information for controlling an operation of a heating device to cool and/or heat the food, and transmit the control information to the cooling device and/or the heating device. 11. The eating behavior monitor device according to any one of 10. The recognition means acquires information related to an image containing the food and drink from a pre-installed camera capable of photographing the food and drink, and based on the information related to the image, determines the type and/or amount of the food and drink. 12. The eating behavior monitoring device according to any one of claims 1 to 11, which recognizes The eating behavior monitoring device includes an information processing device capable of receiving information related to the image from the external camera, the information processing device having the recognition means, a tablet computer having the camera and the recognition means, the camera and the recognition device. 13. The eating behavior monitoring device according to claim 12, wherein the device is a smart phone provided with means, or an augmented reality (AR) type smart glasses provided with the camera and the recognition means. A temperature measuring means capable of measuring the temperature of the food and drink related to the meal of the diner;
food temperature acquisition means for acquiring food temperature information, which is information relating to the temperature of the food, from the temperature measurement means;
a recognition means capable of recognizing the type and/or amount of the food and drink;
food content acquisition means for acquiring food content information, which is information relating to the type and/or amount of the food and drink, from the recognition means;
Selecting or selecting feedback information related to the eating behavior of the diner based on the food temperature information and the food content information using preset feedback criteria or models based on food temperature and content and feedback means for generating and providing. A program that operates a computer that monitors the eating behavior of a diner,
food temperature acquisition means for acquiring food temperature information, which is information relating to the temperature of food and drink, from temperature measuring means capable of measuring the temperature of food and drink related to the meal of the diner;
food content acquisition means for acquiring food content information, which is information relating to the type and/or amount of the food and drink, from a recognition means capable of recognizing the type and/or amount of the food and drink;
Selecting or selecting feedback information related to the eating behavior of the diner based on the food temperature information and the food content information using preset feedback criteria or models based on food temperature and content An eating behavior monitor program characterized by causing a computer to function as feedback means to generate and provide. An eating behavior monitoring method in a computer for monitoring the eating behavior of a diner, comprising:
Acquiring food temperature information, which is information relating to the temperature of the food and drink, from a temperature measuring means capable of measuring the temperature of the food and drink related to the meal of the diner, and recognizing the type and/or amount of the food and drink obtaining food and drink content information, which is information relating to the type and/or quantity of the food and drink, from a possible recognition means;
Selecting or selecting feedback information related to the eating behavior of the diner based on the food temperature information and the food content information using preset feedback criteria or models based on food temperature and content An eating behavior monitoring method, comprising: generating and providing.

Images