JP7009342B2 - Devices, programs and methods that can evaluate meals based on the amount of chewing and smiling - Google Patents

Description

The present invention relates to a technique for estimating information related to a living body from an acquired biological signal.

In recent years, techniques have been developed in which various biological signals caused by various activities of humans and animals are detected by sensors and the biological data obtained by signal processing is used in various situations. Here, as the sensor, for example, a wristwatch type pulse sensor, an earphone type pulse sensor, a headband type brain wave sensor, or the like is used. Further, the biological signal detected by such a sensor is processed and processed by, for example, a smartphone carried by a user, and is used in various applications.

Recently, in particular, due to the increase in the number of elderly people and the increase in health consciousness, attempts have been actively made to analyze the contents of daily meals from the obtained biological data and search for a more preferable dietary mode.

For example, Patent Document 1 is an experiment in which the power spectrum due to the acceleration signal detected by the acceleration detection unit 1 for the body surface differs depending on whether the subject feels delicious when eating or drinking. The results are disclosed. Here, it has also been confirmed that each power spectrum has distinguishable characteristics, and specifically, an experiment has been conducted in which the swallowing response is measured to evaluate the texture of the subject.

Further, Patent Document 2 compares the acquisition unit 11 that acquires mastication data related to mastication from the measuring device 50 that measures the movement of the human body during mastication, and the comparison unit 12 that compares the acquired mastication data and predetermined threshold data. A meal support device 100 including a generation unit 13 that generates proposal information regarding meals based on the results of comparison in unit 12 is disclosed. Here, it is said that the device 100 can also provide a proposal regarding a meal including chewing instruction.

Further, Patent Document 3 analyzes changes in biological information before and after a meal based on the biological information before and after a meal measured by the biological information measuring device 100 and the measurement time at which the biological information before and after the meal was obtained. A meal advice providing system including a biometric information analysis unit 22 and an advice providing unit 23 that provides advice on meals based on the analysis result is disclosed.

Here, according to this meal advice providing system, it is possible to provide advice on meals even if there is no photographed image of meals, and by performing analysis using biological information before and after meals, biological information before and after meals can be provided. Clarify the causal relationship between changes in food and diet, and even when eating in a general place where a camera is not installed on the ceiling, appropriate dietary advice is given based on the analysis results of changes in biometric information. It is said that it will be available.

JP-A-2017-086322 Japanese Unexamined Patent Publication No. 2016-159096 Japanese Patent No. 6285086

Here, when the inventor of the present application generates, for example, advice information on a meal from the acquired biological data, it is more preferable to take into consideration the amount of meal that the user (subject) feels, in particular, to generate more suitable information. I found it important. In fact, the information on how the user perceives the amount of food is one of the very important indicators in evaluating the current meal.

However, in the above-mentioned conventional technique, even if information on meals is collected from biological information and further meal advice is given, no consideration is given to the amount of meals felt by the user. I didn't come. For example, in the technique disclosed in Patent Document 1, by analyzing the acceleration of swallowing in the frequency domain, the user's likes and dislikes of meals, that is, the tastes of meals are only analyzed.

Further, also in the technique disclosed in Patent Document 2, advice for recommending healthy eating behavior such as sensing chewing, increasing the number of chewing times according to a predetermined rule, and selecting the order of eating and firm ingredients. Is just doing. Further, Patent Document 3 proposes a method of analyzing and learning biometric information and a meal image and providing advice on meals without a meal image, but in generating this advice, biometric information ( While estimating the state of eating fast and the state of overeating from the magnitude of the gradient of change in the sugar index) and the magnitude of the amount of change, it was not taken into consideration the amount of food actually felt by the user. Not.

Therefore, the present invention is a dietary evaluation device, system, or program capable of providing information related to the user's diet or health based on the acquired biological signal, taking into consideration the information related to the amount of the user's diet. And to provide methods.

According to the present invention, it is a meal evaluation device that evaluates a user's meal.
An amount determined from a biological signal related to the user's head, which is a biological signal information determining means for determining an amount related to chewing and an amount related to a smile of the user.
A meal amount determination means for determining a value related to the amount of meal using a learned meal amount estimation model based on the amount related to chewing and the amount related to smile .
A dietary evaluation device is provided having a feedback means for generating or selectively providing information related to the user's diet or health based on the determined value of the amount of the diet.

For the meal evaluation device according to the present invention, a plurality of meal logs recorded by associating the user-declared information about at least the value related to the amount of meal with respect to the amount related to chewing and the amount related to smile are used. Therefore, it is also preferable to further have a meal amount estimation model construction means for constructing the learned meal amount estimation model.

Further, it is also preferable that the meal amount determining means in the meal evaluation device according to the present invention determines a value related to the amount of meal based on the meal time.

Further, as an embodiment of the dietary evaluation device according to the present invention, the present dietary evaluation device uses a learned preference estimation model based on the amount related to the chewing and / or the amount related to the smile to determine the taste of the meal. Further possessing a means for determining a dietary preference to determine such a value,
It is also preferred that the feedback means generate or select information related to the user's diet or health based on the determined dietary preference values.

Further, in the above embodiment, the present meal evaluation device records the amount related to chewing and / or the amount related to smile by associating at least the value declared by the user with respect to the value related to meal preference. It is also preferable to further have a preference estimation model construction means for constructing the trained preference estimation model by using a plurality of logs.

Further, it is also preferable that the feedback means of the dietary evaluation device according to the present invention generates or selects information related to the diet or health of the user based on the dietary log relating to at least one past time point.

Further, as an embodiment of the feedback means according to the present invention, the feedback means is based on at least one of the number of times, the pace and the time interval in chewing determined from the amount of the chewing. It is also preferable to select at least one of the music or audio contents preset according to the conditions satisfied by the value related to the amount of the meal and the information related to the meal or health of the user.

Further, according to the present invention, it is a meal evaluation device that evaluates a user's meal.
An amount determined from a biological signal related to the head of the user, and a biological signal information determining means for determining at least an amount related to a smile among the amount related to chewing and the amount related to a smile of the user.
A meal amount determining means for determining a value related to the amount of meal using a learned meal amount estimation model based on at least the amount related to smile among the amount related to chewing and the amount related to smile.
A feedback means that generates or selectively provides information related to the user's diet or health based on the determined dietary value.
Have,
The feedback means is preset according to the condition that the number and / or time of the smile and the value related to the amount of the meal are satisfied, based on the number and / or time of the smile determined from the amount of the smile. Select at least one of the music or audio content and use it as information related to the user's diet or health.
A meal evaluation device characterized by this is provided .

Further, as a further other embodiment of the feedback means according to the present invention, the feedback means may be applied to the diet or health of the user based on the acquired position information and / or voice information related to the meal. It is also preferred to generate or select relevant information.

Further, as an embodiment of the biometric signal information determining means according to the present invention, the biometric signal information determining means generates acceleration component data of an input signal that may include a biometric signal, and in a predetermined time interval in the acceleration component data. It is also preferable to calculate a representative value related to the degree of bias of the data and determine the amount related to the chewing based on the periodicity of the representative value.

Further, according to the present invention, it is a meal evaluation device that evaluates a user's meal.
An amount determined from a biological signal related to the head of the user, and a biological signal information determining means for determining at least an amount related to a smile among the amount related to chewing and the amount related to a smile of the user.
A meal amount determining means for determining a value related to the amount of meal using a learned meal amount estimation model based on at least the amount related to smile among the amount related to chewing and the amount related to smile.
A feedback means that generates or selectively provides information related to the user's diet or health based on the determined dietary value.
The above-mentioned biological signal information determining means has
Acceleration component data of an input signal that can include biological signals is generated, a representative value related to the degree of data bias in a predetermined time interval in the acceleration component data is calculated, and the living body is repeated based on the periodicity of the representative value. Determine the generation of periodic biological signals caused by exercise, and further
Multiple resolution analysis processing is performed on the input signal, and the time interval in which the time-series data of the signal amplitude after the multiple resolution analysis processing shows a predetermined hysteresis is determined as the signal generation time interval in which some biological signal is generated. When it is determined that the periodic biometric signal is not generated in the signal generation time section, the average power frequency of the input signal is calculated, and the amount related to the smile is calculated based on the height of the average power frequency. To decide
A meal evaluation device characterized by this is provided .

Further, according to the present invention, it is a meal evaluation device that evaluates a user's meal.
An amount determined from a biological signal related to the head of the user, and a biological signal information determining means for determining at least an amount related to a smile among the amount related to chewing and the amount related to a smile of the user.
A meal amount determining means for determining a value related to the amount of meal using a learned meal amount estimation model based on at least the amount related to smile among the amount related to chewing and the amount related to smile.
A feedback means that generates or selectively provides information related to the user's diet or health based on the determined dietary value.
The above-mentioned biological signal information determining means has
Acceleration component data of an input signal that can include biological signals is generated, a representative value related to the degree of data bias in a predetermined time interval in the acceleration component data is calculated, and the living body is repeated based on the periodicity of the representative value. Determine the generation of periodic biological signals caused by exercise, and further
Multiple resolution analysis processing is performed on the input signal, and the time interval in which the time-series data of the signal amplitude after the multiple resolution analysis processing shows a predetermined hysteresis is determined as the signal generation time interval in which some biological signal is generated. When it is determined that the periodic biometric signal is not generated in the signal generation time section, the feature amount including the standard deviation and the average power frequency of the input signal is calculated, and the feature amount corresponds to the reference state. The degree of separation, which is the degree of separation from the unit space set by the feature amount of the input signal to be input, is calculated, and the amount related to the smile is determined based on the calculated degree of separation.
A meal evaluation device characterized by this is provided .

Further, as yet another embodiment of the biological signal information determining means according to the present invention, the biological signal information determining means is used.
Image information related to the user's meal is acquired, and based on the image information, the learned image meal type estimation model and / or image meal amount estimation model is used to provide meal type information and / or meal in the user's meal. Determine the amount related to the amount of
Based on the determined meal type information and / or the amount related to the amount of meal, the learned biological signal information estimation model constructed by learning based on the amount related to the past chewing and / or the amount related to the smile is used. It is also preferable to determine the amount of chewing and / or the amount of smile of the user.

Further, in the dietary evaluation device according to the present invention, the biometric signal is specifically a device attached to the user's head, and the reference electrode is one from around the left (or right) auricle to the vicinity of the cheek. It is also preferred that the signal is obtained by a device having an electrode configuration that is in contact with the skin position and the detection electrode is in contact with one skin position near the cheek from around the right (or left) pinna.

According to the present invention, there is also a meal evaluation system including a terminal related to the user and a meal evaluation server for evaluating the user's meal, which is connected to the terminal by communication.
The terminal has a biological signal information determining means that is an amount determined from a biological signal related to the head of the user, and determines an amount related to chewing and an amount related to a smile of the user.
The meal evaluation server is
Image information related to the user's meal is acquired from the terminal, and based on the image information, the trained image meal type estimation model and / or the image meal amount estimation model is used to obtain the meal type information and / or meal type information in the user's meal. / Or determine the amount of food, and based on the determined meal type information and / or the amount of food, the amount of past chewing and the amount of smile obtained from the terminal. Using the trained biometric signal information estimation model constructed by learning, an image information determining means for determining the amount of chewing and the amount of smiling face of the user, and
It has a meal amount determining means for determining a value related to the amount of meal using a learned meal amount estimation model based on the amount related to chewing and the amount related to smile .
The terminal acquires a value related to the amount of the meal from the meal evaluation server, and generates or selects and provides information related to the user's meal or health based on the value related to the amount of the meal. A dietary evaluation system characterized by further possession of means is provided.

According to the present invention, it is a program for operating a computer mounted on a device for evaluating a user's meal.
An amount determined from a biological signal related to the user's head, which is a biological signal information determining means for determining an amount related to chewing and an amount related to a smile of the user.
A meal amount determination means for determining a value related to the amount of meal using a learned meal amount estimation model based on the amount related to chewing and the amount related to smile .
A dietary evaluation program is provided that functions the computer as a feedback means to generate or select and provide information related to the user's diet or health based on the determined value of the amount of the diet.

According to the present invention, further, it is a meal evaluation method in a computer mounted on a device for evaluating a user's meal.
An amount determined from a biological signal related to the user's head, and a step of determining an amount related to chewing and an amount related to smile of the user.
Based on the amount related to the chewing and the amount related to the smile , the step of determining the value related to the amount of the meal using the learned meal amount estimation model, and
A dietary evaluation method is provided that comprises a step of generating or selecting and providing information related to the diet or health of the user based on the determined value of the amount of the diet.

According to the dietary evaluation device, system, program and method of the present invention, based on the acquired biological signal, information related to the user's diet or health is provided in consideration of the information related to the amount of the user's diet. be able to.

It is a schematic diagram which shows one Embodiment of the meal evaluation system including the meal evaluation apparatus by this invention. It is a schematic diagram which shows the other embodiment of the biological signal acquisition apparatus which concerns on this invention. It is a graph which shows an example of the biological signal counting process using the hysteresis in the time series data of the processed biological signal. It is a schematic diagram for schematically explaining one Embodiment of the meal evaluation method by this invention. It is a schematic diagram which shows the other embodiment of the meal evaluation system by this invention. It is a schematic diagram for schematically explaining another embodiment of the meal evaluation method by this invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

[Meal evaluation system]
FIG. 1 is a schematic diagram showing an embodiment of a meal evaluation system including a meal evaluation device according to the present invention.

The meal evaluation system of the present embodiment shown in FIG. 1 is
(A) Glasses 2 with an EMG sensor as a biological signal acquisition device,
(A) A mobile terminal 1 as a meal evaluation device according to the present invention for evaluating a user's meal (wearing glasses 2 with a myoelectric sensor) is provided. Here, both of them are provided with a signal interface 213 and a signal interface 101, which are wireless communication means compliant with, for example, Bluetooth (registered trademark), Wi-Fi (registered trademark) or other short-range wireless communication standards, respectively. , This allows them to communicate with each other. As a modification, both may be connected by wire.

First, the glasses 2 with an myoelectric sensor according to (a) above are glasses-type devices that can be attached to the head of a living body (for example, a human user) to acquire a biological signal. In the present embodiment, the biological signal acquired by this device is a myoelectric signal as an electric signal generated due to the movement of a portion in the face or the movement related to a facial expression. Further, the acquired biological signal may be mixed with a (noise) signal caused by electrode misalignment, which is generated by such a movement.

Here, as the movement of the facial part to be detected or the movement related to the facial expression, in the present embodiment, particular attention is paid to "chewing", and a periodic biological signal having a temporal periodicity because it is caused by repeated movements. It is characterized by efficiently detecting the myoelectric signal related to "chewing". Furthermore, the myoelectric signal related to "smile", which is regarded as raising the corner of the mouth, is also an important detection target.

Similarly, in FIG. 1, the glasses 2 with an EMG sensor are specifically shown.
(A) A frame portion as a device main body portion provided with a signal processing box 21 which is a portion for capturing and processing biological signals, and a frame portion.
(B) A positive electrode pad 23 and a negative electrode as an electrode portion for receiving a biological signal, which are arranged at a position in contact with the skin of the head and capable of receiving at least a part of the weight of the frame portion. Pad 24 and
(C) An elastic support portion provided with a conductive path for transmitting a biological signal received via the positive electrode pad 23 and the negative electrode pad 24 to the signal processing box 21.
(D) It is arranged at a position in contact with the vicinity of the upper part of the nose, and has a nose pad electrode portion 25 provided with a ground (GND) electrode or a noise canceling electrode at the time of receiving a biological signal. The noise canceling electrode (d) may be a DRL (Driven Right Leg) electrode that reduces common mode noise caused by a commercial power source or the like.

Further, the positive electrode pad 23 of the above (b) functions as a detection electrode or a positive electrode when receiving a biological signal, while the negative electrode pad 24 functions as a reference electrode or a negative electrode when receiving a biological signal. .. The biological signal is detected and acquired as a potential difference between the positive electrode pad 23 and the negative electrode pad 24.

As described above, in the glasses 2 with the myoelectric sensor, the electrode portions such as the positive electrode pad 23 and the negative electrode pad 24 function not only as a means for receiving the biological signal but also as a means for supporting the device main body portion. .. Further, the elastic support portion elastically connects these electrode portions and the device main body portion. As a result, for example, even if the mounted head moves significantly, these electrode portions are stabilized in the vicinity of a predetermined position on the skin of the head with the weight of the device main body transmitted through the elastic portion called the elastic support portion. It becomes possible to keep in contact with each other.

Here, one mounting example will be described. The human cheekbones project laterally when viewed from the front of the face, but if the positive electrode pad 23 and the negative electrode pad 24 are brought into contact with the skin slightly above the widest part of the cheekbone, for example, the left and right sides Since the distance between the "electrode portions" is narrower than the maximum width of the cheekbone and is caught in the widened portion of the upper part of the cheekbone, the glasses 2 with the myoelectric sensor are stably supported.

Further, although the glasses 2 with a myoelectric sensor are not shown, the glasses 2 are provided with a conductive path for taking a biological signal from the electrode portion to the signal processing box 21, and the biological signal received by the electrode portion is used in the signal processing box. It enables stable and reliable uptake to 21. That is, the conductive path functions as a stable electrical transmission path connecting the left and right signal processing boxes 21 and each electrode portion.

As a modification, the modern part of the glasses 2 may have a function as a GND electrode or a noise canceling electrode. In this case, the nasal pad electrode portion 25 can be omitted to make the nasal padless. Further, as a further modification, the electrodes of the modern portion and the nasal pad electrode portion 25 may be electrically conductive, and the plurality of electrodes may function as GND electrodes.

Further, as a matter of course, the positive electrode pad 23 and the negative electrode pad 24 may be in an interchangeable shape connected to the left temple portion and the right temple portion, respectively. In any case, by arranging the positive electrode pad 23 and the negative electrode pad 24 separately on the left and right, it is possible to capture the activity of the same kind of muscles existing on the left and right with respect to the median plane.

For example, the muscle activity related to "chewing" and the muscle activity that creates a facial expression "smile", which are of particular interest in the present embodiment, generally occur simultaneously on both sides, not on either side. Therefore, rather than arranging one set of electrodes constituting one channel on either the left or right side and directly above the muscle to be observed, for example, separated by several centimeters, each set of electrodes is arranged separately on the left and right sides. In the end, it is possible to obtain a more stable and large myoelectric signal because it captures the entire left and right muscle activity.

In addition, by separating the left and right electrodes in this way, the skin and electrodes caused by the myoelectric signal caused by "chewing" (opening and closing of the mouth) and the unevenness of the skin surface caused by "chewing". It is also possible to more reliably capture the signal caused by the change in contact resistance between the two.

Further, two elastic support portions are provided in the present embodiment, and the positive electrode pad 23 and the negative electrode pad 24 are located at the positions of the skin below the temples, respectively, when the face is viewed from the front. It elastically presses (contacts) the skin slightly above the widest part of the cheekbone, thereby functioning as a support structure for supporting the glasses 2 with an myoelectric sensor.

Further, both the positive electrode pad 23 and the negative electrode pad 24 can come into contact with the skin at any position within the range from the upper cheek to the base of the ear through the temple, and the frame of the elastic support portion. The position with respect to the part is adjusted.

Similarly, as shown in FIG. 1, the signal processing box 21 arranged in the temple portion on the right side has a built-in battery for driving the processing unit, and the signal processing box 21 arranged in the temple portion on the left side has a battery of this battery. Supply power. It is also preferable that the weights of the left and right signal processing boxes 21 are set to be substantially (almost) equal. As a result, it is possible to balance the weight of the glasses 2 with the myoelectric sensor on the left and right, and it is possible to realize a good wearing feeling without bias.

Here, the signal processing box 21 arranged in the temple portion on the left side has a signal conversion unit 211, a pre-filter processing unit 212, and a signal interface 213. Of these, the signal conversion unit 211 serves as a myoelectric sensor.
(A) A positive (detection) electrode electrically connected to the positive electrode pad 23,
(B) The AC component of the potential difference between the negative electrode pad 24 and the electrically connected negative (reference) electrode.
(C) Amplification is performed by differential amplification between the nose pad electrode 25 and the GND potential of the electrically connected GND electrode, and this analog biological signal is digitized at a constant sampling frequency. Incidentally, this differential amplification may be carried out with a DRL circuit for reducing common mode noise caused by a commercial power source or the like.

This makes it possible to detect the skin potential in the range of plus or minus 0.1 to several hundred μV, for example. Further, as a condition for this digitization, it is also preferable to perform analog / digital (A / D) conversion at a sampling frequency of 500 Hz or higher and a quantization frequency of 10 bits or higher. It should be noted that such a circuit configuration can be realized by using, for example, TGAM1 manufactured by Neurosky.

Further, the pre-filter processing unit 212 has a notch filter function and a low-pass filter (LPF) function. Of these, the notch filter function performs band-stop filter processing that reduces periodic noise related to the commercial power supply (which is often mixed) with respect to the input signal before the acceleration component data is generated. Incidentally, the above-mentioned TGAM1 manufactured by Neurosky is equipped with a notch filter that reduces noise derived from a commercial power source, and this can be used as a notch filter function.

On the other hand, the LPF function performs LPF processing for removing high frequency noise from the input signal subjected to the band removal filter processing. Specifically, LPF processing may be performed by performing high-pass filter (HPF, High-Pass Filter) processing on the input signal and subtracting the result from the original input signal. Here, for example, a DC blocker can be used as the HPF.

This DC blocker is a filter for removing the DC bias component (ultra-low frequency component) from the input signal and extracting the AC component. The following equation (1) y [n] = x [n] －x [n- 1] ＋ r ＊ y [n-1]
It works under the difference equation like. Here, n is a sample position (sample index), and x [n] and y [n] are an input signal and an output signal at the sample position n, respectively. Further, the coefficient r takes a value of 0 to 1, and when r = 0, this filter is equivalent to the difference filter described below. By the way, even after this LPF processing is performed, the AC signal as a myoelectric signal remains.

Incidentally, both the LPF function, the notch filter function, and the LPF function in the pre-filter processing unit 212 described above may be performed in the mobile terminal 1. However, by performing the preprocessing up to the LPF processing on the side of the glasses 2 with the myoelectric sensor as in the present embodiment, the capacity of the signal transmitted to the mobile terminal 1 can be suppressed and good communication can be ensured. Further, it is also possible to incorporate these batteries and the signal preprocessing function into the frame portion instead of the box, so that the entire glasses 2 with the myoelectric sensor have a design that is not significantly different from ordinary glasses in appearance.

Similarly, according to FIG. 1, the mobile terminal 1 as the meal evaluation device of (a) above is
(A) An amount determined from a biological signal related to the user's head, and in the present embodiment, a myoelectric signal acquired by the glasses 2 with a myoelectric sensor, and is an amount determined from the user's "amount related to chewing" and "smiling". The biological signal information determination unit 111, which determines one or both of the "amounts",
(B) A meal that determines a "value related to the amount of meal" using a learned "meal amount estimation model" based on either or both of "amount related to chewing" and "amount related to smile". The quantity determination unit 112 and
(C) It has a feedback unit 114 that generates or selects and provides "information related to meal or health" of the user based on the determined "value related to the amount of meal".

As described above, the mobile terminal 1 as a meal evaluation device is "value related to the amount of meal" in providing "information related to meal or health" of the user based on the acquired biological signal (myoelectric signal). By taking this into consideration, it is possible to provide more appropriate information.

Here, the "value related to the amount of meal" can be a value corresponding to the physical absolute amount of meal, but it is not so, for example, the user feels that the amount of meal is large or small. It is also preferable to use it as a user's subjective index indicating whether or not the person feels.

Such an index is one of the very important indexes for evaluating the current diet. Specifically, the information on whether the user feels a large amount or a small amount of food is a good index for estimating the physical and mental condition / state of the user and the degree of appetite at the time of the meal, and is a more appropriate user. It is a suitable index for generating or selecting "dietary or health-related information".

In addition, the mobile terminal 1 further
(D) Dietary preference to determine "value related to dietary preference" using a learned "preference estimation model" based on either or both of "amount related to chewing" and "amount related to smile". Decision unit 113
It is also preferable to have. In this case, the feedback unit 114 generates or selects the user's "information related to meal or health" based not only on "value related to the amount of meal" but also on "value related to meal preference".

Here, the "value related to the taste of meal" can be, for example, a value equivalent to the evaluation value by a meal evaluation expert, but it is not so, for example, the user tastes (likes) the food in the meal. It is also preferable to use it as a subjective index of whether it feels unpleasant or unpleasant (dislikes).

Such an index is also a good index for estimating not only the tendency of the user's likes and dislikes but also the physical and mental condition / state of the user and the degree of appetite at the time of the meal, and is a more appropriate user's "meal or health". It is a suitable index for generating or selecting "information related to". Therefore, it is possible to generate or select a more appropriate user "information related to diet or health" by considering not only "value related to the amount of meal" but also "value related to dietary preference". It is.

FIG. 2 is a schematic diagram showing another embodiment of the biological signal acquisition device according to the present invention.

FIG. 2A shows a headphone 2'as a biological signal acquisition device according to the present invention. The headphone 2'is a wearable device linked to the mobile terminal 1 as a meal evaluation device, and is the same as the glasses 2 with a myoelectric sensor (FIG. 1) for a signal that may include a myoelectric signal as a detected biometric signal. Pre-filter processing (notch filter processing and LPF processing) is performed, and the pre-filtered signal is transmitted wirelessly (by Bluetooth (registered trademark), Wi-Fi (registered trademark), etc.) or (by cable connection). It is transmitted to the mobile terminal 1 via a wire.

Here, the wired connection may be connected to, for example, an analog audio input / output terminal (jack) for a headphone / microphone of a mobile terminal 1, or may be connected by a USB (Universal Serial Bus). .. In any case, the audio signal of the content is transmitted from the mobile terminal 1 to the headphone 2', for example, from the headphone 2'to the mobile terminal 1 via the wireless or wire, and is detected by the myoelectric sensor and pre-filtered. A signal including the myoelectric signal is transmitted.

In addition, the myoelectric sensor of the headphone 2'also has "detection + (plus) electrode", "reference- (minus) electrode", and "DRL (Driven Right)" as in the case of glasses 2 with myoelectric sensor (FIG. 1). It has three electrodes, "Leg) electrodes". Regarding the arrangement of these electrodes, as in the case of the glasses 2 with myoelectric sensor (FIG. 1), the reference electrode is in contact with one skin position near the cheek from around the left (or right) auricle, and the detection electrode is placed. It can be set to touch one skin position near the cheek from around the right (or left) pinna. Incidentally, in this case, the biological signal that can be detected includes a myoelectric signal caused by a masticatory motion or a mouth angle raising motion.

Of course, the arrangement of the electrodes of the myoelectric sensor is not limited to the above-mentioned form. For example, if the headphone 2'does not have an open-air earcup or earpad, the electrodes are placed near the cheeks from the surface of the support mechanism that contacts the skin around the ears to attach the headphones to the head. You may.

Further, as a head-mounted device provided with a similar myoelectric sensor and an electrode thereof, which is a biological signal processing device according to the present invention, the earphone 2 ″ shown in FIG. 2B can also be mentioned. This earphone 2'' also processes an input signal that may include a myoelectric signal as a detected biological signal, determines whether or not a myoelectric signal is generated, and determines the type of the generated myoelectric signal, and uses this determination result as a result. Such information is transmitted to the mobile terminal 1 via wireless or wired (cable). Further, for example, an audio signal of content is transmitted from the mobile terminal 1 to the earphone 2 ″ via the wireless or wired system.

Caused by "chewing" by using glasses 2 with a myoelectric sensor (FIG. 1), headphones 2'(FIG. 2 (A)), and earphones 2 "(FIG. 2 (B)) as described above. It is also possible to more reliably grasp the generation of the myoelectric signal to be performed, and further, it is possible to detect the myoelectric signal related to the mouth angle raising movement corresponding to the facial expression including "smile".

Incidentally, such a myoelectric signal can be used as a user interface when it is a signal due to a conscious reaction of the user. On the other hand, if the signal is an unconscious reaction, it can be used as a measurement result of the user's emotion and its transition. Further, for example, the user can determine whether or not the headphones 2 ″ or the earphones 2 ″ are attached / not attached from the detection status of the myoelectric signal.

In addition, a dry electrode is used for the glasses 2 with a myoelectric sensor (FIG. 1), the headphones 2'(FIG. 2 (A)), and the earphones 2'' (FIG. 2 (B)), which are worn at positions including the ears. Detection is performed using the property that the signal detected by the myoelectric sensor etc. is AC, while AC signals with small amplitude are not detected, and noise (artifact) due to the displacement of the dry electrode is also used as a biological signal. Since it is not detected, it is possible to more reliably detect a "periodic biometric signal" as an AC signal while reducing the amount of calculation.

[Meal evaluation device]
Returning to the functional block diagram of FIG. 1, the meal evaluation device 1 includes a signal interface 101, a chewing / smile information storage unit 102, a meal log storage unit 103, a feedback information storage unit 104, a camera 105, and a touch panel. It has a display (TP / DP) 106, a microphone 107, a speaker 108, and a processor memory.

Here, this processor memory stores one embodiment of the meal evaluation program according to the present invention, and also has a computer function, and carries out the meal evaluation process by executing this meal evaluation program. do. For this reason, the meal evaluation device 1 may be, for example, a personal computer (PC), a notebook or tablet computer, a smartphone, or the like equipped with the meal evaluation program according to the present invention.

Further, the processor memory includes a biological signal information determination unit 111 having an image information processing unit 111a, a meal amount determination unit 112, a meal preference determination unit 113, a feedback unit 114, a meal log generation management unit 121, and a meal. It has a quantity estimation model construction unit 122, a preference estimation model construction unit 123, a communication control unit 131, and an input / output control unit 132. It should be noted that these functional components can be regarded as the functions of the meal evaluation program stored in the processor memory. Further, the processing flow shown by connecting the functional components of the meal evaluation device 1 in FIG. 1 with arrows is also understood as an embodiment of the meal evaluation method according to the present invention.

Similarly, in the functional block diagram of FIG. 1, the biological signal information determination unit 111 uses the user's “biological signal” from the input signal including the biological signal acquired from the glasses 2 with the myoelectric sensor via the signal interface 101 and the communication control unit 131. One or both of "amount related to chewing" and "amount related to smile" (both in this embodiment) are determined. Here, the determined "amount related to mastication" and "amount related to smile" are first stored in the mastication / smile information storage unit 102, and then appropriately read out to be a meal amount determination unit 112 and a meal preference determination unit. It is also preferable that it is provided to 113 or the feedback unit 114.

Here, the biometric signal information determination unit 111 generates acceleration component data of the input signal, calculates a representative value related to the degree of data bias in a predetermined time interval in the acceleration component data, and has periodicity of this representative value. Based on, the "amount related to chewing" can be determined.

Specifically, the biological signal information determination unit 111 has a second-order difference filter and generates acceleration component data of the input signal subjected to LPF processing. This second-order difference filter can be configured to perform the difference filter processing twice on the input signal. The difference equation showing the principle of the difference filter used here is the following equation (2) y [n] ＝ x [n] －x [n-1].
It becomes the street. In the above equation (2), n is a sample position (sample index), and x [n] and y [n] are an input signal and an output signal at the sample position n, respectively.

In general, when a digital filter is used as described above, the amount of calculation increases as the digital filter becomes more sophisticated. This increase in the amount of calculation causes a decrease in battery life in a mobile device such as the mobile terminal 1, which becomes a big problem. On the other hand, the biological signal information determination unit 111 does not use a filter including trigonometric functions, for example, but uses a filter having a low order to process the biological signal. It can be suppressed.

Next, the biological signal information determination unit 111 divides the generated acceleration component data into predetermined time intervals (window analysis sections), and calculates a representative value related to the degree of data bias in each window analysis section. In general, the time-series data output from the biosensor can be fed back to the user in real time via the user interface by sequentially analyzing it in real time, which makes it very easy to use. At this time, by providing a window analysis section in advance and performing sequential analysis while shifting the analysis section, it is possible to perform analysis processing in a substantially real time.

In the present embodiment, when the sampling frequency of digitization in the signal conversion unit 211 of the glasses 2 with a myoelectric sensor is 512 Hz, the time series data of the acceleration component is input most recently every time 64 samples are input 128. Calculate the standard deviation SD using the sample as the window analysis interval. As a modification, the window analysis section is also set to 128 samples, and the time series data of the acceleration component is divided every 0.25 seconds (every 128 samples), and the standard deviation SD in the acceleration component data in the divided section is divided. May be calculated.

It should be noted that the value calculated here is not limited to the standard deviation SD as a matter of course, and various values can be adopted as long as it is a value related to the degree of deviation of the acceleration component data in the window analysis section.

The biological signal information determination unit 111 further calculates a weight W that is a monotonically decreasing function for the length of the time interval (sample number length len_th) in which the acceleration component in each window analysis interval remains continuously within a predetermined range. The inventor of the present application has found that when the myoelectric signal is not generated, this time interval is considerably longer than when it is generated.

Therefore, by determining a "weight W" that rapidly decreases (or at least becomes a monotonically decreasing function for this time interval) as this time interval becomes longer, and incorporating such a characteristic into the representative value SD _W , it is possible to make a line. It is possible to more reliably avoid the erroneous determination that the myoelectric signal is generated when there is no electric signal or when there is only noise.

Specifically, the derivation of this weight W will be described below. First, in the generated acceleration component data, the amplitude of the acceleration component is scanned from the beginning of the window analysis section to be weighted, and the sample number length len_th in which the amplitude less than the preset threshold th is continuous is determined. .. In addition, the number of observation samples obs that will specify the noise interval is set in advance. For example, th = 10 and obs = 15 can be set.

Incidentally, when there are a plurality of amplitude continuous sections less than the threshold value th in the window analysis section, the sample number length len_th may be the total number of samples in those sections. Alternatively, the number of samples in the longest amplitude continuous interval among them can be set as the sample number length len_th.

For example, as an exponential weight W as a function of the sample number length len_th, the following equation (3) W = exp (1-len_th / obs)
The weight W specified by can be set. Here, for example, if obs = 15 and len_th = 15, W = 1.0. Furthermore, as len_th increases, the exponential weight W decreases sharply and approaches zero. In fact, when len_th corresponds to the window analysis section length (128 samples), the exponential weight W becomes almost zero.

As yet another example, as the inverse proportional weight W as a function of the sample number length len_th, the following equation (4) W = 1 / ((len_th－obs) / a + 1)
The weight W specified by may be set. Here, it is preferable that a takes a value exceeding obs (a> obs) and sets the denominator to a positive value. In this inverse proportional weight W, for example, in the case of a = obs * obs (= 225), if len_th = 15, W = 1.0. Also, as len_th increases, the inverse proportional weight W decreases and approaches zero.

Of course, the weight W is not limited to that described above. If it is a monotonic decrease function of len_th, it can be adopted as a weight W, and preferably, if it is a function that approaches zero as len_th increases, and more preferably, if it is a function that asymptotics to zero, various functions are used as weight W. be able to. Here, for example, the weight W may be a linear function of len_th having a negative slope, but the exponential weight W as described above contributes to a more reliable determination of the generation of a biological signal.

Next, the biological signal information determination unit 111 further determines the calculated standard deviation SD as the representative value SD _W , which is weighted by the similarly calculated weight W. Specifically, the following equation (5) SD _W [k] = W [k] * SD [k]
Calculate the representative value SD _W [k] by. Here, in the above equation (5), k is the window position (window index), and [k] in SD _W [k], W [k], and SD [k] is the value at the window position k, respectively. Indicates that there is.

Next, the biological signal information determination unit 111 performs resonator filtering on the calculated time-series data of the representative value SD _W. This resonator filter processing is a processing for amplifying the specified periodicity component when the periodicity specified in advance exists in the time series data of the calculated representative value SD _W. By performing such processing, it is possible to carry out the periodic signal generation determination processing described later, that is, the determination processing of whether or not the time series data of the representative value has periodicity, with higher determination accuracy. It is.

Specifically, the resonator filter processing is performed by the following difference equation (6) y [n] = a ₁ * y [n-1] + a ₂ * y [n-2] + b ₀ * x [n].
Realized by. In the above equation (6), n is a sample position (sample index), and x [n] and y [n] are an input signal and an output signal at the sample position n, respectively. The coefficients a ₁ , a ₂ and b ₀ are given by the following equation (7) a ₁ = 2 * exp (−π * Q / f _s ) * cos (2π * f ₀ / f _s ).
a ₂ = -exp (-2π * Q / f _s )
b ₀ = 1 －a ₁ －a ₂
Is calculated with. Here, Q is the resonance degree (Q> 0), f _s is the sliding window analysis frequency (unit is Hz), and f ₀ is the resonance frequency (unit is Hz).

Next, the biological signal information determination unit 111 performs multi-resolution analysis (MRA) processing on the input signal before generating the acceleration component data, and the time-series data of the signal amplitude after the MRA processing is predetermined. The time interval indicating hysteresis is determined as the signal generation time interval in which some biological signal is generated.

Here, the inventor of the present application has newly found that it is possible to determine whether or not a biological signal is generated by performing MRA processing on an input signal from a sensor. Based on this discovery, the biological signal information determination unit 111 can specify the generation time interval of the biological signal.

In any case, in the signal generation determination process to be performed after this, not only the periodicity of the calculated representative value SD _W but also the determined signal generation time interval is taken into consideration, so that the periodic biological signal It becomes possible to more reliably determine the occurrence of. Of course, it is also possible to omit this generation time interval determination processing and perform the signal generation determination processing only by the periodicity of the representative value SD _W.

Further, in the present embodiment, the biological signal information determination unit 111 determines the generation of a periodic biological signal based on the periodicity of the time-series data of the representative value SD _W subjected to the resonator filter processing. By the way, in the case of the present embodiment, since the periodic biological signal determined to have been generated is acquired from the glasses 2 with the myoelectric sensor attached to the head, the myoelectric telegraph caused by "chewing". It is judged to be a number. That is, the biological signal information determination unit 111 determines the generation of the myoelectric signal related to mastication.

Here, as a determination process of whether or not the time series data of the representative value SD _W has periodicity,
(A) Calculate the peak position when the representative value SD _W fluctuates beyond the predetermined range.
(B) When the time interval between adjacent peak positions is within a predetermined time range, it can be determined that the time-series data of the representative value SD _W has periodicity and that a periodic biological signal is generated. can.

Next, when the biological signal information determination unit 111 determines that a periodic biological signal (myoelectric signal related to "chewing" in the present embodiment) is not generated in the determined signal generation time interval, this signal is generated. Determine the type of biological signal generated in the time interval. For example, it is determined that the generated biological signal is due to the movement of raising the corner of the mouth (“smile”).

More specifically, for example,
(A) When a biological signal generation time interval exists in the determination target period and a predetermined number or more of peaks (peak positions) are detected in the time interval, a periodic biological signal is generated. And make a judgment.
(B) Further, when the biological signal generation time interval exists in the determination target period, but the peak (peak position) is not detected in the time interval, some (aperiodic) biological signal is generated. Assuming that there is, the type of the biological signal is determined. Incidentally, this type determination will be described in detail later.
(C) Further, when the biological signal generation time interval does not exist in the determination target period, it is determined that the biological signal is not generated.

Here, when the determination in (a) above is made, the count of the periodic biological phenomena (for example, mastication movement) occurring is further based on the generated periodic biological signal (for example, the myoelectric signal related to mastication). (For example, calculate the number of chews). On the other hand, when the determination in (a) above is performed, the number of occurrences of the biological signal whose type is determined, that is, the number of occurrences of the corresponding biological phenomenon is counted. This count will also be described later.

Here, specific examples of the above-mentioned biological signal type determination processing and biological signal counting processing will be described below.

As an embodiment of the biological signal type determination, the biological signal information determination unit 111 averages the input signals in the time interval related to the determination that the periodic biological signal is not generated (although the biological signal is generated). It is also preferable to calculate the power frequency (MPF, mean power frequency) and determine the type of the generated biological signal based on the height of the MPF.

Here, the inventor of the present application performs frequency analysis processing by fast Fourier transform (FFT, Fast Fourier Transform) or the like on the input signal acquired by using the glasses 2 with a myoelectric sensor, and performs MPF in each window analysis section. As a result of the calculation, it was found that the type of the generated myoelectric signal can be determined by the threshold determination of the MPF value.

Specifically, for example, when the MPF value exceeds a predetermined threshold value, it is determined that the generated biological signal is a myoelectric signal due to a clenching operation, while the MPF value is equal to or less than this predetermined threshold value. , The generated biological signal can be determined to be a myoelectric signal due to the mouth angle raising motion (“smile”).

Further, in the window analysis section, it is possible to calculate the signal strength, for example, the standard deviation SD'of the amplitude, and use this value for determining the type of the generated signal in the same manner as the MPF value.

By the way, in general, a considerable amount of calculation is required for frequency analysis processing such as FFT, but in this embodiment, such frequency analysis is performed only in the time interval in which it is determined that some biological signal is generated. Therefore, the amount of calculation can be significantly reduced even though the frequency analysis process is performed in the type determination.

Further, as another embodiment of the biological signal type determination, the biological signal information determination unit 111 is standard in the time interval related to the determination that the periodic biological signal is not generated (although the biological signal is generated). A feature quantity including the deviation SD'and the MPF value (MPF) of the input signal, for example, {SD', MPF} is calculated, and this feature quantity is a unit space set by the feature quantity of the input signal corresponding to the reference state. It is also preferable to calculate the degree of separation, which is the degree of separation from the above, and determine the type of the generated biological signal based on the calculated degree of separation.

In this case, specifically, the biological signal information determination unit 111 is in a state in which a predetermined biological signal is generated and a state in which the biological signal is not generated, based on the degree of separation from the unit space related to the reference state in which the biological signal is not generated. The generation of a predetermined biological signal is determined based on the amount obtained by subtracting the degree of separation from the unit space related to the reference state and the degree of separation from the unit space related to the reference state in which the predetermined biological signal is generated. be able to.

Here, as the above unit space and the degree of separation,
(A) A value calculated from the unit space in the MT (Mahalanobis Taguchi) method and the Mahalanobis distance.
(B) A value calculated from the unit space in the MTA (Mahalanobis-Taguchi Adjoint) method and the Mahalanobis distance.
(C) A value calculated from the unit space and the characteristic value in the T method, or (d) a value calculated from the unit space and the RT distance in the RT (Recognition Taguchi) method can be adopted. Incidentally, the inventor of the present application has also found through experiments that such a method for determining the type of biological signal is effective.

Of these, when the MT method is used, for example, when discriminating the myoelectric signal due to the operation of raising the angle of the mouth,
(A) Facial expressionless state and mouth angle raised state (combined state group)
(B) Expressionless state (c) Design three unit spaces for the raised mouth angle state, and calculate the degree of separation from these unit spaces in the input signal as distance 1, distance 2, and distance 3, respectively. It is known that a more suitable determination result can be obtained by setting (distance for determination) = (distance 2)-(distance 1)-(distance 3). Specifically, when such a determination distance exceeds a predetermined threshold value, it can be determined that the generated biological signal is a myoelectric signal due to a mouth angle raising operation (“smile”).

Next, as a preferred embodiment of the biological signal counting process, a method of utilizing the hysteresis of time series data will be described.

FIG. 3 is a graph showing an embodiment of biological signal counting processing using hysteresis in time-series data of processed biological signals. The graph of FIG. 3 is a line graph connecting time-series data points of the determination distance calculated when the biological signal type determination is performed by the MT method with a line segment.

The biological signal information determination unit 111 (FIG. 1) is
(A) The point at which the judgment distance (the polygonal line indicating the transition) crosses the threshold Thh line from the lower side (the smaller value) to the upper side (the larger value) is defined as the count start point (circle).
(B) The counting end point (triangle mark) is the point where the judgment distance (the polygonal line indicating the transition) crosses the threshold Thr (<Thh) line from above (larger value) to downward (smaller value). ), As
The count is incremented by 1 for each set of these count start points and the count end points following them.

In the embodiment of FIG. 3, since there are four of these sets, the biological signal (of the type related to the determination distance used) was generated four times at the stage when these four sets were determined by the graph. (The number of biological signals generated is counted as 4). Here, by appropriately setting the threshold value (thh) at the start point and the threshold value (thl) at the end point, it is possible to prevent chattering of the signal generation determination result.

Further, when the count end point is not determined even after a predetermined time threshold T _max has elapsed after the count start point is determined, the biological signal information determination unit 111 once from this count start point to that point. After counting the time threshold value T _max , the noise determination period may be set until the determination distance _W falls below the threshold value Thr. In this case, in the embodiment of FIG. 3, it is finally determined that the number of occurrences of the biological signal (of the type related to the determination distance used) is five.

As a modification, when the biological signal generation determination is performed by deriving the SD _W value, a hysteresis graph with the vertical axis of FIG. 3 as the SD _W value is generated (to the height of the MPF at the time of the signal generation determination). It is also possible to count the number of occurrences of a biological signal (related to "smile" or "chewing") for which the biological signal type is determined based on the above.

The process of determining "mastication" and "smile" in the biological signal information determination unit 111 has been described in detail above, but hereinafter, the "amount related to mastication" and "amount related to mastication" determined by the biological signal information determination unit 111 (FIG. 1) The amount of smiles will be explained in detail.

First, as the "amount related to chewing",
(S1) Total number of chews, (S2) Total chewing time, (S3) Total number of mouths, (S4) Total interval time between one bite and the next bite, (S5) Total number of intervals, (S6) Chewing pace, ...・ ・
At least one of them can be adopted. In addition, as "amount related to smile",
(E1) Total number of smiles, (E2) Total smile time, ...
At least one of them can be adopted.

Here, the total number of chews in (S1) is the total number of chews up to that point for each bite movement, with one section determined to be "chewing" by the biological signal information determination unit 111 as a bite. On the other hand, it can be determined by adding the number of times of chewing in the bite.

For the total chewing time of (S2), the start time and end time of one bite are recorded to calculate the bite operation time, and each bite movement occurs with the bite with respect to the total chewing time up to that point. It may be determined by adding the bite operation time. Further, the total number of units in (S3) can be determined by adding 1 to the total number of units up to that point for each bite operation.

For the total interval time in (S4), the interval time between the end time of one bite and the start time of the next bite is added to the total interval time up to that point each time the next bite operation occurs. It can be decided by going. Further, the total number of intervals in (S5) can be determined by adding 1 to the total number of intervals up to that point for each occurrence of the interval time.

The mastication pace in (S6) is the mastication tempo or mastication rhythm, and the following formula (8) "Mastication pace" = "total number of mastications" / "total mastication time" (unit: times / second)
Can be determined using. Alternatively, using the data from one bite, the following formula (9) "Mastication pace" = "Number of chews in one bite" / "Mastication time in the bite" (unit: times / second)
May be determined by.

Further, for the total number of smiles in (E1), the number of occurrences of the smile section is counted with one section determined to be "smile" as the smile section (1 is added to the total number of smiles for each smile section occurrence). It may be decided by. For the total smile time in (E2), the start time and end time of the smile section are recorded to calculate the smile time, and the smile time is added to the total smile time up to that point for each occurrence of the smile section. Can be determined by

Similarly, in the functional block diagram of FIG. 1, the meal amount determination unit 112 has learned the amount of meal based on one or both of "amount related to chewing" and "amount related to smile" (both in this embodiment). The estimation model is used to determine the "value related to the amount of food".

Here, the "value related to the amount of meal" can be a value corresponding to the physical absolute amount of meal, but it is not so, for example, the user feels that the amount of meal is large or small. It is also preferable to use it as a user's subjective index indicating whether or not the person feels. Specifically, for example, it can be a meal amount value from 0 (less) to 10 (more).

In addition, the meal amount estimation model used is characterized by inputting "amount related to mastication", for example, all of the above (S1) to (S4), and "amount related to smile", for example, the above (E1) and (E2). As the amount, as the output, "value related to the amount of meal", for example, the meal amount value from 0 to 10 is output. The process of constructing this meal amount estimation model will be described later.

The dietary preference determination unit 113 uses a learned dietary preference estimation model based on one or both of "amount related to mastication" and "amount related to smile" (both in the present embodiment), and uses a "meal preference estimation model". "Value related to taste" is determined.

Here, the "value related to the taste of meal" can be, for example, a value equivalent to the evaluation value by a meal evaluation expert, but it is not so, for example, the user tastes (likes) the food in the meal. It is also preferable to use it as a subjective index of whether it feels unpleasant or unpleasant (dislikes). Specifically, for example, it can be a meal preference value from 0 (dislike) to 10 (like).

In addition, the dietary preference estimation model used is characterized by inputting "amount related to mastication", for example, all of the above (S1) to (S6), and "amount related to smile", for example, the above (E1) and (E2). As an amount, as an output, a "value related to a meal preference", for example, a meal preference value from 0 to 10 is output. The process of constructing this meal preference estimation model will also be described later.

Similarly, in the functional block diagram of FIG. 1, the meal log generation management unit 121 generates and manages a “meal log” including learning data used for constructing the above-mentioned meal amount estimation model and meal preference estimation model. ..

Here, the meal log records data on meals by the user, and in the present embodiment, the above-mentioned (S1) total number of chews, (S2) total chew time, (S3) total number of mouths, (S4) bite. With the total interval time between and the next bite, (S5) total interval count, (S6) chewing pace, (E1) total smile count, (E2) total smile time,
(A1) User identifier (ID), (A2) Food / menu name (meal name),
(B1) Meal time, (B2) Meal start time,
(C1) Preference (value related to meal preference), (C2) Meal amount (value related to the amount of meal),
(D1) Meal form, (D2) Time allowance, (D3) Mental and physical condition / condition,
Is included as a record item.

Here, the input / output control unit 132 displays, for example, a meal self-evaluation input screen on the touch panel display 106 in order to acquire the data to be recorded in the above-mentioned meal log recording item, and the user finishes one meal. It is also preferable to receive data input by the user each time and provide it to the meal log generation management unit 121.

Specifically, the dish / menu name of (A2) is acquired by text input or selection by the user, and the preference (value related to the meal preference) of (C1) is obtained by the user operating the display slider or the like. It is possible to accept and acquire one value setting input from 0 (dislike) to 10 (like).

Regarding the meal time of (B1), the time when the user wears the glasses 2 with a myoelectric sensor and starts the meal is set as the meal start time (obtained from the clock of the glasses 2), and then the meal time is set. The time when the glasses 2 with a myoelectric sensor are stopped at the end of the meal is set as the end time of the meal (also obtained from the clock of the glasses 2), and the meal time can be determined by the difference between these times.

Furthermore, the amount of meal (value related to the amount of meal) in (C2) can also be obtained by accepting the setting input of one value from 0 (less) to 10 (more) by the user's display slider operation or the like. can.

Furthermore, the meal form of (D1) is acquired by accepting the setting input of lonely meal or dinner by the user selecting a display icon or the like and, for example, tapping, and the time margin of (D2) is the user. Accepts and obtains one value setting input from 0 (quick meal) to 10 (loose meal) by operating the display slider etc., and the physical and mental condition and state of (D3) are also the user's You may accept and acquire the setting input of one value from 0 (absolute condition) to 10 (excellent condition) by operating the display slider or the like.

As described above, the meal log generation management unit 121 is the results of biological signal analysis (S1) to (S6), (E1) and (E2), and the results of self-evaluation of meals by the user (A1), (A2). , (B1), (B2), (C1), (C2), and (D1)-(D3) are associated and recorded as a "meal log".

Here, the characteristics (reasons listed in the meal log) of the recorded items listed in the meal log described above will be described. The total number of chews in (S1) may be affected by taste, amount of food, time allowance, physical and mental condition / condition, and the like. For example, the more food you eat, the more times you chew. It also affects the user's tendency to reduce the total number of chews in order to eat quickly, or to increase the total number of chews in order to taste well. Is considered to receive.

In addition, the total interval time in (S4) may be affected by taste, dietary pattern, time allowance, and mental and physical condition / condition. If you like food, you may try to eat it earlier and the total interval time may be shorter, and if you dislike food, you may want to increase the total interval time. Furthermore, if the eating style is lonely, it will be shorter because there is no conversation, and it will be shorter even if there is no time to spare. On the other hand, if the physical and mental condition is not good, the total interval time is considered to be long.

Furthermore, it is conceivable that the mastication pace of (S6) will be faster if, for example, there is no time to spare, or if the meal form is lonely and the taste is preferred (delicious). In addition, the total smile time in (E2) may be affected by taste, dietary pattern, and mental and physical condition / condition. For example, if the meal is very delicious, it will make you smile, and if you enjoy the conversation during the meal, you will make a smile. On the other hand, if the physical and mental condition is not good, it may be difficult to express a smile.

In addition, the meal time of (B1) may be affected by the amount of meal, meal form, time allowance, and mental and physical condition / condition. Furthermore, the preference of (C1) (value related to meal preference) and the amount of meal (value related to the amount of meal) of (C2) are affected by the physical and mental condition / condition and appetite of the user at the time of meal. it is conceivable that.

In addition, the meal form of (D1) is an index showing the possibility of conversation during meals. In addition, the time margin of (D2) is an index showing the degree of time that can be spent on meals. Furthermore, the mental and physical condition / state of (D3) is an index showing mental disorders due to stress and the like, and physical disorders due to fatigue and illness.

In addition to the above-mentioned record items, the meal log may contain, for example,
(G1) Image data related to meals (ID or file name)
(G2) Voice data related to meals (ID or file name)
(H1) Meal position data (position data (latitude / longitude, etc.), address, store name, home / non-home value, etc.)
Is also preferably recorded as a record item.

Here, for the image data related to the meal of (G1), for example, a camera (image sensor) capable of taking a picture in the direction of the user's line of sight is attached to the glasses 2 with a myoelectric sensor, and the food of the meal is taken at the start of the meal. Can be obtained by Alternatively, it may be acquired by taking a picture with the camera 105 of the mobile terminal 1 carried by the user.

Further, (G2) voice data related to meals can be acquired, for example, by recording a conversation or the like during meals with a microphone 107 of a mobile terminal 1 carried by the user. Here, using known voice recognition technology for the acquired voice data, whether or not the voice belongs to the user, whether or not there is a voice other than the user (whether or not there is a conversation partner). ) Is determined, and it is also preferable to record the determination result in association with each other. The voice data may be acquired by a microphone attached to the glasses 2 with an EMG sensor.

Further, for the meal position data of (H1), for example, the current position (latitude / longitude) information at the time of meal is determined by using the positioning function (such as GPS) equipped in the mobile terminal 1 carried by the user. Can be obtained by. For example, the determined latitude / longitude information itself may be used as the meal position data, but the restaurant name or address information of the place where the person is currently staying is acquired from the current position information by using a known map information search technique, and the meal position data is obtained. It can also be.

Similarly, in the functional block diagram of FIG. 1, the meal amount estimation model construction unit 122 has at least "meal" for one or both of "amount related to chewing" and "amount related to smile" (both in this embodiment). Correct answer data using multiple "meal logs" recorded by associating user-reported information (results of self-evaluation of meals) with respect to "values related to the amount of meals" (that is, "values related to the amount of meals" in the "meal log" As), build a trained "meal estimation model".

In addition, the preference estimation model construction unit 123 has at least a "value related to meal preference" for one or both of "amount related to chewing" and "amount related to smile" (both in the present embodiment). Learned "meal" using multiple "meal logs" recorded by associating user-reported information (meal self-evaluation results) (that is, using the "value related to meal preference" in the "meal log" as correct answer data) Build a "preference estimation model".

Here, both the "meal amount estimation model" and the "meal preference estimation model" can be constructed by using various machine learning algorithms such as a neural network. For example, the feature amount of the recorded item which is a numerical value of 0 to 10 is replaced with a binary value (for example, 0 or 1) of whether or not it is 5 or more, and a classifier such as SVM (Support Vector Machine) or random forest is used. May be constructed.

Further, in the construction of both the "meal amount estimation model" and the "meal preference estimation model", the above-mentioned (A1) user ID and (A2) dish / menu name (meal name) are used as the feature amounts included in the teacher data. , (B1) meal time, (B2) meal start time, (C1) preference (value related to meal preference), (C2) meal amount (value related to the amount of meal), (D1) meal form, (D2) It is also preferable that at least one of the time allowance and (D3) mental and physical condition / state is adopted. Here, in particular, it has been found that it is more preferable to adopt the meal time of (B1) in terms of estimation accuracy.

As a matter of course, the recorded item adopted as the feature amount included in the teacher data is, when the meal amount determination unit 112 estimates the "value related to the amount of meal", and further, the meal preference determination unit 113 "meal preference". When estimating "value related to", it will be used as a feature quantity to be input to the model. Here, the estimation of the "value related to the amount of meal" and the "value related to the taste of meal" can be carried out, for example, for each meal (every time a meal log is generated).

Further, both the "meal amount estimation model" and the "meal preference estimation model" are constructed or updated when the number of meal logs newly generated and newly stored in the meal log storage unit 103 exceeds a predetermined number. It is also preferable.

The meal log generation management unit 121 may generate a meal log including the estimation results of the "value related to the amount of meal" and the "value related to the taste of meal". Here, for example, the estimation result is displayed on the touch panel display 106 via, for example, the input / output control unit 132, and if there is a change after being confirmed by the user, the change result (input by the user) is acquired. It is also preferable to add to the meal log or change / update the meal log.

This also makes it possible to reduce the burden of inputting meal log items by the user. When the meal log is changed / updated, it is also preferable to reconstruct or update the "meal amount estimation model" and the "meal preference estimation model" using the changed / updated meal log.

Similarly, in the functional block diagram of FIG. 1, the feedback unit 114 is based on one or both of the determined “value related to the amount of food” and “value related to food preference” (both in the present embodiment). Generate or select and provide the user's "dietary or health-related information".

For example, if the "value related to the amount of food" is a value corresponding to 8 to 10 (more), reduce the amount of food such as "Be careful not to overeat" from the audio content prepared in advance. Audio information, including recommended comments, may be selected and provided. At this time, in the "***" part of the utterance template that says "Be careful of ***" prepared in advance, "Overeating (preset for the case of values corresponding to 8 to 10)" It is also preferable to select and insert "" to generate the audio information to be provided.

In addition, when the "value related to the amount of meal" is a value corresponding to 0 (less) to 3 and the "value related to the taste of meal" is a value corresponding to 0 (dislike) to 3 in advance. From the prepared voice information, voice information including a comment recommending the change of the meal contents such as "How about changing the menu" may be provided. Here, of course, instead of the voice information described above, it is also preferable to provide screen information including a text presenting the same content.

The above-mentioned voice information may be output from the speaker 108 via the input / output control unit 132, and the screen information may be displayed on the touch panel display 106 via the input / output control unit 132.

In any case, the "diet or health-related information" generated or selected by the feedback unit 114 is first stored in the feedback information storage unit 104, and then appropriately read out to display the touch panel display 106 and the speaker 108. It is also preferable that it is provided to the user via. Further, it can be transmitted to an external information processing device such as a smart speaker or a conversation robot via the communication control unit 131 and the signal interface 101 and provided to the user.

Incidentally, this conversation robot has a camera, and the meal image taken here may be used as image data related to the meal of (G1) in the "meal log". In this case, it is possible to create a situation in which the conversation robot sees the user's meal and exchanges a conversation about the meal with the user.

Further, the feedback unit 114 is based on not only the above-mentioned "value related to the amount of meal" and "value related to the taste of meal" but also the "meal log" related to at least one past time point, and the user's "value". It is also preferable to generate or select "dietary or health-related information".

For example, the feedback unit 114 may use the "at least one" of the above-mentioned total number of chews of (S1), total interval time of (S4), and (S6) chewing pace in the meal log. It is also preferable to select at least one from preset music or audio contents according to the conditions satisfied by "one" and "value related to the amount of meal" and use it as "information related to meal or health" of the user.

Specifically, for example, when the increase in the total number of chews of (S1) during a meal per unit time, that is, when the rate of increase in the number of chews exceeds a predetermined threshold, the pace of the user's meal (chewing pace) is synchronized with the song. You may play Andante (at walking speed) songs prepared in advance, for example, with a tempo of about 1.1 to 1.2 Hz (66 to 76 bpm), for the purpose of lowering the temperature.

On the other hand, when the rate of increase in the number of chews is lower than the above-mentioned predetermined threshold, the tempo prepared in advance, for example, 1.2 to 1.5 Hz (for example,) is prepared in advance for the purpose of increasing the pace of the user's meal (chewing pace) in synchronization with the song. You may play Moderato songs (at medium speed) of around 76-96bpm.

Regarding the predetermined threshold value of the above-mentioned increase rate of the number of chews, for example, from the set of accumulated meal logs,
(10) "Average chewing pace" = "sum of total chewing frequency" / "sum of total meal time" (unit: times / second)
Can be calculated and this "average chewing pace" can be set as a predetermined threshold value.

By the way, the "estimated total number of chews" for the elapsed meal time can be obtained as "estimated total number of chews" = "average chewing pace" x "elapsed meal time" (unit: times).

As a modification, when the rate of increase in the number of mastications is between (“average mastication pace” −p ₁ (predetermined positive value)) and (“average mastication pace” + p ₂ (predetermined positive value)), the user It is also preferable to play a given feedback content to compliment the user, assuming that the pace of the meal is appropriate.

Furthermore, after the meal is completed, a voice including advice for showing a difference from the target value and moving toward the target value may be played according to the acquired meal time of (B1) and the total number of chews of (S1). .. Furthermore, when the total number of chews of (S1) reaches 500 times, for example, a voice indicating that half of the target of 1000 times is reached is played, and then when the target of 1000 times is reached, the voice is played. You can also play a voice that shows praise.

Further, when the "value related to the taste of meal" is a value corresponding to 0 (dislike) to 3, the joke content can be reproduced to cause a smile and give a positive illusion regarding the taste.

Further, the feedback unit 114 determines the total number of smiles and / or the total smile time and "the amount of meal" based on the total number of smiles (E1) and / or the total smile time of (E2) described above in the meal log. It is also preferable to select at least one of the music or audio contents preset according to the condition that the "value" is satisfied and use it as the user's "information related to diet or health". For example, when the total number of smiles falls below a predetermined threshold value, audio content such as "Today's pun" prepared in advance may be provided.

Further, the feedback unit 114 generates "information related to meal or health" of the user based on the above-mentioned voice data related to the meal of (G2) and / or the meal position data of (H1) in the meal log. Alternatively, it is also preferable to select.

For example, when it is determined that the user is talking (during a meal) from the identification result of the voice data related to the meal, the content suitable for the meal prepared in advance may be selected and played. Here, it is also preferable to select music content that does not interfere with the conversation, or to select, for example, comedy content when the conversation is interrupted.

Further, the feedback unit 114 may have a meal history evaluation function that aggregates and analyzes past meal logs stored in the meal log storage unit 103. For example, it is also preferable to aggregate and analyze meal logs for the past week.

Specifically, the feedback unit 114 raises the possibility of physical and mental disorders when, for example, in two or more consecutive meals, the "value related to meal preference" becomes a value corresponding to 0 (dislike) to 3. In consideration of this, it is also preferable to reproduce and provide the audio content of the content that cares about the physical condition to the user. In addition, when the mastication pace is slower than the average mastication pace, or when the total number of mastications measured is smaller than the total number of mastications estimated from the amount of meal estimated based on the image, conversely, from the amount of the meal. If the measured meal time is longer than the expected meal time, or even if the mastication is not measured at all when playing the joke content, we will give feedback considering the possibility of physical and mental disorders. be able to.

In addition, the feedback unit 114 presents goals such as "chewing 1000 times or more per meal" and "spend 20 minutes or more on meal time" at the start of a meal, and based on the aggregated result of the meal log, each meal. It is also preferable to present the achievement status of the goal to the user. Here, the target achievement period may be presented to the user as, for example, two weeks, and if the target is achieved within the period, the audio content praising the user may be played.

Further, the meal logs of the same meal position data (for example, the same restaurant name) may be aggregated and information for looking back on the past meals at the meal position (the restaurant) may be presented. In addition, when the meal time (average value) of (B1) at the meal position is below a predetermined threshold (for example, when the meal time tends to be generally short at a favorite set meal shop), at the start of meal at the meal position, It is possible to provide audio content that recommends spending more time on meals.

Further, it is also possible to take statistical values such as meal time, number of chews, and preferences for each restaurant, generate information comparing the restaurants, and present it to the user.

It is also preferable that the feedback unit 114 aggregates past meal logs, generates ranking information, and provides it to the user. Specifically, the user obtains information on three meals, for example, (G1), which are the highest, second, and third in terms of the number of total chews, the length of meal time, and the height of preference. May be presented to. As a result, the user can look back on past meals and enjoy them, or use them as a reference for future eating habits.

[Meal evaluation method]
FIG. 4 is a schematic diagram for schematically explaining an embodiment of the dietary evaluation method according to the present invention.

As shown in FIG. 4, in the meal evaluation device 1 of the present embodiment, first, from the "myoelectric signal" acquired from the glasses 2 with a myoelectric sensor, "amount related to mastication" and "amount related to smile". Is generated. Then, from these amounts, the "value related to the amount of meal" is estimated using the learned meal amount estimation model, and also from these amounts, the "diet preference" is estimated using the learned meal preference estimation model. "Relevant value" is estimated.

After that, the meal evaluation device 1 generates or selects "information related to meals or health" based on the estimated "values related to the amount of meals" and "values related to meal preferences" and provides them to the user. .. That is, the dietary evaluation device 1 eventually determines "information related to diet or health" suitable for the user from the acquired "myoelectric signal".

Here, the meal amount estimation model and the meal preference estimation model, which are the main components of the present meal evaluation method, carry out the learning process using the recorded contents of the generated "meal log" as also shown in FIG. It is built by. It should be noted that this "meal log" is generated from the "meal self-evaluation result" by the user and the generated "amount related to chewing" and "amount related to smile".

[Meal evaluation system]
FIG. 5 is a schematic diagram showing another embodiment of the dietary evaluation system according to the present invention.

As shown in FIG. 5, the meal evaluation system of the present embodiment is
(A) A mobile terminal 4 that acquires a biological signal (myoelectric signal) from glasses 2 with a myoelectric sensor, and
(B) A meal evaluation server 3 connected to a mobile terminal 4 by communication is provided.

Similarly, in the upper functional block diagram in FIG. 5, the mobile terminal 4 has a signal interface 401, a communication interface 402, a chewing / smile information storage unit 403, a feedback information storage unit 404, a camera 405, and a touch panel display ( It has a TP / DP) 406, a speaker 407, and a processor memory.

Here, this processor memory stores one embodiment of the meal evaluation program on the terminal side according to the present invention, and also has a computer function, and by executing the meal evaluation program on the terminal side. , Carry out meal evaluation processing on the terminal side. For this reason, the mobile terminal 4 may be, for example, a personal computer (PC), a notebook or tablet computer, a smartphone, or the like equipped with the terminal-side meal evaluation program according to the present invention.

Further, the processor memory has a biological signal information determination unit 411, a feedback unit 412, a communication control unit 421, and an input / output control unit 422. It should be noted that these functional components can be regarded as the functions of the meal evaluation program on the terminal side according to the present invention, which are stored in the processor memory. Further, the processing flow shown by connecting the functional components of the mobile terminal 4 in FIG. 5 with arrows is also understood as an embodiment of the meal evaluation method on the terminal side according to the present invention.

Further, in the lower functional block diagram of FIG. 5, the meal evaluation server 3 has a communication interface 301, a mastication / smile information storage unit 302, a meal log storage unit 303, and a processor memory.

Here, this processor memory stores one embodiment of the server-side meal evaluation program according to the present invention, and also has a computer function by executing the server-side meal evaluation program. , Carry out meal evaluation processing on the server side. For this reason, the meal evaluation server 3 may be, for example, a personal computer (PC), a notebook computer, or the like equipped with the server-side meal evaluation program according to the present invention.

Further, the processor memory includes an image information determination unit 311, a meal amount determination unit 312, a meal preference determination unit 313, a meal log generation management unit 321 and a meal amount estimation model construction unit 322, and a preference estimation model construction unit. It has 323 and a communication control unit 331. It should be noted that these functional components can be regarded as the functions of the server-side meal evaluation program stored in the processor memory. Further, the processing flow shown by connecting the functional components of the meal evaluation server 3 in FIG. 5 with arrows is also understood as an embodiment of the meal evaluation method on the server side according to the present invention.

Here, among the functional component units in the mobile terminal 4 and the meal evaluation server 3, the functional component unit having the same name among the functional component units of the mobile terminal 1 in FIG. 1 is substantially the same as the functional component unit having the same name. It has the function of, and the description is omitted here. Hereinafter, the functional components that play a characteristic role in this meal evaluation system will be described.

Incidentally, in the meal evaluation system of the present embodiment, a plurality of mobile terminals 4 can be connected to the meal evaluation server 3 by communication. Therefore, the meal log generated and managed by the meal log generation management unit 321 of the meal evaluation server 3 is stored and managed separately for each (A1) user ID of the corresponding user, and the meal amount determination unit 312 is also stored and managed. And the meal preference determination unit 313 determines "amount related to the amount of meal" and "amount related to meal preference" for each user ID, and these amounts are transmitted to the mobile terminal 4 of the corresponding user. To.

The image information determination unit 311 of the meal evaluation server 3 acquires "image information" related to the user's meal generated by the camera 405 of the mobile terminal 4 via the communication interface 301 and the communication control unit 331.
(A) One or both of the trained "image meal type estimation model" and "image meal amount estimation model" constructed by using a known image recognition model construction technique based on the acquired "image information" (a) In this embodiment, both) are used to determine one or both (both in this embodiment) of "meal type information (for example, food / menu name)" and "amount related to the amount of meal" in the user's meal. And then,
(B) A trained "biological signal information estimation model" based on one or both of the determined "meal type information (cooking / menu name)" and "amount related to the amount of meal" (both in this embodiment). (In this embodiment, a mastication-related amount estimation model and a smile-related amount estimation model) ”, one or both of the user's“ mastication-related amount ”and“ smile-related amount ”(both in this embodiment). ) Is determined.

Next, the meal amount determination unit 312 in which the "amount related to chewing" and the "amount related to smile" determined by the image information determination unit 311 are input, learns in the same manner as the process in the mobile terminal 1 of FIG. Using the completed "meal amount estimation model", a "value related to the amount of food" is generated. Further, the meal preference determination unit 313 in which the “amount related to chewing” and the “amount related to smile” determined by the image information determination unit 311 are input, is the same as the process in the mobile terminal 1 of FIG. The "value related to the meal preference" is generated by using the "meal preference estimation model".

Next, the "value related to the amount of meal" and the "value related to the taste of meal" generated in this way are transmitted to the mobile terminal 4 (feedback unit 412) via the communication control unit 331 and the communication interface 301. To. Next, the feedback unit 412 of the mobile terminal 4 is based on the "value related to the amount of meal" and the "value related to the taste of meal" of the user, as in the process of the mobile terminal 1 of FIG. "Health-related information" is generated or selected and provided to the user.

Here, the "meal amount estimation model" and the "meal preference estimation model" in the meal evaluation server 3 use the meal log generated by the meal log generation management unit 321 as in the process of the mobile terminal 1 of FIG. It is constructed by learning from the past "amount related to chewing" and "amount related to smile".

Further, the "chewing-related amount estimation model" and the "smile-related amount estimation model" can also be constructed using the meal log generated by the meal log generation management unit 321. Furthermore, the information recorded in the meal log can be used for the food / menu name (meal name) and meal amount used when learning the "image meal type estimation model" and the "image meal amount estimation model". can.

By the way, in the present embodiment, the meal log generation management unit 321 carries the "amount related to chewing" and the "amount related to smile" generated (from the myoelectric signal) by the biological signal information determination unit 411 of the mobile terminal 4. The meal self-evaluation result by the user acquired from the terminal 4 and further input to the mobile terminal 4 is also acquired from the mobile terminal 4 to generate a meal log.

Here, a specific example of meal evaluation in this meal evaluation system will be described. First, the mobile terminal 4 transmits the meal image generated by the camera 405 to the meal evaluation server 3 at the start of the user's meal. The meal evaluation server 3 processes the received meal image, generates a "value related to the amount of meal" and a "value related to the taste of meal", and returns it to the mobile terminal 4.

Next, the feedback unit 412 of the mobile terminal 4 that has acquired these values generates or selects the user's "information related to diet or health" and provides the user with the user. As an example of this "meal or health-related information" provided, when it is estimated (from the meal image) that the dish / menu name is "curry", "it's my favorite curry. Curry rice. And stew tend not to increase the number of chews so much, so be sure to chew consciously. "

Also, if it is estimated that the name of the dish / menu is "ramen", it is possible to generate audio content that includes a warning content such as "Today is ramen. It is better to leave the soup." .. Furthermore, even if the food / menu name cannot be estimated, it is preferable to select or generate advice based on the estimated meal time from the estimated "value related to the amount of meal", and "there is a lot. Still 30 minutes. You may generate audio content such as "Please enjoy your meal slowly" and give feedback.

Incidentally, as still another embodiment, the mobile terminal 1 (FIG. 1) has an image information processing unit 111a (FIG. 1), and the image information processing unit 111a has the above-mentioned image information (of the meal evaluation server 3). It is also preferable to have the same function as the determination unit 311 and perform the same processing to determine the "amount related to chewing" and / or the "amount related to smile" of the user. That is, in this case, the mobile terminal 1 carries out processing equivalent to the meal evaluation system including the mobile terminal 4 and the meal evaluation server 3.

[Meal evaluation method]
FIG. 6 is a schematic diagram for schematically explaining another embodiment of the dietary evaluation method according to the present invention.

As shown in FIG. 6, in the meal evaluation system (portable terminal 4 and meal evaluation server 3) of the present embodiment, first, from the "myoelectric signal" acquired from the glasses 2 with a myoelectric sensor, "to chew". "Amount" and "amount related to smile" are generated. Next, a "meal log" is generated from these amounts and the "meal self-evaluation result" by the user, and further, a learning process is performed using the recorded contents of the generated "meal log" to perform a chewing-related amount. An estimation model and a smile-related quantity estimation model are constructed. Further, as in the embodiment shown in FIG. 4, a meal amount estimation model and a meal preference estimation model are also constructed from this “meal log”.

Further, in this meal evaluation system, the "meal / menu name" is estimated from the acquired "image information (meal image)" related to the user's meal by an image meal type estimation model using a known image recognition technology. Also, from the "image information", the "value related to the amount of meal" is estimated by an image meal amount estimation model using a known image recognition technique.

Next, from the estimated "cooking / menu name" and "value related to the amount of meal", the "amount related to mastication" is estimated using the learned mastication-related amount estimation model, and also from these data. The "amount related to mastication" is estimated using the trained mastication-related amount estimation model.

Furthermore, from the estimated "amount related to chewing" and "amount related to smile", "value related to the amount of meal" is estimated using the learned meal amount estimation model, and also learned from these amounts. The "value related to the dietary preference" is estimated using the completed dietary preference estimation model.

The meal evaluation system subsequently generates or selects "information related to meals or health" based on the estimated "values related to the amount of meals" and "values related to meal preferences" and provides them to the user. .. That is, in the end, this dietary evaluation system uses an estimation model constructed from the acquired "myoelectric signal" to determine "information related to diet or health" suitable for the user.

As described in detail above, according to the present invention, in providing the user's "information related to diet or health" based on the acquired biological signal, at least "value related to the amount of diet" is taken into consideration. This makes it possible to provide more appropriate information.

Therefore, for example, in the elderly care field or the medical field, it is possible to give appropriate advice to the meals of the elderly and patients, and to provide a situation / environment suitable for meals.

With respect to the various embodiments of the present invention described above, various changes, modifications and omissions within the scope of the technical idea and viewpoint of the present invention can be easily carried out by those skilled in the art. The above explanation is just an example and does not attempt to limit anything. The present invention is limited only to the scope of claims and their equivalents.

1 Mobile terminal (meal evaluation device)
101, 401 Signal interface 102, 302, 403 Chewing / smile information storage unit 103, 303 Meal log storage unit 104, 404 Feedback information storage unit 105, 405 Camera 106, 406 Touch panel display (TP / DP)
107 Mike 108, 407 Speaker 111, 411 Biosignal information determination unit 111a Image information processing unit 112, 312 Meal amount determination unit 113, 313 Meal preference determination unit 114, 412 Feedback unit 121, 321 Meal log generation management unit 122, 322 Meals Quantity estimation model construction unit 123, 323 Preference estimation model construction unit 131, 331, 421 Communication control unit 132, 422 Input / output control unit 2 Glasses with myoelectric sensor (biological signal acquisition device)
2'Headphones (biological signal processor)
2'' Earphone (biological signal processor)
21 Signal processing box 211 Signal conversion unit 212 Pre-filter processing unit 213 Signal interface 23 Positive electrode pad 24 Negative electrode pad 25 Nose pad Electrode unit 3 Meal evaluation server 301, 402 Communication interface 4 Mobile terminal 311 Image information determination unit

Claims (17)

A meal evaluation device that evaluates a user's meal.
An amount determined from a biological signal related to the user's head, which is a biological signal information determining means for determining an amount related to chewing and an amount related to a smile of the user.
A meal amount determination means for determining a value related to the amount of meal using a learned meal amount estimation model based on the amount related to chewing and the amount related to smile .
A dietary evaluation device comprising a feedback means for generating or selectively providing information related to the diet or health of the user based on a determined value relating to the amount of the diet. The learned meal amount estimation model using a plurality of meal logs recorded by associating the amount related to the chewing and the amount related to the smile with the information reported by the user at least for the value related to the amount of meal. The meal evaluation device according to claim 1, further comprising a means for constructing a meal amount estimation model. The meal evaluation device according to claim 2, wherein the feedback means generates or selects information related to the user's meal or health based on the meal log relating to at least one past time point. .. The feedback means is preliminarily based on at least one of the number of times, pace and time interval in mastication determined from the amount of the mastication, according to the condition that the at least one and the value related to the amount of the meal are satisfied. The meal evaluation device according to any one of claims 1 to 3 , wherein at least one of the set music or audio contents is selected and used as information related to the meal or health of the user. The biometric signal information determining means generates acceleration component data of an input signal that may include a biometric signal, calculates a representative value related to the degree of data bias in a predetermined time interval in the acceleration component data, and calculates a period of the representative value. The dietary evaluation device according to any one of claims 1 to 4 , wherein the amount of the chew is determined based on the sex. A meal evaluation device that evaluates a user's meal.
An amount determined from a biological signal related to the head of the user, and a biological signal information determining means for determining at least an amount related to a smile among the amount related to chewing and the amount related to a smile of the user.
A meal amount determining means for determining a value related to the amount of meal using a learned meal amount estimation model based on at least the amount related to smile among the amount related to chewing and the amount related to smile.
A feedback means that generates or selectively provides information related to the user's diet or health based on the determined dietary value.
Have,
The feedback means is preset according to the condition that the number and / or time of the smile and the value related to the amount of the meal are satisfied, based on the number and / or time of the smile determined from the amount of the smile. Select at least one of the music or audio content to be information related to the user's diet or health.
A meal evaluation device characterized by that . The biological signal information determining means is
Image information related to the user's meal is acquired, and based on the image information, the learned image meal type estimation model and / or image meal amount estimation model is used to provide meal type information and / or meal in the user's meal. Determine the amount related to the amount of
Based on the determined meal type information and / or the amount related to the amount of meal, the learned biological signal information estimation model constructed by learning based on the amount related to the past chewing and / or the amount related to the smile is used. The meal evaluation device according to any one of claims 1 to 6 , wherein the amount related to the user's chewing and / or the amount related to the smile is determined. A meal evaluation device that evaluates a user's meal.
An amount determined from a biological signal related to the head of the user, and a biological signal information determining means for determining at least an amount related to a smile among the amount related to chewing and the amount related to a smile of the user.
A meal amount determining means for determining a value related to the amount of meal using a learned meal amount estimation model based on at least the amount related to smile among the amount related to chewing and the amount related to smile.
A feedback means that generates or selectively provides information related to the user's diet or health based on the determined dietary value.
The biological signal information determining means has
Acceleration component data of an input signal that can include biological signals is generated, a representative value related to the degree of data bias in a predetermined time interval in the acceleration component data is calculated, and the living body is repeated based on the periodicity of the representative value. Determine the generation of periodic biological signals caused by exercise, and further
Multi-resolution analysis processing is performed on the input signal, and the time interval in which the time-series data of the signal amplitude after the multi-resolution analysis processing shows a predetermined hysteresis is determined as the signal generation time interval in which some biological signal is generated. When it is determined that the periodic biometric signal is not generated in the signal generation time section, the average power frequency of the input signal is calculated, and the amount related to the smile is calculated based on the height of the average power frequency. A dietary evaluation device characterized by determining. A meal evaluation device that evaluates a user's meal.
An amount determined from a biological signal related to the head of the user, and a biological signal information determining means for determining at least an amount related to a smile among the amount related to chewing and the amount related to a smile of the user.
A meal amount determining means for determining a value related to the amount of meal using a learned meal amount estimation model based on at least the amount related to smile among the amount related to chewing and the amount related to smile.
A feedback means that generates or selectively provides information related to the user's diet or health based on the determined dietary value.
The biological signal information determining means has
Acceleration component data of an input signal that can include biological signals is generated, a representative value related to the degree of data bias in a predetermined time interval in the acceleration component data is calculated, and the living body is repeated based on the periodicity of the representative value. Determine the generation of periodic biological signals caused by exercise, and further
Multiple resolution analysis processing is performed on the input signal, and the time interval in which the time-series data of the signal amplitude after the multiple resolution analysis processing shows a predetermined hysteresis is determined as the signal generation time interval in which some biological signal is generated. When it is determined that the periodic biometric signal is not generated in the signal generation time section, the feature amount including the standard deviation and the average power frequency of the input signal is calculated, and the feature amount corresponds to the reference state. The food is characterized in that the degree of separation, which is the degree of separation from the unit space set by the characteristic amount of the input signal to be input, is calculated, and the amount related to the smile is determined based on the calculated degree of separation. Thing evaluation device. The meal evaluation device according to any one of claims 1 to 9, wherein the meal amount determining means determines a value related to a certain amount of meal based on a meal time. The meal evaluation device further has a meal preference determination means for determining a value related to a meal preference by using a learned preference estimation model based on the amount related to the chewing and / or the amount related to the smile.
13 . The described dietary evaluation device. A preference estimation model is constructed by using a plurality of meal logs recorded by associating at least the value declared by the user with respect to the amount related to chewing and / or the amount related to smile. The meal evaluation device according to claim 11 , further comprising a means for constructing a preference estimation model. The feedback means is characterized in that it generates or selects information related to the diet or health of the user based on the acquired location information relating to the user and / or voice information relating to the meal. The dietary evaluation device according to any one of 12 to 12 . The biometric signal is a device attached to the user's head, with the reference electrode in contact with one skin position near the cheek from around the left (or right) pinna, and the detection electrode on the right (or left). ), The dietary evaluation device according to any one of claims 1 to 13, wherein the signal is acquired by a device having an electrode configuration such that it is in contact with one skin position near the cheek from the periphery of the auricle. .. A meal evaluation system having a terminal related to a user and a meal evaluation server for evaluating the user's meal, which is connected to the terminal by communication.
The terminal has a biological signal information determining means that is an amount determined from a biological signal related to the head of the user, and determines an amount related to chewing and an amount related to a smile of the user.
The meal evaluation server is
Image information related to the user's meal is acquired from the terminal, and based on the image information, the trained image meal type estimation model and / or the image meal amount estimation model is used to obtain the meal type information and / or meal type information in the user's meal. / Or determine the amount of food, and based on the determined meal type information and / or the amount of food, the amount of past chewing and the amount of smile obtained from the terminal. Using the trained biometric signal information estimation model constructed by learning, an image information determining means for determining the amount of chewing and the amount of smiling face of the user, and
It has a meal amount determining means for determining a value related to the amount of meal using a learned meal amount estimation model based on the amount related to chewing and the amount related to smile .
The terminal acquires a value related to the amount of the meal from the meal evaluation server, and generates or selects and provides information related to the user's meal or health based on the value related to the amount of the meal. A dietary evaluation system characterized by having more means. A program that activates a computer installed in a device that evaluates a user's diet.
An amount determined from a biological signal related to the user's head, which is a biological signal information determining means for determining an amount related to chewing and an amount related to a smile of the user.
A meal amount determination means for determining a value related to the amount of meal using a learned meal amount estimation model based on the amount related to chewing and the amount related to smile .
A dietary evaluation program characterized in that a computer functions as a feedback means for generating or selecting and providing information related to the user's diet or health based on the determined value of the amount of diet. It is a meal evaluation method in a computer installed in a device that evaluates a user's meal.
An amount determined from a biological signal related to the user's head, and a step of determining an amount related to chewing and an amount related to smile of the user.
Based on the amount related to the chewing and the amount related to the smile , the step of determining the value related to the amount of the meal using the learned meal amount estimation model, and
A dietary evaluation method comprising: a step of generating or selecting and providing information related to the diet or health of the user based on a determined value relating to the amount of the diet.

Images