JP2023094218A - Information processing device, mastication training system, mastication evaluation method, mastication training program, and recording medium - Google Patents

Abstract

To provide a technique that can enhance an effect of mastication training.SOLUTION: An information processing device (1) comprises: an evaluation unit (102) which, on the basis of images obtained by imaging the appearance of the mastication of an object person in time series, evaluates the magnitudes of the vertical and horizontal motions of the mouth while the object person is masticating; and an output control unit (105) which causes an output device (16) to output feedback information according to a result of the evaluation.SELECTED DRAWING: Figure 1

Description

The present invention relates to an information processing apparatus and the like that can be used for mastication training.

Conventionally, attempts have been made to obtain useful knowledge by analyzing the behavior of people eating. For example, Patent Document 1 below discloses a technique for quantitatively estimating the texture of a sample by measuring myoelectric potentials when a subject chews a sample and analyzing the measured myoelectric potentials. .

JP 2016-52516 A

The masticatory power may decline due to various factors such as aging and malfunction of sites related to mastication. In addition, there are many people who cannot masticate properly because there is no opportunity to be taught how to masticate properly. From such a point of view, mastication training is desired that enables mastication in a correct manner and training of masticatory power. It is desirable to have a technique that maximizes the effect of mastication training.

However, there has been no technology that enhances the effects of mastication training. For example, according to the technique of Patent Document 1, since the texture of a sample can be quantified, the technique can be used when selecting foods to be used in mastication training. Alternatively, it is not possible to evaluate the mastication training effect and make the user recognize it.

One aspect of the present invention has been made in view of the above-described problems, and an example of the purpose thereof is to enhance the training effect or evaluate the training effect as compared with mastication training in which only mastication is performed. It is to provide a technique that can be recognized by the user by doing so.

In order to solve the above-described problems, an information processing apparatus according to an aspect of the present invention provides a vertical direction and a horizontal direction of the mouth of a subject during mastication based on images of the subject's mastication taken in time series. An evaluation unit that evaluates the magnitude of directional movement, and an output control unit that causes an output device to output feedback information according to the result of the evaluation.

In order to solve the above problems, a mastication training system according to an aspect of the present invention includes a photographing device for photographing a state of mastication of a subject in time series, and based on the time-series images photographed by the photographing device, An information processing device for evaluating vertical and horizontal movements of the mouth of the subject during mastication, and an output device for outputting feedback information according to the evaluation result.

In order to solve the above problems, a mastication evaluation method according to an aspect of the present invention is a mastication evaluation method executed by one or more information processing apparatuses, in which a state of mastication of a subject is photographed in chronological order. an evaluation step of evaluating the magnitude of vertical and horizontal movements of the mouth during mastication of the subject based on the image; an output control step of causing an output device to output feedback information according to the evaluation result; including.

The information processing device and system according to each aspect of the present invention may be realized by a computer. A program for implementing the system on a computer and a computer-readable recording medium recording the program are also included in the scope of the present invention.

According to one aspect of the present invention, the effect of mastication training can be enhanced.

1 is a block diagram showing an example of a main configuration of an information processing apparatus according to Embodiment 1 of the present invention; FIG. It is a figure which shows the outline|summary of the said information processing apparatus. It is a figure explaining an example of the process which the evaluation part of the said information processing apparatus performs. FIG. 10 is a diagram showing an example of presentation of feedback information; FIG. 10 is a diagram showing another presentation example of feedback information; It is a flow chart which shows a flow of processing which the above-mentioned information processor performs. FIG. 10 is a diagram showing a configuration example of a mastication training system according to Embodiment 2 of the present invention; Fig. 2 is a block diagram showing an example of the main configuration of each device included in the mastication training system; FIG. 10 is a diagram showing an example of presentation of feedback information; FIG. 10 is a diagram showing another presentation example of feedback information;

[Embodiment 1]
(overview)
An outline of an information processing apparatus 1 according to an embodiment of the present invention will be described with reference to FIG. FIG. 2 is a diagram showing an overview of the information processing device 1. As shown in FIG. The information processing device 1 is a device capable of enhancing the effects of mastication training for a subject. Although FIG. 2 shows an example in which the information processing device 1 is a smart phone, any information processing device, that is, a computer can be made to function as the information processing device 1 .

Here, mastication training refers to behavior for assisting at least one of maintaining masticatory ability, improving masticatory ability, and mitigating the rate or degree of decline in masticatory ability. Masticatory ability refers to the ability to eat, chew, mix, and swallow specific foods (foods for evaluating mastication vary depending on age and body condition). When the ability of any one of these four movements is reduced, it interferes with food intake. Therefore, the masticatory ability is an important ability for humans to ingest food, and it is of great technical significance to enhance the effect of masticatory training or to evaluate and recognize the training effect.

A subject for mastication training holds an arbitrary mastication object in his mouth and masticates it. The object to be masticated may be food for mastication training (eg gummy, jelly, gum, etc.) or general food. Then, the subject takes an image of his/her mastication state with the information processing device 1 . Of course, you may ask someone else to take the picture.

The information processing device 1 evaluates the vertical and horizontal movements of the mouth of the subject during mastication based on time-series images obtained by photography, and provides feedback information according to the evaluation results. Output. This makes it possible for the subject to be conscious of moving the mouth not only in the vertical direction but also in the horizontal direction during mastication, and to learn the correct way to move the mouth. In this way, according to the information processing apparatus 1, it is possible to perform mastication training that is more effective than mastication training in which only mastication is performed.

In order to perform the above evaluation, the information processing apparatus 1 detects the magnitude of movement of the mouth of the subject during mastication based on time-series images obtained by photography. The information processing device 1 may detect the vertical and horizontal sizes of the mouth as distances, or may detect the size of the movement of the mouth as an area. Specifically, the distance may be the vertical and horizontal movement distance of the mouth or its surrounding feature points. The area may be the area of the region surrounded by the trajectory drawn by the mouth or its surrounding feature points.

It is also conceivable to use an image captured by an imaging device 3 such as a three-dimensional camera capable of capturing movement of the mouth in the depth direction. In this case, the information processing apparatus 1 may detect the size of the mouth in the vertical, horizontal, and depth directions as distances, or may detect the size of the movement of the mouth as the volume. Specifically, the volume may be the volume of the space surrounded by the trajectory drawn by the mouth or its surrounding feature points.

For example, the display unit 16 of the information processing apparatus 1 shown in FIG. 2 displays an image 161 of the subject who is chewing and advice to the subject as feedback information (hereinafter referred to as FB (Feed Back) information). message 162 is displayed.

By displaying the image 161, the subject can control the movement around the mouth while watching the movement around the mouth during mastication. In experiments by the inventors of the present application, it was found that such visual feedback increases muscle activity even with soft foods and increases movement around the mouth during mastication. Therefore, including the image 161 in the FB information is effective in enhancing the effect of mastication training.

Also, the message 162 prompts the subject to chew by imagining drawing a large circle with the tip of the chin. Such a message is displayed when the result of the evaluation of the degree of mouth movement during chewing of the subject indicates that the mouth movement is small, especially in the lateral direction. The message 162 is effective in increasing the effect of mastication training because it helps the subject to have an ideal mastication mode.

Note that the FB information shown in FIG. 2 is merely an example, and any content can be used as long as it can enhance the effect and efficiency of mastication training. Moreover, the feedback to the subject is not limited to visual feedback, and feedback may be performed by sound, vibration, or the like, or by a combination thereof. Moreover, feedback may be performed via another device such as displaying the FB information on a display device external to the information processing device 1 .

(Configuration of information processing device 1)
A more detailed configuration of the information processing apparatus 1 will be described with reference to FIG. FIG. 1 is a diagram showing an example of the main configuration of an information processing apparatus 1. As shown in FIG. As shown in the figure, the information processing apparatus 1 includes a control section 10 that centrally controls each section of the information processing apparatus 1 and a storage section 11 that stores various data used by the information processing apparatus 1 . The information processing device 1 also includes a photographing unit 12 which is a photographing device for photographing an image, a communication unit 13 for communication between the information processing device 1 and other devices, and a sound output unit 14 which is an output device for outputting sound. I have it. Further, the information processing device 1 includes an input unit 15 that receives input of various data to the information processing device 1, and a display unit 16 that is a display device that displays and outputs various information. The functions of the input unit 15 and the display unit 16 can also be realized by a touch panel.

The control unit 10 also includes an image acquisition unit 101 , an evaluation unit 102 , a notification unit 103 , a model generation unit 104 and an output control unit 105 . The evaluation unit 102 also includes a feature point detection unit 1021 , a motion analysis unit 1022 and a comparison unit 1023 . These processing units can also be realized by installing, in the information processing apparatus 1, a mastication training program for causing the information processing apparatus 1 to function as these processing units, for example. The storage unit 11 also stores an image 111 and an evaluation result 112 . As will be described later, the image 111 and the evaluation result 112 are used as one piece of FB information by users of the information processing apparatus 1, such as the subject himself/herself, the subject's supervisor, and medical staff who care for the subject. (below, simply referred to as “subjects, etc.”).

The image acquisition unit 101 acquires images of the subject's mastication in time series. In this embodiment, an example in which the image acquisition unit 101 acquires an image captured by the imaging unit 12 will be described. may be obtained by Alternatively, the image acquisition unit 101 may acquire an image stored in a detachable external memory or the like in the information processing apparatus 1 via a terminal (not shown). Further, the image acquisition unit 101 stores the acquired image as the image 111 in the storage unit 11 . The image 111 may be a moving image, or may be a time-series still image captured continuously. Further, the image acquiring unit 101 may acquire a moving image and store frame images extracted from the moving image as the image 111 .

The evaluation unit 102 evaluates the vertical and horizontal movements of the mouth during mastication of the subject based on the images 111 obtained by photographing the state of mastication of the subject in time series. The function of the evaluation unit 102 is implemented by a feature point detection unit 1021 , a motion analysis unit 1022 and a comparison unit 1023 .

The feature point detection unit 1021 detects a predetermined feature point on the subject's face from the image 111 and stores the position information as the evaluation result 112 in the storage unit 11 . Further, the motion analysis unit 1022 analyzes the motion of the mouth during mastication using the time-series position information, and calculates an index value indicating the motion. The comparison unit 1023 then compares the index value calculated by the motion analysis unit 1022 with a predetermined threshold value and records the result in the evaluation result 112 . Details of these processes will be described later with reference to FIG.

The notification unit 103 notifies the subject of mastication training of the timing to masticate, in other words, the timing to chew the object to be masticated. Specifically, the notification unit 103 causes the sound output unit 14 to output a sound indicating the timing at which mastication should be performed. For example, the notification unit 103 may output a rhythm sound indicating the timing of mastication. In addition, the notification unit 103 may output a sound indicating the timing of mastication in accordance with music (melody).

In addition, the notification unit 103 may transmit the attributes of the object to be masticated (for example, information related to mastication such as type, hardness, amount, and size) and the attributes of the subject (for example, age, sex, current number of teeth, etc.). information related to mastication). In this way, by notifying the subject of the timing of mastication according to at least one of the subject and the object to be masticated by the subject, the appropriate timing reflecting at least one of the subject and the object to be masticated can be obtained. The subject can be urged to masticate at appropriate timing. The attributes of the object to be masticated and the subject may be input by the subject or the like, or may be specified by the reporting unit 103 by analyzing the images of the object to be masticated and the subject.

The model generating unit 104 generates model information that serves as a model of mastication of the subject. The model information is information for improving the mastication mode of the subject, and is, for example, an image showing an ideal mastication mode. The model information may be a still image, but is preferably a moving image. For example, the model information may be an animation in which points indicating positions of feature points around the mouth move on a two-dimensional plane. The model generation unit 104 generates model information according to the attributes of the object to be masticated and the person to be masticated as described above, and causes the display unit 16 to display the model information via the output control unit 105 .

Instead of providing the model generation unit 104, a plurality of patterns of model information corresponding to the mastication object or the subject's attributes may be prepared in advance and stored in the storage unit 11 or the like. In this case, the output control unit 105 reads the model information corresponding to the object to be masticated or the attribute of the subject from the storage unit 11 or the like and displays it. As a result, the model information is presented to the subject etc. as one of the FB information.

The output control unit 105 causes the output device to output FB information according to the evaluation result by the comparison unit 1023 . The content of the FB information is arbitrary as long as it can improve the effect and efficiency of mastication training. Details of the FB information will be described later.

An output device that outputs FB information may be, for example, the display unit 16, the sound output unit 14, or both of them, and may be wired or wirelessly connected to the information processing device 1. It may also be an external output device. In addition to this, for example, an output device that outputs FB information by print output can also be applied. An example in which the output control unit 105 performs FB (feedback) by causing the display unit 16 to display an image will be described below. and "display" can be read as "output".

As described above, the information processing apparatus 1 evaluates the magnitude of vertical and horizontal movements of the mouth during mastication of the subject based on the images 111 of the subject's mastication taken in time series. 102, and an output control unit 105 that causes the display unit 16 to display FB information according to the evaluation result. Therefore, it is possible to perform more effective mastication training than mastication training in which only mastication is performed.

(Detection of feature points)
A method of detecting feature points by the feature point detection unit 1021 will be described with reference to FIG. FIG. 3 is a diagram showing an example of feature points to be detected and an example of evaluation of mouth movements based on the detection result of the feature points.

A schematic diagram 31 of a face in FIG. 3 shows an example of feature points to be detected. Since the feature points detected by the feature point detection unit 1021 are used for analyzing the movement of the mouth during mastication, it may be a part that moves in a way that reflects the mode of mastication during mastication. For example, the feature point detection unit 1021 detects points such as points 314 (upper lip apex), 315 (lower lip apex), and 316 (chin center position) in the schematic diagram 31, or the tip of the chin (not shown). A region around the mouth on the center line L of the face may be detected as a feature point. By detecting portions on the center line L as feature points, it is easy to specify the magnitude of lateral movement of the mouth during mastication.

Further, the feature point detection unit 1021 may detect points 317 and 318 (mouth corners) in the schematic diagram 31 as feature points. In this case, the movement analysis unit 1022 may analyze the movement of the mouth by regarding the movement of the midpoint between the point 317 (right corner of the mouth) and the point 318 (left corner of the mouth) as the movement of the mouth. In addition, for example, the feature point detection unit 1021 may detect a region of the cheek near the masseter muscle as a feature point. do it.

Further, the feature point detection unit 1021 may detect a motionless part caused by mastication. For example, the feature point detection unit 1021 may detect the point 313 (muzzle). In this case, the motion analysis unit 1022 can represent the position of the feature point appearing in the image 111 with the position of the point 313 (muzzle) as a reference. This makes it possible to accurately identify the time-series movement of each feature point even when the position of the subject is shifted between the time-series images 111 .

Furthermore, the feature point detection unit 1021 may detect a plurality of parts that do not move due to mastication in order to obtain a distance on the image 111 that serves as a reference for evaluating the magnitude of mouth movement. For example, the feature point detection unit 1021 may detect points 311 and 312 (inner corners of the eye). Accordingly, the comparison unit 1023 can evaluate the magnitude of the movement of the mouth based on the distance between the point 311 (right inner corner of the eye) and the point 312 (left inner corner of the eye). The feature point detection unit 1021 may use the points 311, 312, 313, 317, 318, etc. to obtain the center line L of the face.

In this way, the evaluation unit 102 calculates the longitudinal direction of the mouth during mastication of the subject based on the distance between predetermined feature points on the subject's face whose positions do not change due to mastication, which are detected from the image 111 . and the magnitude of lateral movement may be evaluated. As a result, the magnitude of the movement of the mouth can be accurately evaluated regardless of whether the subject appears large or small in the image 111 .

Note that the subject or the like may input a distance that serves as a reference for evaluating the magnitude of the movement of the mouth. In addition, when the distance to the subject is known (for example, when shooting with a 3D camera that outputs distance information), the magnitude of mouth movement can be evaluated according to the distance, so it is possible to calculate the reference distance. No site detection is required.

A method for detecting feature points as described above is not particularly limited, and for example, an image analysis technique, a machine learning technique, or a combination thereof may be applied. Further, when detecting a plurality of feature points, a feature point detection unit 1021 may be provided for each feature point to be detected. Alternatively, a marker for detection, a two-dimensional code, or a three-dimensional code may be pasted on the detection target portion of the subject's face to detect them from the image.

(analysis of movement)
The movement analysis unit 1022 analyzes the movement of the mouth during mastication from the time-series positional changes of the feature points detected as described above, and calculates an index value indicating the movement. This will be explained based on the graph 32 shown in FIG.

A graph 32 is a graph showing changes in position of feature points on the image 111 over time. The graph 32 can be generated from the positional information of the feature points appearing in the images 111 detected from each of the plurality of time-series images 111 .

For example, the apex of the lower lip (point 315 in schematic diagram 31) and the position of the chin (point 316 in schematic diagram 31) change as shown in graph 32 during mastication. In the graph 32, an initial position 321 indicates the position of the feature point at the start of mastication (immediately before opening the mouth). As shown in graph 32, after the start of mastication, the feature point moves to the lower left, then to the upper right, and then to the left. It should be noted that the left and right described here are the left and right when viewed from the side of photographing (observing) the subject, and that the left and right are reversed when viewed from the subject.

In this way, the graph 32 has the feature that the feature point moves from the initial position 321 at the start of mastication and returns to the vicinity of the initial position 321 again. Therefore, the motion analysis unit 1022 can use this feature to identify one chewing motion from the graph 32 (more precisely, the position information of feature points detected from each of the plurality of time-series images 111). . In addition, this makes it possible to calculate an index value indicating the number of times of mastication from when the object to be masticated is started to when the object is swallowed.

Further, from the graph 32, it is possible to specify the position when the feature point moves to the leftmost and the position when it moves to the rightmost in one chewing motion. Therefore, the motion analysis unit 1022 can also identify that the magnitude of lateral motion of the feature point is Mox from the position information of the feature point detected from each of the plurality of time-series images 111 . Similarly, the motion analysis unit 1022 determines the position (for example, initial position 321 at the start of mastication) and It is also possible to specify the position when the feature point moves to the lowest position and specify that the magnitude of vertical movement of the feature point is Moy.

Specifically, the motion analysis unit 1022 extracts position information included in one chewing motion from position information of feature points detected from each of the plurality of time-series images 111 . It is also possible to detect one chewing motion based on the movement of the corners of the mouth, the thickness of the lips, the displacement of the area of the cheek near the masseter muscle, and the like. Then, motion analysis section 1022 calculates the distance Mox between the leftmost position and the rightmost position among the pieces of positional information as an index value indicating the size of the mouth in one chewing motion. At the same time, the distance Moy between the position information indicating the highest position and the position information indicating the lowest position is calculated.

At this time, the motion analysis unit 1022 detects the vertical and horizontal directions of the subject's mouth based on the distance between predetermined feature points whose positions do not change due to mastication on the subject's face detected from the image 111 . An index value indicating the magnitude of the movement of the may be calculated. For example, the motion analysis unit 1022 calculates the distance Ex between the point 311 (right inner corner of the eye) and the point 312 (left inner corner of the eye) in the schematic diagram 31, and uses it as an index value indicating the magnitude of lateral movement of the mouth. A value of Mox/Ex may be calculated, and a value of Moy/Ex may be calculated as an index value indicating the magnitude of vertical movement of the mouth. This makes it possible to make the values of Mox and Moy dimensionless and cancel the difference in the size of the subject in the image 111 .

Note that the index value indicating the movement of the mouth during mastication is not limited to the above example. For example, it can be said that the area of the portion surrounded by the graph 32 (hereinafter referred to as Mos) indicates the magnitude of mouth movement. value. Mos may also be made dimensionless by dividing by the distance (eg, the above-mentioned Ex) between predetermined feature points whose positions do not change due to mastication.

Further, the movement analysis unit 1022 may calculate the length of the diagonal distance obtained from Mox and Moy by the Pythagorean theorem as an index value representing the magnitude of vertical and horizontal movement of the mouth. . Specifically, with reference to the graph 32 shown in FIG. 3, the motion analysis unit 1022 calculates the length of the diagonal of a rectangle (indicated by a dashed line) whose horizontal length is Mox and whose vertical length is Moy. may be calculated as an index value representing the magnitude of vertical and horizontal movement of the mouth.

The motion analysis unit 1022 may also calculate an index value other than the index value indicating the magnitude of the mouth movement. For example, the motion analysis unit 1022 may use the number of times of mastication per unit time, the number of times of mastication from holding an object to be masticated in the mouth until swallowing, the rhythm of mastication, and the like as index values. In addition to this, the motion analysis unit 1022 may, for example, determine which side is chewed on the left or right side based on the movement of the corners of the mouth, and calculate the number of times of chewing on the left or right side or the ratio thereof as an index value. Also, for example, the motion analysis unit 1022 may determine whether or not the motion of the feature point is within the normal range, and output an index value indicating the determination result.

In obtaining the index value, the motion analysis unit 1022 may use the values of Mox and Moy in one chewing motion, or may use the average values of Mox and Moy in an arbitrary number of chewing motions. Alternatively, the total values of Mox and Moy in an arbitrary number of chewing motions may be used.

(comparison)
The comparison unit 1023 compares the index value calculated by the motion analysis unit 1022 as described above with a predetermined threshold value. The result of this comparison is the result of evaluating the magnitude of vertical and lateral movements of the mouth during mastication of the subject. The predetermined threshold value is the longitudinal and lateral movements of the mouth of a comparison subject (e.g., ideal mastication for a subject or a group with the same attributes as the subject, or mastication performed by the subject in the past). It should be set based on the size.

For example, when the motion analysis unit 1022 calculates Mox and Moy as index values indicating the magnitude of vertical and horizontal motion of the mouth, the comparison unit 1023 calculates Mox and Moy calculated by the motion analysis unit 1022 and , with set Mox and Moy thresholds, respectively, to assess the magnitude of longitudinal and lateral movement of the subject's mouth. Mox and Moy, which are set as thresholds, indicate the range of motion required for the subject in the vertical and horizontal directions, and are hereinafter referred to as the required range of motion. For example, when the comparison result of the comparison unit 1023 indicates that the Mox calculated by the movement analysis unit 1022 is less than the required range of motion in the lateral direction, it is determined that the mouth movement in the lateral direction was insufficient for mastication. I can say. Similarly, when the comparison result of the comparison unit 1023 indicates that Moy calculated by the movement analysis unit 1022 is less than the required range of motion in the vertical direction, the mouth movement in the vertical direction is insufficient for mastication. It can be said that

As described above, Mox and Moy used for such comparison are dimensionless based on the distance between predetermined feature points whose positions do not change due to mastication on the subject's face. good. That is, the comparison unit 1023 may compare and evaluate the Mox/Ex and Moy/Ex described above with the required range of motion. In this case, the required range of motion is also made dimensionless in the same manner.

In addition, the evaluation unit 103
Moy/Mox*Ex≦required range of motion Equation 1
or,
Mos/Ex≦required range of motion Equation 2
is satisfied, the mouth movement may be evaluated as insufficient.

In addition, the comparison unit 1023 may divide one chewing action into a plurality of stages, evaluate each stage, and obtain a final evaluation result by combining the evaluation results. This will be explained based on the graph 33 in FIG. A graph 33 in FIG. 3 shows an example in which one chewing motion is divided into 10 stages from level 0 (start/end of chewing motion) to level 10 (maximum mouth opening) and each stage is evaluated. The left and right shown in the graph 33 correspond to the left and right seen from the subject.

In this case, the motion analysis unit 1022 calculates an index value indicating the magnitude of lateral motion of the mouth at each stage. Then, the comparison unit 1023 compares these index values with the required range of motion in the horizontal direction at each stage, and evaluates the horizontal movement of the mouth of the subject based on the comparison results. For example, the comparison unit 1023 calculates the ratio or difference between each index value and each required range of motion, and compares the total value or average value of the calculation results with a predetermined threshold to determine the width of the subject's mouth. Directional movement may be evaluated.

Note that the evaluation unit 103 may evaluate the movement of the mouth based on the degree of similarity of the figures drawn by the trajectory of one mastication, instead of the evaluation divided into 10 stages. The degree of similarity is a value indicating the degree to which the shapes of figures are similar, and the method of calculating the degree of similarity is not particularly limited.

Moreover, the evaluation unit 103 may evaluate the movement of the mouth taking into account the movement in the vertical direction in addition to the movement in the horizontal direction. In addition, the index value may be normalized by the maximum value of Moy, or may be dimensionless by Ex or the like.

Moreover, when the motion analysis unit 1022 calculates index values other than the index value indicating the magnitude of the mouth movement, the comparison unit 1023 may also compare these index values. For example, the comparison unit 1023 compares the evaluation values such as the number of times of mastication per unit time, the number of times of mastication from holding one mastication object in the mouth to swallowing, the rhythm of mastication, and the like in ideal mastication. can be compared with

In addition, the comparison unit 1023 may comprehensively evaluate the subject's mastication by integrating various indices calculated by the motion analysis unit 1022 . For example, the comparison unit 1023 may calculate mastication efficiency as information indicating the subject's evaluation result. In this case, the motion analysis unit 1022 calculates various index values that are correlated with the masticatory efficiency, and uses a masticatory efficiency prediction model that models the relationship between the index values and the masticatory efficiency. The masticatory efficiency can be calculated from the values. In this case, comparison section 1023 may compare the calculated mastication efficiency with the ideal mastication efficiency. Also, the ideal masticatory efficiency may be calculated based on the subject's attribute information such as the number of teeth of the subject.

Note that the evaluation method is not limited to the above example, as long as at least evaluation results can be obtained for the magnitude of vertical and horizontal movements of the mouth of the subject during mastication. For example, the evaluation unit 102 may calculate the degree of similarity between the mastication mode (for example, ideal mastication mode) that serves as a reference for evaluation and the mastication mode of the subject. Specifically, the evaluation unit 102 may calculate the degree of similarity between a moving image showing a mastication mode, which serves as an evaluation criterion, and a moving image of the subject's mastication state. The degree of similarity of the extracted time-series still images may be calculated. Further, for example, the evaluation unit 102 may calculate the degree of similarity of patterns of time-series changes in the positions of feature points extracted from those images.

In addition, the evaluation unit 102 may use an evaluation model constructed by machine learning to evaluate the magnitude of vertical and horizontal movements of the mouth of the subject during mastication. This evaluation model uses the results of evaluation of the vertical and horizontal movements of the mouth during mastication as objective variables, and images taken chronologically of the state of mastication or feature values extracted from the images as explanatory variables. , can be constructed by learning the relationship between the objective variable and the explanatory variable. According to this configuration, highly accurate evaluation results can be generated.

As the feature amount, for example, time-series position information of the feature points detected by the feature point detection unit 1021 can be used. When the image 111 is input to the evaluation model and evaluated, the feature point detection unit 1021 to the comparison unit 1023 are unnecessary.

Note that the evaluation unit 102 may evaluate the magnitude of vertical and horizontal movements of the mouth during mastication of the subject. Calculation of an index value that indicates is also included in the scope of evaluation. The FB information in this case may indicate the calculated index value. Also, in this case, the comparison unit 1023 is unnecessary.

The evaluation unit 102 may evaluate the masticatory motion of the subject by comparing the mastication mode actually observed in the subject with model data indicating the mastication mode as a model prepared in advance. . Alternatively, the model generation unit 104 uses the evaluation results 112 of the subject accumulated in the process of using the information processing apparatus 1 for evaluation, and generates a target indicating an ideal mastication mode that the subject should aim for. data may be generated. The evaluation unit 102 may evaluate the chewing action of the subject by comparing the chewing behavior actually observed in the subject with the generated target data.

The exemplar data and target data can be configured in any data structure, depending on what information is employed as the object of comparison. For example, when the index values Mox and Moy are used for comparison, the model data and target data are also generated to include the index values Mox and Moy. In another example, the evaluation unit 102 may evaluate the chewing motion of the subject by comparing a chewing cycle representing the motion path of the mandible during mastication of the subject with an ideal chewing cycle. good. In this case, the exemplar data and target data are generated to contain information indicative of an ideal chewing cycle.

(About feedback information)
The output control unit 105 displays FB information according to the above-described evaluation result by the comparison unit 1023 . The FB information may be anything as long as it corresponds to the presentation mode, and may be, for example, text information, voice information, or image information (still image or moving image). good. For example, the FB information may be animation, sound effects, and the like.

Also, the content of the FB information may be anything as long as it corresponds to the evaluation result of the evaluation unit 102 . For example, the FB information may present an index value indicating the magnitude of longitudinal and lateral movements of the mouth during mastication of the subject. Further, for example, the FB information may be information that calls attention to the subject, or information that indicates that the subject's mastication mode is good. It may be the information shown. In addition, the FB information may indicate, for example, whether or not mastication training for increasing mastication efficiency has been completed, or may indicate a mastication mode for increasing mastication efficiency or an improvement target for mastication mode. There may be.

In addition, as described above, the evaluation unit 102 compares the vertical and horizontal movements of the mouth during mastication of the subject with the vertical and horizontal movements of the mouth during mastication of the comparison target. You may evaluate based on a difference. In this case, the FB information may indicate measures for reducing the difference. This enables the subject to efficiently improve his or her masticatory behavior.

As a specific example, when the comparison unit 1023 determines that Mox (or Mox/Ex) is less than the required range of motion, the output control unit 105 outputs FB information indicating measures for bringing Mox closer to the required range of motion. may be displayed. For example, the output control unit 105 may display, as the FB information, a message saying, "Imagine drawing a large circle with the tip of your chin." In general, mastication while imagining drawing a large circle with the tip of the chin increases the movement of the mouth, so presenting such FB information to the subject can contribute to improving the subject's mastication mode. can.

(Example 1 of presentation of FB information)
FIG. 4 is a diagram showing a presentation example of FB information. The display screen 4 shown in FIG. 4 displays an image 41, an image 42, messages 43 to 45, and an image 46 showing the evaluation result of mastication of the subject. Also, an arrow indicating the movement of the tip of the chin is displayed superimposed on the image 42 . This arrow, messages 43 to 45, and image 46 are all FB information. Also, it can be said that the entire display screen including the FB information, such as the display screen 4, is the FB information. The output control unit 105 may display the FB information during the mastication training, or may display the FB information after the mastication training.

An image 41 is an image showing an ideal movement of the mouth during mastication. Thus, the output control unit 105 may display the image 41 together with the FB information. As a result, the subject can be made to recognize the movement of the mouth during ideal mastication, so that the subject can be trained to approximate such movement. The image 41 may be a time-series still image, but is preferably a moving image.

Also, the image 41 showing the movement of the mouth during ideal mastication may be an image (animation) of the mouth and its surroundings as shown in FIG. A graph such as 32) or a figure may be used.

Also, the image 41 is preferably an image corresponding to at least one of the subject and the object to be masticated by the subject. Although the ideal mode of mastication may vary depending on the subject or the object to be masticated, according to the above configuration, it is possible to present an appropriate model image taking such differences into account. A plurality of patterns of the images 41 are prepared according to the attributes of the subject and the object to be masticated, and the output control unit 105 selects and displays an image that matches the attributes of the subject and the object to be masticated. Just do it. Alternatively, the model generation unit 104 may generate the image 41 as model information suitable for the attributes of the subject and the object to be masticated.

The image 42 is an image showing the movement of the mouth of the subject during mastication. Thus, the output control unit 105 may display the image 42 together with the FB information. As a result, the subject can be made aware of his/her own mastication mode, and thus an improvement in the training effect can be expected. In particular, by displaying the image 42 together with the image 41 as in the display screen 4, the difference between the ideal mouth movement and the target person's mouth movement can be recognized, so a higher training effect can be expected. can. Like the image 41, the image 42 is preferably a moving image. A moving image from which the image 111 is based may be displayed as the image 42 .

The message 43 indicates measures for reducing the difference between the vertical and horizontal movements of the mouth during mastication of the subject and the vertical and horizontal movements of the mouth during mastication of the comparison subject. It is an example of FB information shown.

A message 44 is an example of FB information displayed when the number of times of chewing until the object to be chewed held in the mouth is swallowed is less than the threshold. For example, the motion analysis unit 1022 calculates the number of times of chewing until the object to be chewed held in the mouth is swallowed, and the comparison unit 1023 determines whether the calculated number of times of chewing is less than the threshold. The output control unit 105 should display such a message 44 when it is determined that it is. It is preferable that the threshold value is displayed on the display unit 16 before the mastication training is started so that the subject recognizes the threshold value.

A message 45 is an example of FB information displayed when the subject's mastication is biased toward left or right. For example, the motion analysis unit 1022 calculates the number of plots located on the left side of 321 and the number of plots located on the right side of 321 among the plots constituting the graph 32 shown in FIG. It may be determined whether or not the calculated difference or ratio of the numbers is equal to or greater than the threshold, and the output control unit 105 may display such a message 45 when it is determined to be equal to or greater than the threshold.

An image 46 represents the evaluation result of the comparison unit 1023 according to the type of animal. Image 46 shown in FIG. 4 is an image of a rabbit. Since rabbits eat by moving their mouths in vertical directions, they are suitable as symbols for chewing modes in which the mouth of the subject does not move much. For example, when the comparison unit 1023 evaluates that the movement of the subject's mouth is less than the required range of motion, the output control unit 105 may display such an image 46 . This makes it possible to present the evaluation results impressively and in a way that even young people can intuitively understand them.

The image 46 may be a still image, but is preferably a moving image, and more preferably an animation image in which the subject performs a mastication motion in time with the subject's mastication timing. In addition, it is preferable to let the subject know what kind of evaluation result or mastication mode is shown by which animal on the display unit 16 before starting the mastication training.

(Example 2 of FB information presentation)
FIG. 5 is a diagram showing an example of presentation of FB information after completion of mastication training. FIG. 5 shows a total of two examples of display screens 51 and 52 . The display screen 51 displays an image 511 showing the comprehensive evaluation result of the subject and a message 512 to the subject. On one display screen 52, a message 521 for the subject, an image 522 showing ideal mouth movements, and an image 523 showing the subject's mouth movements are displayed.

Image 511 is an image of a rabbit, and represents that the subject's mouth movement was small as described above. Also, message 512 indicates a strategy for increasing the movement of the mouth. For example, when the percentage of the evaluation results for each mastication motion during mastication training that the range of motion is less than the required range of motion is equal to or greater than the threshold, the output control unit 105 generates and displays the display screen 51. You may make it display on the part 16.

Note that “camel chewing” included in the message 512 is suitable as a symbol of a chewing mode with sufficiently large mouth movements, since a camel eats while moving its mouth significantly in the vertical and horizontal directions. If the percentage of the evaluation results for each mastication motion during mastication training that the range of motion is greater than or equal to the required range of motion is greater than or equal to the threshold, the camel image is displayed instead of the rabbit image 511 . In addition to the camel, an animal such as an alpaca may be used as a chewing symbol with a sufficiently large mouth movement. As described above, it is preferable for the subject to recognize in advance the correspondence between the animal and the evaluation result or mastication mode.

Images 522 and 523 contained in one of the display screens 52 are graphical representations of the movement of the mouth, more precisely the movement of the mouth or of the feature points around it during chewing. Such an image is data supporting the evaluation, and by presenting it to the subject, it is possible to confirm the validity of the evaluation result.

A graph showing movements of the mouth or characteristic points around it during mastication may be displayed during mastication training. In this case, the output control unit 105 may display an animation in which plots are added to the graph as the subject's chewing motion progresses. An animation of moving on a plane may be displayed. As a result, the subject can recognize that his/her own masticatory motion and the graph are linked, so that it is possible to finely adjust the opening of the mouth, etc., and approach the ideal masticatory motion. . A figure showing how the mouth is opened may be displayed instead of the graph.

Moreover, when displaying the above animations during mastication training, the model generation unit 104 may be caused to generate similar animations and display them together. As a result, the subject can easily recognize the similarities and differences between the ideal mouth movement and the actual mouth movement through the animation.

Thus, the FB information may be a graph or graphic representing the magnitude of vertical and horizontal movement of the mouth. As a result, the subject can easily recognize the magnitude of vertical and horizontal movements of the mouth, and thus an improvement in the effect of mastication training can be expected. The graph or figure representing the magnitude of vertical and horizontal movements of the mouth may be graph 32 or graph 33 shown in FIG. is Mox and the vertical length is Moy, and the diagonal length of a quadrangle (indicated by a dashed line) may be shown.

(Processing flow)
The flow of processing executed by the information processing apparatus 1 will be described with reference to FIG. FIG. 6 is a flowchart showing an example of processing (mastication evaluation method) executed by the information processing apparatus 1 . In addition, below, the example which feedbacks during mastication training is demonstrated. The processing of S12 to S15, which will be described below, is an evaluation step of evaluating the magnitude of vertical and horizontal movements of the mouth during mastication of the subject based on the images 111 taken in chronological order of the mastication of the subject. is.

In S<b>11 , the image acquisition unit 101 acquires an image of the subject's mastication state, and stores it in the storage unit 11 as an image 111 . For example, when a moving image of a subject who masticates is captured by the imaging unit 12 , the image acquisition unit 101 may acquire frame images extracted from the moving image as the images 111 .

In S12, the feature point detection unit 1021 detects feature points indicating the movement of the subject's mouth from the image 111 acquired in S11, and stores the position information of the detected feature points in the storage unit 11 as the evaluation result 112. . Note that, as described above, the feature point detection unit 1021 may also detect feature points such as the inner corners of the eyes and the nose, in addition to the feature points in and around the mouth that indicate the movement of the mouth of the subject. .

In S13, the motion analysis unit 1022 analyzes the movement of the mouth of the subject during mastication based on the detection result of S12, and attempts to detect the mastication motion. Then, in S14, the motion analysis unit 1022 determines whether or not a chewing motion has been detected. By repeating the processing of S11 and S12 after the start of mastication, it is possible to specify the time-series position change of the feature points as shown in the graph 32 of FIG. Therefore, the motion analysis unit 1022 can identify one chewing motion from the position change pattern. Then, when identifying one chewing motion, the motion analysis unit 1022 associates the position information corresponding to the chewing motion stored as the evaluation result 112 as one corresponding to the one chewing motion. Also, the movement analysis unit 1022 calculates an index value (for example, Mox and Moy described above) indicating the magnitude of the mouth movement in the chewing action from the position information associated with one chewing action. It should be noted that the processing of S13 may be started when the images 111 are accumulated to such an extent that the chewing motion can be detected and the detection of feature points from the images 111 is finished.

In S<b>15 , the comparison unit 1023 compares the vertical and horizontal movements of the mouth of the subject during mastication with the required range of motion, and adds the result to the evaluation result 112 . In subsequent S16, the comparison unit 1023 determines whether or not FB (feedback) is necessary based on the evaluation result of S15. If it is determined to FB in S16, the process proceeds to S17, and if it is determined not to FB, the process returns to S11.

For example, when the motion analysis unit 1022 calculates Mox and Moy, in S15, the comparison unit 1023 compares the calculated Mox and Moy with the required lateral and vertical ranges of motion. As a result of this comparison, it may be determined to perform FB when both of the calculated Mox and Moy are less than the required range of motion, or FB when at least one of Mox and Moy is less than the required range of motion may be determined to perform In another example, the comparison unit 1023 may determine to output positive FB information when evaluating that the requested ranges of motion of Mox and Moy are satisfied.

In S17 (output control step), the output control unit 105 causes the display unit 16 to display and output FB information (for example, the display screen 4 shown in FIG. 4) corresponding to the evaluation result 112 . For example, when both or one of Mox and Moy is evaluated to be less than the required range of motion, the output control unit 105 may display a message prompting to move the mouth beyond the required range of motion as FB information. .

In S18, the image acquisition unit 101 determines whether or not to end the processing. If it is determined not to end in S18, the process returns to S11, and if it is determined to end, the process of FIG. 6 ends. Note that the end condition of S18 is not particularly limited, and for example, it may be determined that the process is ended when an operation to end the process is performed via the input unit 15 .

After the acquisition of the image of the subject's mastication ends, that is, after the mastication training ends (YES in S18), the output control unit 105 may execute S19 (not shown). In S19, the output control unit 105 may cause the display unit 16 to display the FB information (for example, the display screen 51 and the display screen 52 shown in FIG. 5) after training.

As described above, the mastication evaluation method according to the present embodiment is based on the images 111 obtained by photographing the state of mastication of the subject in time series. and an output control step (S17) for outputting feedback information according to the evaluation result to the display unit 16 (output device). Therefore, it is possible to perform more effective mastication training than mastication training in which only mastication is performed.

In addition, as described above, the evaluation unit 102 may perform the evaluation while the subject masticates. In this case, the output control unit 105 generates a The feedback information may be output to an output device such as the display unit 16 or the like. This allows real-time or near-real-time feedback to be given to the subject, thus enhancing the effectiveness of mastication training.

The evaluation in S15 may be performed by comparing with the subject's past evaluation results. For example, in S15, the comparison unit 1023 may compare Mox and Moy calculated most recently with Mox and Moy calculated earlier. In this case, in S17, the output control unit 105 may output feedback information such as that the movement of the mouth has increased or decreased based on the above comparison result. Also, at this time, how much the size of the mouth has changed may be displayed numerically.

[Embodiment 2]
(overview)
An overview of the mastication training system 2 according to this embodiment will be described with reference to FIG. FIG. 7 is a diagram showing an outline of the mastication training system 2. As shown in FIG. The mastication training system 2 is a system for making a subject perform highly effective mastication training. As illustrated, the mastication training system 2 includes a server 21 (information processing device), a terminal device 22, a display device 23 (output device), an imaging device 24, and a myoelectric potential measuring device 25. FIG. Although FIG. 7 shows an example in which the server 21 and the terminal device 22 are connected via a network, the connection mode between the server 21 and the terminal device 22 is arbitrary.

The server 21 evaluates the vertical and horizontal movements of the mouth of the subject during mastication based on the time-series images captured by the imaging device 24 . The evaluation method is generally the same as in the first embodiment, but the present embodiment differs from the first embodiment in that evaluation can be performed based on the measurement results of the myoelectric potential measuring device 25 .

The mastication training system 2 is provided with the myoelectric potential measuring device 25, so that the masticatory muscles of the subject generated based on the signal indicating the activity of the masticatory muscles of the subject measured during mastication of the subject. It is possible to output FB information including information on the magnitude of such load. As a result, even more effective mastication training can be realized in consideration of the magnitude of the load on the masticatory muscles.

The terminal device 22 exchanges data between devices included in the mastication training system 2 . For example, the terminal device 22 performs processing of transmitting an image captured by the imaging device 24 to the server 21, processing of displaying FB information according to the evaluation result by the server 21 on the display device 23, and the like. Note that FIG. 7 shows an example in which the terminal device 22 is a desktop personal computer, but any information processing device can be applied as the terminal device 22 . For example, a smart phone can be used as the terminal device 22 as in the first embodiment.

The display device 23 displays FB information on mastication of the subject. As described in the first embodiment, the FB information may be output in a form other than display, and a device other than the display device 23 may be applied as the FB information output device.

The photographing device 24 photographs the state of the subject's mastication in chronological order. The imaging device 24 may be one that shoots moving images, or one that shoots time-series still images.

The myoelectric potential measuring device 25 is a device for measuring myoelectric potential during mastication of the subject. FIG. 7 shows an example in which the myoelectric potential measuring device 25 is attached to each of the left and right masticatory muscles of the subject, but it may be attached to only one.

Instead of the myoelectric potential measuring device 25, another device capable of generating a signal indicating the activity of the muscle of mastication during mastication of the subject may be used. For example, a device that outputs a myogram may be used. Alternatively, a non-contact device for measuring the displacement of the masticatory muscle may be used. For example, it is possible to measure the displacement of the masticatory muscle by analyzing the image captured by the imaging device 24 or a proximity sensor for measuring the movement of the masticatory muscle. good too. In addition, for example, a 9-axis acceleration sensor can be used to measure the activity of the masticatory muscles. However, the activity of the masticatory muscle can be evaluated more accurately by using a device that measures a biosignal indicating the activity of the masticatory muscle, such as the myoelectric potential measuring device 25 .

As described above, the mastication training system 2 includes the photographing device 24 that photographs the state of mastication of the subject in chronological order, and the length of the mouth of the subject during mastication based on the time-series images photographed by the photographing device 24 . It includes a server 21 for evaluating the magnitude of directional and lateral movement, and a display device 23 for outputting feedback information according to the results of the evaluation. According to this mastication training system 2, it is possible to perform more effective mastication training than mastication training of simply mastication.

(Principal configuration of terminal device 22 and server 21)
The configurations of the terminal device 22 and the server 21 will be described with reference to FIG. FIG. 8 is a diagram showing an example of the main configuration of the terminal device 22 and the server 21. As shown in FIG. As illustrated, the terminal device 22 includes an image acquisition section 221 , a muscle activity information acquisition section 222 and an output control section 223 . The server 21 also includes a feature point detection unit 211 , a motion analysis unit 212 , a motion comparison unit 213 , a load analysis unit 214 , a load comparison unit 215 and an evaluation result transmission unit 216 .

Although not shown in FIG. 8, the terminal device 22 includes a control unit that controls each unit of the terminal device 22 and a storage unit that stores various data used by the terminal device 22. . The function of each block included in the terminal device 22 may be implemented by the control unit described above. The same applies to the server 21 as well.

The image acquisition unit 221 acquires images of the subject's mastication in chronological order. Specifically, the image acquisition unit 221 acquires images of the subject's mastication in chronological order captured by the imaging device 24 . The function of the image acquisition unit 221 is generally the same as that of the image acquisition unit 101 included in the information processing apparatus 1 of the first embodiment.

The muscle activity information acquisition unit 222 acquires information indicating activity of the subject's masticatory muscles measured during mastication by the subject. Specifically, the muscle activity information acquisition unit 222 acquires a signal indicating the myoelectric potential during mastication of the subject measured by the myoelectric potential measuring device 25 . The muscle activity information acquisition unit 222 may perform processing such as noise removal and frequency analysis on the acquired signal.

The output control unit 223 causes the display device 23 to output FB information according to the evaluation result by the server 21 . The function of the output control section 223 is substantially the same as that of the output control section 105 included in the information processing apparatus 1 .

The functions of the feature point detection unit 211, the motion analysis unit 212, and the motion comparison unit 213 are the same as those of the feature point detection unit 1021, the motion analysis unit 1022, and the comparison unit 1023 provided in the information processing apparatus 1, respectively.

The load analysis unit 214 acquires from the terminal device 22 the information indicating the activity of the masticatory muscles of the subject measured during mastication of the subject, which is acquired by the muscle activity information acquisition unit 222, and uses the information to analyze the activity of the subject. Calculate the index value that indicates the activity of the masticatory muscle.

In calculating the index value, the load analysis unit 214 may first detect the chewing motion. Since the myoelectric potential repeatedly rises and falls each time a masticatory action is performed, the load analysis unit 214 specifies, for example, the time when the myoelectric potential becomes equal to or higher than a threshold as the start timing of one chewing action. becomes less than the threshold value, and the period immediately before the myoelectric potential becomes equal to or greater than the threshold value again may be specified as a period in which one chewing motion is performed. Note that the motion analysis unit 1022 may use the identification result of the load analysis unit 214 instead of detecting the chewing motion by itself. Conversely, the load analysis unit 214 may use the mastication motion detection result by the motion analysis unit 1022 .

Then, the load analysis unit 214 may calculate the amount of muscle activity of the masticatory muscles in one mastication motion as an index value indicating the magnitude of the load in the mastication.

The load comparison unit 215 compares the index value calculated by the load analysis unit 214 with a predetermined threshold. The evaluation result of the load comparison unit 215 is the evaluation result of the activity of the masticatory muscle of the subject by the server 21 .

The evaluation result transmission unit 216 notifies the terminal device 22 of the evaluation result of the subject's mastication by the server 21 . For example, the evaluation result transmission unit 216 may transmit information indicating the comparison result by the load comparison unit 215 and the comparison result by the motion comparison unit 213 to the terminal device 22 . In this case, the output control unit 223 generates a display screen including FB information based on the received information, and causes the display device 23 to display it. Further, the evaluation result transmission unit 216 may generate FB information or a display screen including the FB information based on each of the above comparison results, and transmit the generated display screen to the terminal device 22 .

For example, when the load comparison unit 215 determines that mastication of the required load amount or more is not possible, the evaluation result transmission unit 216 displays FB information or a display screen containing the FB information to alert the subject to this fact. may be generated.

The load comparison unit 215 refers to the result of muscle activity information analyzed by the load analysis unit 214 . Then, the load comparison unit 215 compares the index value obtained from the muscle activity information obtained while the subject masticates a certain food item with the threshold value, and determines whether or not the required load amount is satisfied. do. As an example, the load comparison unit 215 compares the actually measured load amount calculated based on the number of times of mastication and the amount of muscle activity observed for the subject to a predetermined required load based on the ideal number of times of mastication and the amount of muscle activity. You can compare the quantity. The load comparison unit 215 determines that mastication of the required load amount or more is not possible when the actually measured load amount is less than the required load amount.

Moreover, the load comparison unit 215 may compare the actually measured load amount based on the number of times of mastication and the required load amount, and may also compare the actually measured load amount based on the amount of muscle activity and the required load amount.

If either condition (1) actually measured load amount based on the number of mastications≧required load amount or condition (2) actually measured load amount based on muscle activity≧required load amount is not satisfied, the load comparison unit 215 determines whether the required load amount is equal to the required load amount. It may be determined that mastication is not performed more than the load amount.

When the load comparison unit 215 determines that the user cannot masticate at the required load amount or more, the evaluation result transmission unit 216 sends a message such as, for example, “Food with a little more chewiness is more effective.” A message including a warning and a suggestion for improvement may be generated and transmitted to the terminal device 22 .

According to the above-described configuration, when the required load amount is set as ``training around the masticatory muscles at a constant % of the maximum occlusal pressure for a certain number of times or more'' or the like, is the required load amount satisfied? It is possible to appropriately determine whether or not.

The load comparison unit 215 may calculate the difference (insufficient amount of load) between the actual measurement value of the load amount obtained by mastication of the subject and the threshold representing the required load amount required for the mastication. Then, the evaluation result transmitting unit 216 may generate FB information that encourages behavior modification such that the calculated difference becomes small or the actual load amount exceeds the above threshold.

The terminal device 22 converts the video signal acquired by the image acquisition unit 221 from time to time and the biological signal acquired by the muscle activity information acquisition unit 222 from time to time during the measurement of the subject into real-time visualization data or audible data. It may have an information output mechanism for feedback with data. For example, the output control unit 223 as an information output mechanism may cause the display device 23 to display the video signal acquired by the image acquisition unit 221 and the biological signal acquired by the muscle activity information acquisition unit 222 as visualization data at any time. good. The mastication training system 2 may also include an audio output device (not shown) such as a speaker. The output control unit 223 of the terminal device 22 may output an audio signal obtained from an audio input device (not shown) such as a microphone to the audio output device as audible data at any time together with the video signal.

In addition, the output control unit 223 may output the audible data as FB information generated by the evaluation result transmission unit 216 of the server 21 to the audio output device at any time. For example, the audible data as FB information may be mastication sounds according to the amount of muscle activity during mastication of the subject. More specifically, the output control unit 223 outputs mastication sounds such as “cripply” and “popping” at a low volume in accordance with the mastication timing of the subject when the amount of muscle activity exerted is small. Alternatively, when the amount of muscle activity exerted is large, mastication sounds such as “Bari-Bari” and “Bori-Bori” with a high volume may be output.

(Example of presentation of FB information)
FIG. 9 is a diagram showing a presentation example of FB information. The display screen 6 shown in FIG. 9 includes, for example, an image 61 showing ideal mouth movements during mastication as model information, and an image 62 showing mouth movements during mastication of the subject. good too. The display screen 6 may also include a graph 65 showing muscle activity information acquired by the muscle activity information acquiring section 222 . Graph 65 is called an electromyogram. In addition to being able to recognize the difference between the ideal mouth movement and the target person's mouth movement, a high training effect can be expected. It can be expected that the target person can be made to have an awareness of muscle training, and the training effect can be further improved.

Display screen 6 may further include message 63 , message 64 and message 66 . Similar to the message 43, the message 63 indicates the difference between the vertical and horizontal movements of the mouth during mastication of the subject and the vertical and horizontal movements of the mouth during mastication of the comparison subject. It is an example of the FB information which shows the measure for making it small. The message 64 is an example of FB information displayed when the number of times of chewing before swallowing the chewing object held in the mouth is less than the threshold, similarly to the message 44 . The message 66 is an example of FB information that urges the subject, etc., to change their behavior in order to increase the training effect when the amount of activity of the masticatory muscles is less than the required load amount for which training effects can be expected.

Furthermore, the feedback information displayed on the display device 23 is, for example, information indicating at least one of a mastication object, a mastication training device, a medical institution, and a doctor (including a dentist) according to the evaluation result. may contain Thereby, it can contribute to the improvement of a subject's mastication.

(Example 2 of FB information presentation)
FIG. 10 is a diagram showing another presentation example of FB information. In particular, the display screen 7 shows an example of presentation of FB information after completion of mastication training.

The display screen 7 may include, for example, a comprehensive evaluation graph 71 and a message 72 . The comprehensive evaluation graph 71 is a graph in which the horizontal axis represents the amount of movement of the mouth of the subject and the vertical axis represents the amount of muscle activity (biting strength) exerted by the subject during mastication. is ideal, and it is configured so that the target person or the like intuitively recognizes the positioning from the ideal mastication action of the target person.

For example, a rabbit icon 711 may be arranged on the comprehensive evaluation graph 71 as a symbol indicating that the level of biting strength on the vertical axis is low and the level of mouth movement on the horizontal axis is low. For example, in the comprehensive evaluation graph 71, a camel icon 712 may be arranged as a symbol indicating that the level of biting strength on the vertical axis is low and the level of mouth movement on the horizontal axis is high. For example, a crocodile icon 714 may be arranged in the comprehensive evaluation graph 71 as a symbol indicating that the level of bite force on the vertical axis is high and the level of mouth movement on the horizontal axis is low. For example, in the comprehensive evaluation graph 71, an icon 715 of a strong camel may be arranged as a symbol indicating that the level of bite force on the vertical axis is high and the level of mouth movement on the horizontal axis is high. For example, in the comprehensive evaluation graph 71, a dog icon 713 may be arranged as a symbol indicating that both the bite force level on the vertical axis and the mouth movement level on the horizontal axis are intermediate. Then, on the comprehensive evaluation graph 71, the target person's icon 716 is arranged at a position corresponding to the level of chewing strength and the level of mouth movement in the masticatory action of the target person.

This allows the subject to intuitively recognize the gap between the current amount of mouth movement and the ideal, as well as the gap between the amount of muscle activity that the subject is currently exerting and the ideal. , can be expected to improve the effect of mastication training.

[Modification]
The output control unit 105 of the information processing device 1 or the evaluation result transmission unit 216 of the server 21 transmits the latest evaluation result of the subject, the past evaluation result evaluated in the past, and the same attributes such as age and gender. Based on the statistical information obtained from the population, individual FB information suitable for the subject may be generated. For example, when the expected improvement in masticatory function of the subject is not expected, the following information may be provided.

(1) Oral function information according to age (2) Information on devices for developing and improving masticatory function (3) Information on specialists according to the residential area of the subject Furthermore, the output control unit 105 of the information processing apparatus 1 , or the evaluation result transmission unit 216 of the server 21,
(4) Information indicating the type of food used for training, the number of chewing times suitable for the food, and the softness of the food that can be judged to be actually swallowable by the subject, and
(5) At present, information recommending foods for chewing training suitable for individual subjects may be provided as FB information.

The output control unit 105 of the information processing device 1 or the evaluation result transmission unit 216 of the server 21 converts the information (4) and (5) into mastication amount, mastication width, mouth opening amount, mastication speed, mastication cycle area, The content can be such that the value of the masticatory efficiency of the food estimated from the current number of teeth is corrected in the direction of increasing (higher masticatory efficiency score).

The output control unit 105 of the information processing device 1 and the output control unit 223 of the terminal device 22 quantify the feature points around the mouth obtained from the motion around the mouth when chewing the chewing function training food. The obtained time-series data may be used to display visualization data for visualizing the rhythm of mouth movements on the display unit 16 and the display device 23, respectively. The effect of mastication training can be enhanced by the subject or the like recognizing such visualized data.

Prior to the measurement of the subject, moving image data of the model person may be captured as a model in advance, and the numerical mastication data obtained from such moving image data or the accumulated numerical mastication data is used as a comparison target. You may employ|adopt as a threshold value.

〔effect〕
In addition, according to the configuration of the present disclosure, it is possible to train the masticatory power that may decline due to various factors such as aging and dysfunction, and to promote the development of the masticatory power of children in the process of growth, thereby improving the oral function. It can support maintenance and improvement. Chewing food, swallowing behavior, saliva secretion, etc. have a great influence on the brain and the whole body, and are factors that greatly affect mental and physical health and healthy life expectancy. Therefore, according to the configuration of the present disclosure, maintenance and improvement of oral functions are achieved, and it is expected to contribute to maintenance of mental and physical health, promotion of development, extension of healthy life expectancy, and the like. Ultimately, it contributes to the achievement of Goal 3 of the Sustainable Development Goals (SDGs), "Good Health and Well-Being for All."

[Example of realization by software]
The functions of the information processing device 1, the server 21, and the terminal device 22 (hereinafter referred to as "devices") are programs for causing a computer to function as the device, and each control block of the device (especially the control unit 10, It can be realized by a program (mastication training program) for causing a computer to function as the server 21 and each part included in the terminal device 22).

In this case, the apparatus comprises a computer having at least one control device (eg processor) and at least one storage device (eg memory) as hardware for executing the program. Each function described in each of the above embodiments is realized by executing the above program using the control device and the storage device.

The program may be recorded on one or more computer-readable recording media, not temporary. The recording medium may or may not be included in the device. In the latter case, the program may be supplied to the device via any transmission medium, wired or wireless.

Also, part or all of the functions of the above control blocks can be realized by logic circuits. For example, integrated circuits in which logic circuits functioning as the control blocks described above are formed are also included in the scope of the present invention. In addition, it is also possible to implement the functions of the control blocks described above by, for example, a quantum computer.

The present invention is not limited to the above-described embodiments, but can be modified in various ways within the scope of the claims, and can be obtained by appropriately combining technical means disclosed in different embodiments. is also included in the technical scope of the present invention.

1 information processing device 2 mastication training system 10 control unit 11 storage unit 12 imaging unit 13 communication unit 14 sound output unit 15 input unit 16 display unit 21 server 22 terminal device 23 display device 24 imaging device 25 myoelectric potential measurement device 101 image acquisition unit 102 evaluation unit 103 notification unit 104 model generation unit 105 output control unit 112 evaluation result 211 feature point detection unit 212 motion analysis unit 213 comparison unit 214 load analysis unit 215 load comparison unit 216 evaluation result transmission unit 221 image acquisition unit 222 muscle activity Information acquisition unit 223 Output control unit 1021 Feature point detection unit 1022 Motion analysis unit 1023 Comparison unit

Claims (16)

an evaluation unit that evaluates the vertical and horizontal movements of the mouth of the subject during mastication based on images taken in chronological order of the subject's mastication;
and an output control unit that causes an output device to output feedback information according to the result of the evaluation. The evaluation unit performs the evaluation during mastication of the subject,
2. The information processing apparatus according to claim 1, wherein said output control unit causes said output device to output feedback information generated based on said evaluation result each time said evaluation result is obtained. 3. The information processing apparatus according to claim 2, further comprising a notification unit that notifies said subject of timing of mastication according to at least one of said subject and a mastication object that said subject masticates. 4. The information processing apparatus according to any one of claims 1 to 3, wherein said feedback information is a graph or figure representing magnitudes of vertical and horizontal movements of the mouth. The evaluation unit evaluates the magnitude of longitudinal and lateral movements of the mouth during mastication of the subject based on the difference from the magnitude of longitudinal and lateral movements of the mouth during mastication of a comparison subject. ,
5. The information processing apparatus according to any one of claims 1 to 4, wherein said feedback information indicates measures for reducing said difference. Based on the distance between predetermined feature points on the face of the subject whose position does not change due to mastication, detected from the image, the evaluation unit measures the longitudinal direction and the lateral direction of the mouth during mastication of the subject. 6. The information processing apparatus according to any one of claims 1 to 5, wherein the magnitude of directional movement is evaluated. The information processing apparatus according to any one of claims 1 to 6, wherein the output control unit causes the output device to output an image showing movement of the mouth during ideal mastication together with the feedback information. 8. The information processing apparatus according to claim 7, wherein the image showing the movement of the mouth during ideal mastication is an image corresponding to at least one of the subject and an object to be masticated by the subject. 9. The information processing apparatus according to any one of claims 1 to 8, wherein the output control unit causes the output device to output an image obtained by photographing a state of mastication of the subject person in time series together with the feedback information. wherein the feedback information includes information about the magnitude of the load applied to the masticatory muscles of the subject, which is generated based on a signal indicating activity of the masticatory muscles of the subject measured during mastication of the subject. Item 10. The information processing apparatus according to any one of Items 1 to 9. 11. The feedback information according to any one of claims 1 to 10, wherein the feedback information includes information indicating at least one of a mastication object, a mastication training device, a medical institution, and a doctor according to the result of the evaluation. information processing equipment. The evaluation unit uses the results of evaluating the magnitude of vertical and horizontal movements of the mouth during mastication as objective variables, and describes images taken in chronological order of the state of mastication or feature values extracted from the images. 12. Any one of claims 1 to 11, wherein the vertical and horizontal movements of the mouth of the subject during mastication are evaluated using an evaluation model that has learned the relationship between the objective variable and the explanatory variable as variables. 1. The information processing apparatus according to 1. a photographing device for photographing the state of mastication of a subject in chronological order;
an information processing device that evaluates the magnitude of vertical and horizontal movements of the mouth of the subject during mastication based on time-series images captured by the imaging device;
and an output device that outputs feedback information according to the result of the evaluation. A mastication evaluation method executed by one or more information processing devices,
an evaluation step of evaluating the vertical and horizontal movements of the mouth of the subject during mastication based on images of the subject's mastication taken in chronological order;
and an output control step of causing an output device to output feedback information according to the result of the evaluation. A mastication training program for causing a computer to function as the information processing apparatus according to claim 1, wherein the mastication training program is for causing the computer to function as the evaluation unit and the output control unit. A computer-readable recording medium recording the mastication training program according to claim 15.

Images