JP2023094220A - Information processing method, manufacturing method, information processing device, and control program - Google Patents

Abstract

To evaluate adaptability to mastication training of a mastication object.SOLUTION: An information processing method of the disclosure, which is executed by one or more information processing devices (1), comprises: detection steps (S12 to S15) which, on the basis of images obtained by photographing in time series the appearance of an object person masticating a mastication object, detects the magnitude of a mouth movement when the object person is masticating; and an output step (S20) which outputs evaluation support information required to evaluate adaptability to mastication training of the mastication object on the basis of the detected magnitude of the mouth movement.SELECTED DRAWING: Figure 4

Description

The present invention relates to an information processing method, a manufacturing method, an information processing apparatus, and a control program for evaluating suitability of an object to be masticated for mastication training.

Conventionally, attempts have been made to obtain useful knowledge by analyzing the behavior of people eating. For example, Patent Document 1 below discloses a technique for quantitatively estimating the texture of a sample by measuring myoelectric potentials when a subject chews a sample and analyzing the measured myoelectric potentials. .

JP 2016-52516 A

Masticatory force may decline due to various factors such as aging and malfunction of sites related to mastication. In addition, the development of healthy oral functions is promoted by having sufficient opportunities to masticate food with appropriate hardness according to the developmental process in childhood. From this point of view, mastication training for training the masticatory power has been attracting attention in recent years. In order to perform effective mastication training, it is desirable to use a mastication object that allows movements of various organs related to mastication (for example, opening and closing movements of the mouth) within an appropriate range.

However, until now, there has been no technique for quantitatively evaluating the suitability of a masticatory object as a composition for use in training to train masticatory strength.

One aspect of the present invention has been made in view of the above problems, and an example of its purpose is to provide a technique for evaluating the suitability of a mastication object for mastication training.

To solve the above problems, an information processing method according to an aspect of the present invention is an information processing method executed by one or a plurality of information processing apparatuses, in which a subject masticates an object to be masticated. a detection step of detecting the magnitude of movement of the mouth of the subject during mastication based on the images captured in series; and an output step of outputting evaluation support information necessary for evaluating suitability.

In order to solve the above-described problems, an information processing apparatus according to an aspect of the present invention provides an image of the subject's mouth during mastication, based on images of the subject's chewing of the object to be masticated in chronological order. A detection unit for detecting the magnitude of movement, and an output for outputting evaluation support information necessary for evaluating suitability of the object to be masticated for mastication training based on the detected magnitude of movement of the mouth. and a control unit.

The information processing apparatus according to each aspect of the present invention may be implemented by a computer. In this case, the information processing apparatus is implemented by the computer by operating the computer as each part (software element) provided in the information processing apparatus. A control program for an information processing apparatus realized by a computer and a computer-readable recording medium recording it are also included in the scope of the present invention.

According to one aspect of the present invention, the suitability of an object to be masticated for mastication training can be evaluated.

BRIEF DESCRIPTION OF THE DRAWINGS It is a figure which shows the outline|summary of the information processing apparatus which concerns on Embodiment 1 of this invention, and the manufacturing method of the object to be masticated. 1 is a block diagram showing the main configuration of an information processing apparatus according to Embodiment 1 of the present invention; FIG. It is a figure explaining an example of the process which the said information processing apparatus performs. It is a flowchart which shows an example of the information processing method performed by the said information processing apparatus. FIG. 7 is a diagram showing an overview of an information processing device and a method of manufacturing a masticatory object according to Embodiment 2 of the present invention; FIG. 5 is a block diagram showing the main configuration of an information processing apparatus according to Embodiment 2 of the present invention; It is a flowchart which shows an example of the information processing method performed by the said information processing apparatus.

[Embodiment 1]
The inventors of the present invention believe that mastication training is not only about chewing strength, but rather, it is important to open and close the mouth wide to exercise various organs related to mastication. It can also be assumed that if the mouth moves well, the masticatory muscles are well loaded. Therefore, the present inventors considered that there is a need for a technology that can evaluate that an object to be masticated, which can induce a subject to open and close the mouth well, is highly suitable as a food for mastication training. I came up with the technology of In view of the above, the present embodiment provides a technique for quantitatively evaluating the suitability of an object to be masticated for mastication training.

Here, mastication training refers to behavior for assisting at least one of maintaining masticatory ability, improving masticatory ability, and mitigating the rate or degree of decline in masticatory ability. Masticatory ability refers to the ability to eat, chew, mix, and swallow specific foods (foods for evaluating mastication vary depending on age and body condition). When the ability of any one of these four movements is reduced, it interferes with food intake. Therefore, the ability to masticate is an important ability for humans to ingest food, and it is of great technical significance to evaluate the suitability of an object to be masticated for mastication training.

From the above, it can be concluded that the ability of the subject to perform at least one of the above four actions by masticating the object to be masticated can be maintained, improved, and the speed or degree of deterioration of the ability can be mitigated. When at least one of the effects can be expected, the object to be masticated may be evaluated as "suitable for mastication training."

<Overview>
An outline of an information processing apparatus 1 according to an embodiment of the present invention and a method of manufacturing a masticatory object using the information processing apparatus 1 will be described with reference to FIG. FIG. 2 is a diagram showing an overview of the information processing device 1 and the method of manufacturing a masticatory object according to the first embodiment. The information processing device 1 is a device that enables quantitative evaluation of a mastication object by outputting evaluation support information necessary for evaluating the suitability of the mastication object for mastication training. As the information processing device 1, it is possible to apply any information processing device, that is, a computer such as a server device, a personal computer, a notebook computer, a tablet terminal, and a smart phone.

In this embodiment, one or more subjects are allowed to eat a prototype of food as an object to be masticated for the purpose of mastication training. Subjects hold the prototype in their mouths and chew it. The object to be masticated is food for mastication training, and gummy, jelly, gum, etc. can be assumed as an example. The state of mastication of the subject is photographed by the photographing device 3. - 特許庁The photographing device 3 is a device, such as a camera or a video camera, that acquires an image of a subject as digital data. If the target person is present at the place where the information processing device 1 is installed, the photographing device 3 may be built in the information processing device 1 . The photographing device 3 may be communicably connected to the information processing device 1 by wire or wirelessly, and transmits images of the subject chewing the object to be masticated in chronological order as information via a wire or wirelessly. It may be transmitted to the processing device 1 . Alternatively, the photographing device 3 may record the photographed image in a detachable external memory, and the external memory in which the image is recorded is connected to the information processing device 1 to cause the information processing device 1 to acquire the image. may The photographing using the photographing device 3 may be performed by the subject himself/herself, or may be performed by another person in the vicinity of the subject.

The information processing apparatus 1 detects the magnitude of movement of the mouth of the subject during mastication based on time-series images obtained by photography. The information processing device 1 may detect the vertical and horizontal sizes of the mouth as distances, or may detect the size of the movement of the mouth as an area. Specifically, the distance may be the vertical and horizontal movement distance of the mouth or its surrounding feature points. The area may be the area of the region surrounded by the trajectory drawn by the mouth or its surrounding feature points. Then, the information processing apparatus 1 outputs evaluation support information including information that enables determination of suitability based on the magnitude of the detected mouth movement.

It is also conceivable to use an image captured by an imaging device 3 such as a three-dimensional camera capable of capturing movement of the mouth in the depth direction. In this case, the information processing apparatus 1 may detect the size of the mouth in the vertical, horizontal, and depth directions as distances, or may detect the size of the movement of the mouth as the volume. Specifically, the volume may be the volume of the space surrounded by the trajectory drawn by the mouth or its surrounding feature points.

For example, the evaluation support information may be an index value indicating the size of the movement of the mouth detected by the information processing device 1, or may be visualization information such as a graph or illustration that visualizes the size. good. Alternatively, the evaluation support information may be an evaluation result obtained by the information processing apparatus 1 evaluating the suitability of the prototype for mastication training based on the magnitude of the detected mouth movement. The evaluation result may include the difference between the above-described detected index value and a predetermined reference threshold, or the information processing apparatus 1 may determine suitability based on the difference. may contain The determination result may be binary information indicating suitability or unsuitability of the prototype as a chewing training food, or may be a score indicating the degree of suitability calculated based on the above-described difference.

The information processing device 1 may output the generated evaluation support information to the display device 2, for example. The evaluation support information displayed on the display device 2 is viewed by, for example, a user such as a manufacturer who attempts to manufacture a mastication object suitable as a mastication training food.

This enables the user to quantitatively evaluate the suitability of the object to be masticated for mastication training by utilizing the evaluation support information. Specifically, an object to be masticated such that movements of various organs related to mastication (for example, opening and closing of the mouth) are within an appropriate range can be evaluated as good.

Further, an adjustment step of adjusting the recipe of the object to be masticated so that the evaluation support information output by the information processing method executed by the information processing apparatus 1 satisfies a predetermined condition, and the recipe adjusted in the adjustment step. and a manufacturing step of manufacturing an object to be masticated based on the manufacturing method is also within the scope of the present invention. This makes it possible to quantitatively evaluate the suitability of the object to be masticated for mastication training, and then to easily manufacture a mastication object suitable for mastication training.

That is, the user can utilize the evaluation support information to manufacture a mastication object that can be expected to have the effect of mastication training.

Specifically, if the user does not obtain good results regarding the suitability of the prototype for mastication training, the user refers to the evaluation support information displayed on the display device 2 and considers the direction of improvement. becomes easier. For example, the user may adjust the recipe of the chewable object in order to bring the current prototype closer to the ideal chewing training food (adjustment step).

Then, a new prototype of the object to be masticated may be manufactured based on the adjusted recipe (manufacturing step). Similarly, the subject is allowed to eat the new prototype, the state of mastication is photographed by the photographing device 3, and evaluation support information for evaluating suitability is output based on the photographed image. may A chewable object as a final product may be manufactured (manufacturing step) based on the adjusted recipe when such trial and error is repeated and good results are obtained in terms of suitability.

The adjusting step and the manufacturing step described above may be realized by a computer or a computer system composed of a plurality of computers. For example, the computer or computer system may adjust the recipe by, for example, updating the parameters included in the recipe for the mastication object to new values in accordance with the evaluation support information output from the information processing device 1 ( adjustment step). A recipe may include, for example, parameters that define the shape, size, and/or composition of the object to be chewed. The computer or computer system may input the recipe with updated parameters to the manufacturing device, control the manufacturing device, and manufacture the chewable object based on the adjusted recipe (manufacturing step).

<Configuration of information processing device 1>
A more detailed configuration of the information processing apparatus 1 will be described with reference to FIG. FIG. 1 is a diagram showing an example of the main configuration of an information processing apparatus 1 according to the first embodiment. As shown in the figure, the information processing apparatus 1 includes a control section 10 that centrally controls each section of the information processing apparatus 1 and a storage section 11 that stores various data used by the information processing apparatus 1 .

Although not shown, the information processing apparatus 1 includes a photographing unit such as a camera for photographing images, a communication unit for communicating with other devices, and a sound output unit such as a speaker for outputting sound. good too. Further, the information processing apparatus 1 may include an input section for receiving input of various data to the information processing apparatus 1 and a display section such as a monitor for displaying and outputting various information. Note that the functions of the input unit and the display unit can also be realized by a touch panel.

The control unit 10 also includes an image acquisition unit 101 , a detection unit 102 , an evaluation unit 103 and an output control unit 105 . The detection unit 102 also includes a feature point detection unit 1021 and a motion analysis unit 1022 . These processing units in the control unit 10 can also be realized by, for example, installing a control program in the information processing device 1 for causing the information processing device 1 to function as these processing units. The storage unit 11 also stores an image 111 and evaluation support information 112 . As described above, the evaluation support information 112 includes information necessary for evaluating suitability of an object to be masticated, such as index values, visualization information, and evaluation results.

The image acquisition unit 101 acquires images of the subject's mastication in time series. In this embodiment, the image acquisition unit 101 acquires an image captured by the imaging device 3 . The image acquisition unit 101 may receive images from the imaging device 3 by communicating with the imaging device 3 via a communication unit (not shown). Alternatively, the image acquisition unit 101 may acquire an image stored in a detachable external memory or the like in the information processing apparatus 1 via a terminal (not shown). Further, the image acquisition unit 101 stores the acquired image as the image 111 in the storage unit 11 . The image 111 may be a moving image, or may be a time-series still image captured continuously. Further, the image acquiring unit 101 may acquire a moving image and store frame images extracted from the moving image as the image 111 .

The detection unit 102 detects the magnitude of the movement of the mouth of the subject during mastication based on the images 111 obtained by photographing the state of the subject's mastication in time series. As an example, the detection unit 102 may detect the magnitude of vertical and horizontal movements of the mouth. More specifically, the function of the detection unit 102 may be implemented by the feature point detection unit 1021 and the motion analysis unit 1022 .

The feature point detection unit 1021 detects predetermined feature points on the subject's face from the image 111 and outputs the position information to the motion analysis unit 1022 . The feature point detection unit 1021 may store position information of the detected feature points in the storage unit 11 as the evaluation support information 112 .

The motion analysis unit 1022 analyzes the motion of the mouth during mastication using the time-series position information. The movement analysis unit 1022 outputs to the evaluation unit 103 information indicating the magnitude of the movement of the mouth specified as a result of the analysis. The motion analysis unit 1022 may cause the storage unit 11 to store information indicating the magnitude of the motion of the mouth as the evaluation support information 112 . Specifically, the movement analysis unit 1022 may calculate an index value indicating the magnitude of mouth movement as information indicating the magnitude of mouth movement. The movement analysis unit 1022 may generate visualization information in which the magnitude of the mouth movement is visualized as information indicating the magnitude of the movement of the mouth. The visualization information may be a graph, a figure, an illustration, or the like representing the magnitude of the movement of the mouth. The motion analysis unit 1022 generates the graphs, figures, illustrations, etc. described above based on time-series position information and index values.

The evaluation unit 103 evaluates the suitability of the object to be masticated by the subject for mastication training based on the magnitude of the movement of the mouth of the subject detected by the detection unit 102 .

As an example of evaluation, the evaluation unit 103 may compare the index value calculated by the motion analysis unit 1022 with a predetermined threshold. The evaluation unit 103 may store the difference between the index value and the threshold obtained as a result of the comparison in the storage unit 11 as one of the evaluation results. That is, the difference described above is the evaluation support information 112 necessary for evaluating the suitability of the object to be masticated for mastication training.

To give a more specific example, the evaluation unit 103 determines whether the object to be masticated is suitable for mastication training based on the magnitude of vertical movement and lateral movement of the mouth of the subject detected by the detection unit 102 . Suitability may be assessed. More specifically, the evaluation unit 103 compares an index value indicating the magnitude of vertical movement of the mouth with a threshold corresponding to the vertical direction. The evaluation unit 103 also compares the index value indicating the magnitude of lateral movement of the mouth with a threshold value corresponding to the lateral direction. The evaluation unit 103 may cause the storage unit 11 to store the difference between the index value and the threshold for vertical movement and the difference between the index value and the threshold for horizontal movement as evaluation results.

As another example of evaluation, the evaluation unit 103 may determine whether the object to be masticated is suitable or unsuitable as a mastication training food according to the difference between the index value and the threshold. The evaluation unit 103 may cause the storage unit 11 to store, as one of the evaluation results, the determination result indicating whether the object to be masticated is suitable or unsuitable. That is, the determination result described above is the evaluation support information 112 necessary for evaluating the suitability of the object to be masticated for mastication training.

Further, when comparing the magnitude of vertical movement of the mouth and the magnitude of horizontal movement of the mouth with threshold values, the evaluation unit 103 may make determinations as follows. That is, the evaluation unit 103 may determine that the motion of the mouth in the vertical direction and the motion in the horizontal direction are both equal to or greater than the threshold value, and may determine that the motion of the mouth is unsuitable if either one of them is less than the threshold value. .

As another example of evaluation, the evaluation unit 103 may calculate a score indicating the degree of suitability of the object to be masticated as food for mastication training, according to the difference between the index value and the threshold. For example, the evaluation unit 103 may calculate the score so that the more the index value diverges from the threshold, the lower the degree of conformity. In particular, the evaluation unit 103 may calculate the score so that the degree of conformity decreases as the index value diverges below the threshold. According to this, for example, a lower score is calculated for an object to be masticated that is smaller in size or softer than the subject cannot open his/her mouth wide.

The output control unit 105 causes the display device 2 to output the evaluation support information 112 output from the detection unit 102 or the evaluation unit 103 . The evaluation support information 112 is information necessary for evaluating the suitability of the object to be masticated for mastication training based on the magnitude of the detected mouth movement. The evaluation support information 112 may include at least one of the above-described index value and visualization information output by the detection unit 102 . The evaluation support information 112 includes the evaluation results output by the evaluation unit 103, for example, the difference between the index value and the threshold value, the binary judgment result indicating suitability/nonconformity, and the judgment result calculated as the score of suitability. may contain.

The output destination to which the output control unit 105 outputs the evaluation support information 112 is not limited to the display device 2. For example, an output device such as a printer that outputs the evaluation support information 112 in printed form can be applied.

As described above, the information processing apparatus 1 includes the detection unit 102 that detects the magnitude of the movement of the mouth of the subject during mastication based on the images of the subject chewing the object to be masticated in time series. and an output control unit 105 for outputting evaluation support information necessary for evaluating suitability of the object to be masticated for mastication training based on the magnitude of the detected movement of the mouth. . This makes it possible to quantitatively evaluate the suitability of the object to be masticated for mastication training by utilizing the evaluation support information. For example, it is possible to assist the user in evaluating whether the object to be masticated has an appropriate range of movement of various organs related to mastication (for example, opening and closing of the mouth).

<Detection of feature points>
A method of detecting feature points by the feature point detection unit 1021 will be described with reference to FIG. FIG. 3 is a diagram showing an example of feature points to be detected and an example of evaluation of mouth movements based on the detection result of the feature points.

A schematic diagram 31 of a face in FIG. 3 shows an example of feature points to be detected. Since the feature points detected by the feature point detection unit 1021 are used for analyzing the movement of the mouth during mastication, it may be a part that moves in a way that reflects the mode of mastication during mastication. For example, the feature point detection unit 1021 detects points such as points 314 (upper lip apex), 315 (lower lip apex), and 316 (chin center position) in the schematic diagram 31, or the tip of the chin (not shown). A region around the mouth on the center line L of the face may be detected as a feature point. By detecting portions on the center line L as feature points, it is easy to specify the magnitude of lateral movement of the mouth during mastication.

Further, the feature point detection unit 1021 may detect points 317 and 318 (mouth corners) in the schematic diagram 31 as feature points. In this case, the movement analysis unit 1022 may analyze the movement of the mouth by regarding the movement of the midpoint between the point 317 (right corner of the mouth) and the point 318 (left corner of the mouth) as the movement of the mouth. In addition, for example, the feature point detection unit 1021 may detect a region of the cheek near the masseter muscle as a feature point. do it.

Further, the feature point detection unit 1021 may detect a motionless part caused by mastication. For example, the feature point detection unit 1021 may detect the point 313 (muzzle). In this case, the motion analysis unit 1022 can represent the position of the feature point appearing in the image 111 with the position of the point 313 (muzzle) as a reference. This makes it possible to accurately identify the time-series movement of each feature point even when the position of the subject is shifted between the time-series images 111 .

Furthermore, the feature point detection unit 1021 may detect a plurality of parts that do not move due to mastication in order to obtain a distance on the image 111 that serves as a reference for evaluating the magnitude of mouth movement. For example, the feature point detection unit 1021 may detect points 311 and 312 (inner corners of the eye). Thus, the evaluation unit 103 can evaluate the magnitude of the movement of the mouth based on the distance between the point 311 (right inner corner of the eye) and the point 312 (left inner corner of the eye). The feature point detection unit 1021 may use the points 311, 312, 313, 317, 318, etc. to obtain the center line L of the face.

In this way, the detection unit 102 detects the vertical direction of the mouth during mastication of the subject based on the distance between predetermined feature points on the subject's face whose positions do not change due to mastication, which are detected from the image 111 . and the magnitude of lateral motion. As a result, the magnitude of the movement of the mouth can be accurately detected regardless of whether the subject appears large or small in the image 111 .

Note that the subject or the like may input a distance that serves as a reference for evaluating the magnitude of the movement of the mouth. In addition, when the distance to the subject is known (for example, when shooting with a 3D camera that outputs distance information), the magnitude of mouth movement can be specified according to the distance, so it is possible to calculate the reference distance. Part detection can be omitted.

A method for detecting feature points as described above is not particularly limited, and for example, an image analysis technique, a machine learning technique, or a combination thereof may be applied. Further, when detecting a plurality of feature points, a feature point detection unit 1021 may be provided for each feature point to be detected. Alternatively, a marker for detection, a two-dimensional code, or a three-dimensional code may be pasted on the detection target portion of the subject's face to detect them from the image.

<Movement analysis>
The movement analysis unit 1022 analyzes the movement of the mouth during mastication from the time-series positional changes of the feature points detected as described above, and calculates an index value indicating the movement. This will be explained based on the graph 32 shown in FIG.

A graph 32 is a graph showing changes in position of feature points on the image 111 over time. The graph 32 can be generated from the positional information of the feature points appearing in the images 111 detected from each of the plurality of time-series images 111 .

For example, the apex of the lower lip (point 315 in schematic diagram 31) and the position of the chin (point 316 in schematic diagram 31) change as shown in graph 32 during mastication. In the graph 32, an initial position 321 indicates the position of the feature point at the start of mastication (immediately before opening the mouth). As shown in graph 32, after the start of mastication, the feature point moves to the lower left, then to the upper right, and then to the left. It should be noted that the left and right described here are the left and right when viewed from the side of photographing (observing) the subject, and that the left and right are reversed when viewed from the subject.

In this way, the graph 32 has the feature that the feature point moves from the initial position 321 at the start of mastication and returns to the vicinity of the initial position 321 again. Therefore, the motion analysis unit 1022 can use this feature to identify one chewing motion from the graph 32 (more precisely, the position information of feature points detected from each of the plurality of time-series images 111). . In addition, this makes it possible to calculate an index value indicating the number of times of mastication from when the object to be masticated is started to when the object is swallowed.

Further, from the graph 32, it is possible to specify the position when the feature point moves to the leftmost and the position when it moves to the rightmost in one chewing motion. Therefore, the motion analysis unit 1022 can also identify that the magnitude of lateral motion of the feature point is Mox from the position information of the feature point detected from each of the plurality of time-series images 111 . Similarly, the motion analysis unit 1022 determines the position (for example, initial position 321 at the start of mastication) and It is also possible to specify the position when the feature point moves to the lowest position and specify that the magnitude of vertical movement of the feature point is Moy.

Specifically, the motion analysis unit 1022 extracts position information included in one chewing motion from position information of feature points detected from each of the plurality of time-series images 111 . It is also possible to detect one chewing motion based on the movement of the corners of the mouth, the thickness of the lips, the displacement of the area of the cheek near the masseter muscle, and the like. Then, motion analysis section 1022 calculates the distance Mox between the leftmost position and the rightmost position among the pieces of positional information as an index value indicating the size of the mouth in one chewing motion. At the same time, the distance Moy between the position information indicating the highest position and the position information indicating the lowest position is calculated.

At this time, the motion analysis unit 1022 detects the vertical and horizontal directions of the subject's mouth based on the distance between predetermined feature points whose positions do not change due to mastication on the subject's face detected from the image 111 . An index value indicating the magnitude of the movement of the may be calculated. For example, the motion analysis unit 1022 calculates the distance Ex between the point 311 (right inner corner of the eye) and the point 312 (left inner corner of the eye) in the schematic diagram 31, and uses it as an index value indicating the magnitude of lateral movement of the mouth. A value of Mox/Ex may be calculated, and a value of Moy/Ex may be calculated as an index value indicating the magnitude of vertical movement of the mouth. This makes it possible to make the values of Mox and Moy dimensionless and cancel the difference in the size of the subject in the image 111 .

Note that the index value indicating the movement of the mouth during mastication is not limited to the above example. For example, it can be said that the area of the portion surrounded by the graph 32 (hereinafter referred to as Mos) indicates the magnitude of mouth movement. value. Mos may also be made dimensionless by dividing by the distance (eg, the above-mentioned Ex) between predetermined feature points whose positions do not change due to mastication.

Further, the movement analysis unit 1022 may calculate the length of the diagonal distance obtained from Mox and Moy by the Pythagorean theorem as an index value representing the magnitude of vertical and horizontal movement of the mouth. . Specifically, with reference to the graph 32 shown in FIG. 3, the motion analysis unit 1022 calculates the length of the diagonal of a rectangle (indicated by a dashed line) whose horizontal length is Mox and whose vertical length is Moy. may be calculated as an index value representing the magnitude of vertical and horizontal movement of the mouth.

The motion analysis unit 1022 may also calculate an index value other than the index value indicating the magnitude of the mouth movement. For example, the motion analysis unit 1022 may use the number of times of mastication per unit time, the number of times of mastication from holding an object to be masticated in the mouth until swallowing, the rhythm of mastication, and the like as index values. In addition to this, the motion analysis unit 1022 may, for example, determine which side is chewed on the left or right side based on the movement of the corners of the mouth, and calculate the number of times of chewing on the left or right side or the ratio thereof as an index value. Also, for example, the motion analysis unit 1022 may determine whether or not the motion of the feature point is within the normal range, and output an index value indicating the determination result.

In obtaining the index value, the motion analysis unit 1022 may use the values of Mox and Moy in one chewing motion, or may use the average values of Mox and Moy in an arbitrary number of chewing motions. Alternatively, the total values of Mox and Moy in an arbitrary number of chewing motions may be used.

<Comparison>
The evaluation unit 103 compares the index value calculated by the motion analysis unit 1022 as described above with a predetermined threshold value. The difference between the index value obtained as a result of this comparison and the threshold value is one of the evaluation results, and serves as the evaluation support information 112 for determining the suitability of the mastication object as a mastication training food. The predetermined threshold value is the longitudinal and lateral movements of the mouth of a comparison subject (e.g., ideal mastication for a subject or a group with the same attributes as the subject, or mastication performed by the subject in the past). It may be set based on the size. The threshold may be determined, for example, based on the ideal size of the mouth that the subject should open to obtain the effect of chewing training.

For example, assume that the motion analysis unit 1022 calculates Mox and Moy as index values indicating the magnitude of vertical and horizontal motion of the mouth, respectively. In this case, the evaluation unit 103 compares the Mox and Moy calculated by the motion analysis unit 1022 with preset thresholds for Mox and Moy, respectively, to determine the vertical and horizontal movements of the mouth of the subject. can be evaluated. Mox and Moy, which are set as threshold values, represent the range of motion in the longitudinal and lateral directions required of the subject to obtain the effect of mastication training, and are hereinafter referred to as the required range of motion. For example, when the comparison result of the evaluation unit 103 indicates that the Mox calculated by the movement analysis unit 1022 is less than the required range of motion in the lateral direction, it is determined that the mouth movement in the lateral direction was insufficient in the mastication. I can say. Similarly, when the comparison result of the evaluation unit 103 indicates that the Moy calculated by the movement analysis unit 1022 is less than the required range of motion in the vertical direction, the mouth movement in the vertical direction is insufficient for mastication. It can be said that

As described above, Mox and Moy used for such comparison are dimensionless based on the distance between predetermined feature points whose positions do not change due to mastication on the subject's face. good. That is, the evaluation unit 103 may compare and evaluate the Mox/Ex and Moy/Ex described above with the required range of motion. In this case, the required range of motion is also made dimensionless in the same manner.

In addition, the evaluation unit 103
Moy/Mox*Ex≦required range of motion Equation 1
or,
Mos/Ex≦required range of motion Equation 2
is satisfied, the mouth movement may be evaluated as insufficient.

In addition, the evaluation unit 103 may divide one chewing action into a plurality of stages, evaluate each stage, and obtain a final evaluation result by synthesizing the evaluation results. This will be explained based on the graph 33 in FIG. A graph 33 in FIG. 3 shows an example in which one chewing motion is divided into 10 stages from level 0 (start/end of chewing motion) to level 10 (maximum mouth opening) and each stage is evaluated. The left and right shown in the graph 33 correspond to the left and right seen from the subject.

In this case, the motion analysis unit 1022 calculates an index value indicating the magnitude of lateral motion of the mouth at each stage. Then, the evaluation unit 103 compares these index values with the required range of motion in the horizontal direction at each stage, and evaluates the horizontal movement of the mouth of the subject based on the comparison result. For example, the evaluation unit 103 calculates the ratio or difference between each index value and each required range of motion, and compares the total value or average value of the calculation results with a predetermined threshold to determine the width of the subject's mouth. Directional movement may be evaluated.

Note that the evaluation unit 103 may evaluate the movement of the mouth based on the degree of similarity of the figures drawn by the trajectory of one mastication, instead of the evaluation divided into 10 levels. The degree of similarity is a value indicating the degree to which the shapes of figures are similar, and the method of calculating the degree of similarity is not particularly limited.

Moreover, the evaluation unit 103 may evaluate the movement of the mouth taking into account the movement in the vertical direction in addition to the movement in the horizontal direction. In addition, the index value may be normalized by the maximum value of Moy, or may be dimensionless by Ex or the like.

Further, when the motion analysis unit 1022 calculates index values other than the index value indicating the magnitude of the mouth movement, the evaluation unit 103 may also compare these index values. For example, the evaluation unit 103 evaluates evaluation values such as the number of times of mastication per unit time, the number of times of mastication from holding an object to be masticated in the mouth until swallowing, the rhythm of mastication, etc. can be compared with

In addition, the evaluation unit 103 may evaluate the suitability of the object to be masticated by integrating various index values calculated by the motion analysis unit 1022 and comprehensively evaluating the mastication of the subject. For example, the evaluation unit 103 may calculate the masticatory efficiency as information indicating the subject's evaluation result. In this case, the motion analysis unit 1022 calculates various index values that are correlated with the masticatory efficiency, and uses a masticatory efficiency prediction model that models the relationship between the index values and the masticatory efficiency. The masticatory efficiency can be calculated from the values. In this case, the evaluation unit 103 may compare the calculated mastication efficiency with the ideal mastication efficiency. Also, the ideal masticatory efficiency may be calculated based on the subject's attribute information such as the number of teeth of the subject.

As an evaluation method to be executed by the evaluation unit 103, if the evaluation support information 112 that enables evaluation of the suitability of the object to be masticated based on the magnitude of movement of the mouth during mastication of the subject can be provided. Any evaluation method can be adopted.

For example, the evaluation unit 103 may calculate the degree of similarity between the mastication mode (for example, an ideal mastication mode in which a training effect can be expected) that serves as an evaluation reference and the subject's mastication mode. Specifically, the evaluation unit 103 may calculate the degree of similarity between a moving image showing a mastication mode, which serves as a reference for evaluation, and a moving image of the subject's mastication state. The degree of similarity of the extracted time-series still images may be calculated. Further, for example, the evaluation unit 103 may calculate the degree of similarity of patterns of time-series changes in the positions of feature points extracted from those images.

In addition, the evaluation unit 103 uses an evaluation model constructed by machine learning to evaluate the magnitude of vertical and horizontal movements of the mouth of the subject during mastication, thereby determining suitability of the object to be masticated. may be evaluated. This evaluation model uses the results of evaluation of the vertical and horizontal movements of the mouth during mastication as objective variables, and images taken chronologically of the state of mastication or feature values extracted from the images as explanatory variables. , can be constructed by learning the relationship between the objective variable and the explanatory variable. According to this configuration, highly accurate evaluation results can be generated.

As the feature amount, for example, time-series position information of the feature points detected by the feature point detection unit 1021 can be used. When the image 111 is input to the evaluation model and evaluated, the motion analysis unit 1022 can be omitted.

<About evaluation support information>
The output control unit 105 causes the display device 2 to display the evaluation support information 112 output by at least one of the detection unit 102 and the evaluation unit 103 . The evaluation support information 112 may be any information that is necessary for evaluating the suitability of the mastication object for mastication training based on the magnitude of the detected mouth movement.

For example, the evaluation support information 112 may include detection results output by the detection unit 102 . Specifically, the evaluation support information 112 may be an index value Mox indicating the magnitude of horizontal movement of the mouth and an index value Moy indicating the magnitude of vertical movement of the mouth. For example, the evaluation support information 112 may be visualization information generated by the detection unit 102 (for example, graphs 32 and 33 in FIG. 3, etc.).

Also, the evaluation support information 112 may include the evaluation result output by the evaluation unit 103 . Specifically, the evaluation support information 112 may be the difference between the index value compared by the evaluation unit 103 and the threshold. The evaluation support information 112 may be the determination result of the evaluation unit 103 regarding the object to be masticated. The determination result may be binary information indicating suitability or non-suitability, or may be multivalued information calculated by the evaluation unit 103 as a score indicating the degree of suitability as a chewing training food.

Based on the difference between the index value and the threshold, the evaluation unit 103 may determine the direction of recipe improvement to fill the gap between the current prototype and the ideal object to be masticated. In this case, the evaluation unit 103 may generate information indicating the direction of improvement of the recipe, and store this in the storage unit 11 as the evaluation support information 112 .

As a specific example, when the evaluation unit 103 determines that Mox (or Mox/Ex) is less than the required range of motion, the evaluation unit 103 determines that the lateral movement of the mouth of the subject is increased. , a refinement may be determined to make the shape of the object to be masticated flatter. In this case, for example, the output control unit 105 displays, as the evaluation support information 112, a message stating that "it is recommended that the shape of the object to be masticated be made flatter in order to increase the lateral movement of the mouth." You may make it display on the apparatus 2. In this way, the user is motivated to make a prototype of an object to be masticated that encourages the subject to move the mouth in the lateral direction more greatly, which can contribute to the improvement of the prototype.

<Process flow>
An information processing method executed by the information processing apparatus 1 according to the first embodiment is executed by one information processing apparatus 1 or a plurality of computers functioning as the information processing apparatus 1 . The information processing method generally includes a detection step of detecting the magnitude of movement of the mouth of the subject during mastication based on images taken in time series of the subject chewing the mastication object; and an outputting step of outputting evaluation support information necessary for evaluating suitability of the object to be masticated for mastication training based on the magnitude of mouth movement.

FIG. 4 is a flow chart showing an example of the above-described information processing method executed by the information processing apparatus 1. As shown in FIG. In the illustrated example, S12 to S15 correspond to an example of the detection steps described above. Also, S20 corresponds to an example of the output step described above.

The information processing method of the present disclosure may further include an evaluation step of evaluating suitability of the mastication object for mastication training based on the magnitude of mouth movement detected in the detection step. In this case, the output step may output the evaluation result of the evaluation step as the evaluation support information. In the illustrated example, S16 to S19 correspond to one example of the evaluation steps described above.

A series of processes of the information processing method is started, for example, by inputting an image captured by the imaging device 3 to the information processing device 1 .

In S<b>11 , the image acquisition unit 101 acquires an image of the subject's mastication state, and stores it in the storage unit 11 as an image 111 . For example, when a moving image of a subject who masticates is captured by the imaging device 3 , the image acquisition unit 101 may acquire frame images extracted from the moving image as the image 111 .

In S<b>12 , the feature point detection unit 1021 detects feature points indicating the movement of the subject's mouth from the image 111 acquired in S<b>11 , and outputs position information of the detected feature points to the motion analysis unit 1022 . The feature point detection unit 1021 stores data obtained by arranging the position information of the detected feature points in time series, or visualization information generated based on the data (for example, the graph 32 in FIG. 3) as the evaluation support information 112. 11 may be stored. Note that, as described above, the feature point detection unit 1021 may also detect feature points such as the inner corners of the eyes and the nose, in addition to the feature points in and around the mouth that indicate the movement of the mouth of the subject. .

In S13, the motion analysis unit 1022 analyzes the movement of the mouth of the subject during mastication based on the detection result of S12, and attempts to detect the mastication motion.

In S14, the motion analysis unit 1022 determines whether or not one chewing motion has been detected. When the motion analysis unit 1022 detects a chewing motion, the process proceeds from YES in S14 to S15. While the motion analysis unit 1022 does not detect one chewing motion, the process returns from NO in S14 to S11, and the processes after S11 are repeated for the next frame image.

By repeating the processing of S11 to S14 for a series of frame images from the frame image at the start of one chewing motion to the frame image at the end of the chewing motion, the graph 32 in FIG. Time-series positional changes of feature points can be identified. Therefore, the motion analysis unit 1022 can detect one chewing motion from the position change pattern.

In S15, the motion analysis unit 1022 associates the time-series position information output from the feature point detection unit 1021 regarding the detected chewing motion for one time with the chewing motion. Then, the movement analysis unit 1022 calculates an index value indicating the magnitude of the movement of the mouth in the chewing action based on the time-series position information corresponding to one chewing action.

For example, the movement analysis unit 1022 may detect the magnitude of vertical and horizontal movement of the mouth of the subject during mastication based on the image. Specifically, the motion analysis unit 1022 may detect an index value Mox indicating the magnitude of horizontal movement of the mouth and an index value Moy indicating the magnitude of vertical movement of the mouth.

Further, the movement analysis unit 1022 may calculate the length of the diagonal distance obtained from Mox and Moy by the Pythagorean theorem as an index value representing the magnitude of vertical and horizontal movement of the mouth. .

More specifically, the motion analysis unit 1022 detects the movement of the mouth of the subject during mastication based on the distance between predetermined feature points on the subject's face whose positions do not change due to mastication, which are detected from the image. Vertical and lateral motion magnitudes may be detected.

In S16, the evaluation unit 103 compares the index value calculated in S15 with a preset threshold value (that is, the required range of motion).

For example, the evaluation unit 103 determines the vertical and horizontal sizes of the mouth of the subject during mastication, and the vertical and horizontal sizes of the mouth during mastication required to obtain the effects of mastication training. Alternatively, a difference from each threshold value indicating the degree may be calculated.

More specifically, the evaluation unit 103 compares the index value Mox calculated in S15 with the required range of motion of the lateral movement of the mouth, and compares the index value Moy calculated in S15 with the The amount of vertical movement may be compared to the required range of motion. In another example, the evaluation unit 103 may compare the length of the diagonal distance obtained from Mox and Moy by the Pythagorean theorem with a preset threshold value.

Then, the evaluation unit 103 may cause the storage unit 11 to store the difference between the index value and the threshold value as the evaluation support information 112 .

In S17, the evaluation unit 103 may evaluate the suitability of the prototype (object to be masticated) masticated by the subject as food for mastication training based on the difference obtained as a result of the comparison.

As an example, when the index value is equal to or greater than the threshold, the evaluation unit 103 may evaluate that the prototype is suitable for mastication training, and proceed from NO in S17 to S18. ``The fact that the index value is above the threshold (required range of motion) means that ``the prototype is able to induce the subject to open and close the mouth as wide as the training effect can be expected.'''' The evaluation unit 103 is configured to perform the above evaluation based on.

On the other hand, if the index value is less than the threshold, the evaluation unit 103 may evaluate that the prototype is not suitable for mastication training, and proceed from YES in S17 to S19. ``If the index value is less than the threshold (required range of motion), it means that ``the prototype is not able to induce the subject to open and close the mouth as wide as the training effect can be expected.'''' The evaluation unit 103 is configured to perform the above evaluation based on.

In S<b>18 , the evaluation unit 103 may cause the storage unit 11 to store the evaluation result “prototype conforms” as the evaluation support information 112 .

In S<b>19 , the evaluation unit 103 may cause the storage unit 11 to store, as the evaluation support information 112 , the evaluation result that the “prototype is unsuitable”.

If the user does not need the judgment result indicating conformity/nonconformity of the prototype, S18 and S19 can be omitted.

In S<b>20 , the output control unit 105 outputs the evaluation support information 112 generated by the detection unit 102 or the evaluation unit 103 . For example, the output control unit 105 causes the display device 2 to display the evaluation support information 112 .

As an example, the output control unit 105 outputs at least one of an index value indicating the size of the mouth movement detected in the detection steps (S12 to S15) and visualization information visualizing the size to the evaluation support information 112. can be output as

As described above, according to the information processing method of the present disclosure, it is possible to obtain evaluation support information necessary for evaluating suitability of an object to be masticated for mastication training based on the magnitude of movement of the mouth. Then, by utilizing the evaluation support information, it becomes possible to quantitatively evaluate the suitability of the object to be masticated for mastication training. For example, it is possible to assist the user in evaluating whether the object to be masticated has an appropriate range of movement of various organs related to mastication (for example, opening and closing of the mouth).

<Modification>
In the information processing method described above, at least one of an index value indicating the magnitude of the mouth movement detected in the detection step and visualization information visualizing the magnitude can be presented to the user as evaluation support information. This allows the user to quantitatively and simply grasp the suitability of the object to be masticated for mastication training. This helps the user to easily find the direction of adjustment for improving the prototype.

In the information processing method described above, the evaluation result obtained by the evaluation step can be presented to the user as evaluation support information. The evaluation result may be a difference from a threshold value, may be binary judgment result information indicating conformity/nonconformity, or may be a score calculated as a score indicating the degree of conformity based on the above-described difference. It may be value judgment result information. As a result, even a user who does not have specialized knowledge about the correlation between the magnitude of the mouth movement and the suitability of the object to be masticated for mastication training can understand the suitability of the object to be masticated for mastication training. can be recognized directly.

In the above-described information processing method, the vertical and horizontal dimensions of the mouth during mastication of the subject and the vertical and horizontal dimensions of the mouth during mastication required to obtain the effect of mastication training. Suitability can be evaluated based on the difference from each threshold indicating magnitude. As a result, it is possible to more appropriately evaluate the suitability of the object to be masticated for mastication training by considering not only the longitudinal movement of the mouth but also the lateral movement thereof.

(The object to be chewed is evaluated based on multiple chewing motions per subject)
FIG. 4 shows an example of evaluating an object to be masticated based on the magnitude of mouth movement for one mastication motion. However, in the information processing method of the present disclosure, the object to be chewed may be evaluated based on multiple chewing motions per subject.

For example, it is assumed that N chewing motions to be analyzed are determined to be effective in evaluating the suitability of the prototype. In this case, the information processing apparatus 1 can be configured to proceed to the evaluation step after N chewing motions are detected.

Specifically, the information processing apparatus 1 may be configured to repeat the series of processes S11 to S14 for detecting one chewing motion N times. For example, after S14, the motion analysis unit 1022 determines whether or not N chewing motions have been detected, and if N chewing motions have been detected, the process proceeds to S15. In S15, the movement analysis unit 1022 statistically processes the index value indicating the magnitude of the mouth movement of the N chewing motions to calculate an integrated index value for one subject. may

This makes it possible to more properly evaluate the suitability of the object to be masticated for mastication training for one subject.

(Evaluate the object to be masticated based on the mastication motions of a plurality of subjects)
FIG. 4 shows an example of evaluating chewing objects based on the magnitude of mouth movements of one subject. However, in the information processing method of the present disclosure, the object to be chewed may be evaluated based on the chewing motions of a plurality of subjects.

For example, it is assumed that the number of subjects effective for evaluating the suitability of the prototype is determined to be M persons. In this case, the information processing apparatus 1 can be configured so that the evaluation step is performed after the detection step is performed for M moving images in which the chewing motions of M people are captured.

Specifically, the information processing apparatus 1 may be configured to repeat a series of processes S11 to S15 for obtaining an index value for one subject M times. Then, the motion analysis unit 1022 may statistically process the index value obtained in S15 for each subject to obtain an integrated index value for a group of subjects, and use it as an input for the evaluation step.

As a result, it is possible to more properly evaluate the suitability as a food for mastication training with high versatility.

(Evaluate the object to be masticated based on multiple types of index values)
In the evaluation performed in S17, the evaluation unit 103 selects the food for mastication training when both the calculated index value Mox and index value Moy are less than the threshold (required range of motion) as a result of the comparison in S16. It may be judged as non-conforming. Alternatively, the evaluation unit 103 may determine that the food for mastication training is unsuitable when at least one of the index value Mox and the index value Moy is less than a threshold value.

<Application example>
By applying the information processing method and manufacturing method of the present disclosure, it is possible to easily manufacture, for example, a product that can be expected to have the effect of mastication training.

(Development of foods for mastication training for the general public)
For example, when developing a food for mastication training for the elderly, a plurality of subjects belonging to the elderly are asked to eat the prototype, and videos of their mastication are collected. Then, the collected moving images are input to the information processing apparatus 1 to obtain the evaluation support information 112 for the prototype.

Using the obtained evaluation support information 112, the user can find the direction of improvement, adjust the recipe of the prototype, and manufacture more suitable food for mastication training for the elderly. can.

According to the information processing method of the present disclosure, the photographing device 3 is the means for measuring the subject chewing the prototype and obtaining the measurement data. The imaging device 3 is more readily available than a myoelectric potential measurement device and is considered to be widely owned. For example, a smartphone camera can be used as the imaging device 3 .

Therefore, it is possible to collect a large amount of measurement data much more easily than the development method of gathering measurement data by inviting many subjects to a facility equipped with myoelectric potential measurement devices. be. For example, the subject eats the prototype mailed to his/her home, takes a picture of the state of mastication using the photographing device 3 at home, and then sends the photographed video to the developer's information processing device 1 or the like. Just send it. As described above, according to the information processing method of the present disclosure, development costs can be reduced and development efficiency can be significantly improved compared to the case where the myoelectric potential measuring device is used to evaluate prototypes.

Also, for example, when developing foods for chewing training for children, according to the information processing method of the present disclosure, there are further advantages. For example, in a configuration in which a prototype is evaluated using a myoelectric potential measuring device or the like, it is necessary to wear various devices for measurement on a child as a subject. It is very stressful for children to wear an unknown device, and they may not be able to measure their mastication as usual, and they may not be able to eat it in the first place. labor is likely to be expended. On the other hand, according to the information processing method of the present disclosure, the means for obtaining measurement data is the photographing device 3, so measurement data required for evaluation can be easily obtained without contacting the subject. As described above, according to the information processing method of the present disclosure, development costs can be reduced and development efficiency can be significantly improved compared to the case where the myoelectric potential measuring device is used to evaluate prototypes.

(Manufacturing and sales of chewing training foods for individuals)
The information processing method and manufacturing method of the present disclosure can also be applied to manufacturing and selling foods for individual chewing training. For example, a food manufacturer can offer the following services to its customers.

A personalized chewing training food is mailed to a customer who purchases the chewing training food. The food product may be a prototype manufactured to a standard recipe selected based on a pre-entered customer profile or the like. The customer, who is the target person, eats the mailed food and transmits a moving image of chewing the food to the information processing device 1 of the manufacturer/seller.

The information processing device 1 analyzes the transmitted video and outputs evaluation support information 112 . The manufacturer/marketer can utilize the obtained evaluation support information 112 to find the direction of improvement and adjust the recipe of the prototype. Then, the manufacturer/seller can manufacture and sell more suitable foods as foods that enhance the effect of mastication training for individual customers. By storing recipes for each customer, it is possible to immediately respond to additional orders, and it is also possible to adjust the recipe according to the improvement of the customer's masticatory function.

[Embodiment 2]
Other embodiments of the invention are described below. For convenience of description, members having the same functions as those of the members described in the above embodiments are denoted by the same reference numerals, and description thereof will not be repeated.

<Overview>
An outline of an information processing apparatus 1 according to an embodiment of the present invention and a method of manufacturing an object to be masticated using the information processing apparatus 1 will be described with reference to FIG. FIG. 5 is a diagram showing an overview of the information processing device 1 and the method of manufacturing a masticatory object according to the second embodiment.

A different point from the information processing method according to the first embodiment is that, in addition to the photographing device 3, a myoelectric potential measuring device 4 is employed as means for measuring the state of mastication of the subject. Myoelectric potential measuring device 4 is a device for measuring myoelectric potential during mastication of a subject. Although FIG. 5 shows an example in which the myoelectric potential measuring device 4 is attached to each of the left and right masticatory muscles of the subject, it may be attached to only one. Myoelectric potential measurement device 4 transmits muscle activity information indicating the measured myoelectric potential to information processing device 1 . By using a device such as the myoelectric potential measuring device 4 that measures a biological signal indicating the activity of the masticatory muscle, the activity of the masticatory muscle can be accurately evaluated.

Instead of the myoelectric potential measuring device 4, another device capable of generating a signal indicating the activity of the muscle of mastication during mastication of the subject may be used. For example, a device that outputs a myogram may be used. Alternatively, a non-contact device for measuring the displacement of the masticatory muscle may be used. For example, it is possible to measure the displacement of the masticatory muscle by analyzing the image captured by the imaging device 3 or a proximity sensor for measuring the movement of the masticatory muscle. good too. In addition, for example, a 9-axis acceleration sensor can be used to measure the activity of the masticatory muscles.

<Configuration of information processing device 1>
FIG. 6 is a diagram showing an example of the main configuration of the information processing apparatus 1 according to the second embodiment. A different point from the information processing apparatus 1 according to the first embodiment is that the control unit 10 of the information processing apparatus 1 further includes an analysis unit 104 . The analysis unit 104 can also be realized by installing, in the information processing apparatus 1, a control program for causing the information processing apparatus 1 to function as the analysis unit 104, for example.

The analysis unit 104 acquires muscle activity information indicating the activity of the masticatory muscles of the subject measured during mastication of the subject, and analyzes the load applied to the masticatory muscles of the subject. Specifically, the analysis unit 104 acquires a signal indicating the myoelectric potential during mastication of the subject measured by the myoelectric potential measuring device 4 . The analysis unit 104 may perform processing such as noise removal and frequency analysis on the acquired signal. Then, the analysis unit 104 analyzes the signal indicating the myoelectric potential and calculates an index value indicating the activity of the masticatory muscle of the subject.

In calculating the index value, the analysis unit 104 may first detect the chewing motion. The myoelectric potential repeatedly rises and falls each time a masticatory action is performed. Therefore, for example, the analysis unit 104 specifies the time point when the myoelectric potential becomes equal to or higher than the threshold as the start timing of one chewing motion, and thereafter, the myoelectric potential becomes smaller than the threshold and immediately before the myoelectric potential becomes equal to or higher than the threshold again. may be identified as the period during which one chewing motion was performed. Note that the motion analysis unit 1022 may use the identification result of the analysis unit 104 instead of detecting the chewing motion by itself. Conversely, the analysis unit 104 may use the mastication motion detection result by the motion analysis unit 1022 .

The analysis unit 104 may calculate the amount of muscle activity of the masticatory muscles in one mastication motion as an index value indicating the magnitude of the load in the mastication. The analysis unit 104 may store the muscle activity amount as the index value in the storage unit 11 as the evaluation support information 112 .

The evaluation unit 103 compares the index value calculated by the analysis unit 104 with a predetermined threshold. The evaluation unit 103 may cause the storage unit 11 to store, as the evaluation support information 112, the difference between the index value and the threshold for the muscle activity amount of the masticatory muscle. The threshold may indicate the requested load amount. The required load amount means the amount of muscle activity required of the subject in order to obtain the effect of mastication training.

The evaluation unit 103 compares the index value regarding the amount of movement of the mouth with a threshold value (required range of motion), and also compares the index value regarding the amount of muscle activity of masticatory muscles with a threshold value (required load). . As a result of the comparison, the evaluation unit 103 may determine that the food is unsuitable as a mastication training food when both the index value relating to the degree of mouth movement and the index value relating to the amount of muscle activity are less than the threshold values. Alternatively, the evaluation unit 103 may determine that the food is unsuitable for mastication training when at least one of the index value relating to the degree of mouth movement and the index value relating to the amount of muscle activity is less than a threshold.

<Process flow>
The information processing method executed by the information processing apparatus 1 according to the second embodiment includes, in addition to the detection step, the evaluation step, and the output step described in the information processing method according to the first embodiment, An analysis step of analyzing the magnitude of the load applied to the masticatory muscles of the subject based on the signal indicating the activity of the masticatory muscles of the subject is included. Then, in the evaluation step, suitability is evaluated based on the magnitude of the load and the magnitude of the movement of the mouth.

FIG. 7 is a flow chart showing an example of the above-described information processing method executed by the information processing apparatus 1. As shown in FIG. In the illustrated example, S102 corresponds to the detection steps S12 to S15 of the first embodiment. S103 corresponds to an example of the analysis step described above. S104 to S107 correspond to an example of the evaluation steps described above. Also, S108 corresponds to an example of the output step described above.

In S<b>101 , the image acquisition unit 101 acquires an image of the subject's mastication state, and stores it in the storage unit 11 as an image 111 .

In S102, the detection unit 102 executes detection steps S12 to S15.

In S<b>103 , the analysis unit 104 analyzes the magnitude of the load applied to the muscle of mastication based on the muscle activity information measured by the myoelectric potential measurement device 4 . For example, the analysis unit 104 may calculate the amount of muscle activity of the masticatory muscles in one mastication motion as an index value indicating the magnitude of the load in the mastication.

In S104, the evaluation unit 103 compares the index values calculated in S102 and S103 with thresholds corresponding to the respective index values.

As an example, the evaluation unit 103
(1) comparing the index value Mox calculated in S15 with the required range of motion of the lateral movement of the mouth;
(2) comparing the index value Moy calculated in S15 with the required range of motion of the vertical movement of the mouth;
(3) The index value indicating the magnitude of the load on the masticatory muscle calculated in S103 is compared with the required load amount.

In S105, the evaluation unit 103 may evaluate the suitability of the prototype (object to be masticated) masticated by the subject as food for mastication training based on the difference obtained as a result of the comparison.

As an example, the evaluation unit 103 evaluates that the prototype is suitable for mastication training when the index value Mox, the index value Moy, and the index value indicating the magnitude of the load are all equal to or greater than the threshold value, and the process proceeds from NO in S105 to S106. may proceed. The above evaluation is based on the idea that ``all index values are above the threshold, ``the prototype allows the subject to exercise the masticatory muscles and open and close the mouth, which can be expected to have training effects.'' The evaluation unit 103 is configured to perform

On the other hand, if any index value is less than the threshold, the evaluation unit 103 may evaluate that the prototype is not suitable for mastication training, and proceed from YES in S105 to S107. Based on the idea that ``if any of the index values are less than the threshold, ``the prototype does not allow the subject to exercise the masticatory muscles or open and close the mouth, which can be expected to have training effects,''" The evaluation unit 103 is configured to perform evaluation.

In S<b>106 , the evaluation unit 103 may cause the storage unit 11 to store the evaluation result “prototype conforms” as the evaluation support information 112 .

In S<b>107 , the evaluation unit 103 may cause the storage unit 11 to store the evaluation result “prototype not conforming” as the evaluation support information 112 .

It should be noted that if the user does not need the determination result indicating conformity/nonconformity of the prototype, S106 and S107 can be omitted.

In S108, the output control unit 105 outputs the evaluation support information 112 generated by the detection unit 102, the analysis unit 104, and the evaluation unit 103. FIG. For example, the output control unit 105 causes the display device 2 to display the evaluation support information 112 .

As described above, according to the information processing method of the present disclosure, it is necessary to evaluate the suitability of a mastication object for mastication training by considering not only the magnitude of mouth movement but also the amount of activity of masticatory muscles. Evaluation support information can be obtained. By also taking into account the amount of masticatory muscle activity, it is possible to more accurately and quantitatively evaluate the suitability of the object to be masticated for mastication training. For example, it is preferable that the object to be masticated has an appropriate range of movements of various organs related to mastication (for example, opening and closing of the mouth) and an appropriate range of load applied to the muscles of mastication. It is possible to support the user to make an evaluation to do.

<Application example>
The information processing method and manufacturing method according to the present embodiment can more precisely evaluate products that are expected to have the effect of mastication training. Therefore, the information processing method and manufacturing method according to the present embodiment can be applied to the development of foods for mastication training for the general public, and are particularly preferably used in, for example, the following use cases.

For example, in medical institutions, rehabilitation facilities, and the like, it is assumed that patients with dysfunction are given masticatory function recovery training. In order to maximize the effect of functional recovery training, it is required that the subject patient be provided with food for functional recovery training that is optimal for the patient.

The patient is made to wear the myoelectric potential measuring device 4, eat the prototype, and the state of mastication is photographed by the photographing device 3. - 特許庁Then, the muscle activity information measured by the myopotential measuring device 4 and the moving image captured by the imaging device 3 are input to the information processing device 1 to obtain the evaluation support information 112 for the prototype for the patient. The manufacturer/seller can utilize the obtained evaluation support information 112 to find the direction of improvement and adjust the recipe of the prototype with high accuracy. Then, the manufacturer/marketer can manufacture and sell more suitable foods as foods that enhance the effect of mastication training for patients. Adjustments to recipes may be made by health care workers seeking to restore function to the patient. By storing recipes for each patient, it is possible to respond immediately to additional orders and to adjust recipes as the patient recovers masticatory function.

[About manufacturing method]
<Overview>
an adjustment step of adjusting the recipe of the object to be masticated so that the evaluation support information 112 output by the information processing method according to the first or second embodiment satisfies a predetermined condition; and a manufacturing step of manufacturing an object to be masticated by using the method of manufacturing is also included in the scope of the present invention. As a result, it is possible to easily manufacture a mastication object that allows the subject to open and close the mouth widely during mastication, that is, is suitable for mastication training.

The evaluation support information 112 may indicate the magnitude of the mouth movement detected in the detection step in the information processing method described above. In the adjustment step, if the magnitude of mouth movement indicated by the evaluation support information 112 is less than a predetermined threshold, the recipe may be adjusted so that the magnitude of mouth movement during mastication is increased. As a result, it is possible to easily improve the prototype, which was unable to make the subject open the mouth wide during mastication, so that it is more suitable for mastication training.

A recipe may include parameters that define the shape, size, and/or composition of the object to be chewed. In the adjusting step, the parameters may be adjusted such that the magnitude of mouth movement during mastication is greater. As a result, the prototype, which was unable to make the subject open their mouth wide during mastication, was changed in shape, size, or composition of materials, etc., to make the prototype more suitable for mastication training. can be easily improved.

an adjusting step of adjusting the recipe of the object to be masticated so that the evaluation support information 112 output by the information processing method according to the second embodiment satisfies a predetermined condition; and the evaluation support information 112 includes the magnitude of the mouth movement detected in the detection step (S102) and the magnitude of the load on the masticatory muscle analyzed in the analysis step (S103). In the adjustment step, when the magnitude of mouth movement indicated by the evaluation support information 112 is less than a predetermined threshold, the recipe is adjusted so that the magnitude of mouth movement during mastication becomes greater. A manufacturing method that adjusts a recipe to increase the load when the magnitude of the load indicated by the evaluation support information 112 is less than a predetermined threshold is also within the scope of the present invention. As a result, it is possible to easily manufacture a mastication object suitable for mastication training, which not only allows the subject to open and close the mouth widely during mastication, but also allows the subject to masticate forcefully using the muscles of mastication.

<Details>
The inventors considered that the degree of mouth movement during mastication is highly correlated with the effect of mastication training. Then, the magnitude of movement of the locomotory organs involved in mastication is measured when the subject chews the prototype, and a food is designed that can encourage the subject to move the locomotory organs at the target size. I came up with this. As a result, it is possible to produce a food for mastication training that can be expected to have the effect of mastication training.

If it is found from the evaluation support information 112 output by the information processing apparatus 1 that the prototype is insufficient in the size of the mouth movement in the vertical direction, the prototype is improved so that the mouth moves more in the vertical direction. goals are set. In this case, for example, the thickness of the prototype is increased, or the compounding ratio of materials and additives is adjusted to make it harder. Thus, after repeated trial and error, it is possible to design and produce an ideal food for mastication training.

If it is found from the evaluation support information 112 output by the information processing device 1 that the prototype has insufficient lateral movement of the mouth, the prototype is improved so that the mouth moves more laterally. goals are set. In this case, for example, the prototype may be flattened or elongated, or the compounding ratio of materials and additives may be adjusted to make it harder. Thus, after repeated trial and error, it is possible to design and produce an ideal food for mastication training.

If it is found from the evaluation support information 112 output by the information processing apparatus 1 that the prototype is insufficient in both the magnitude of the mouth movement in the vertical direction and the magnitude of the mouth movement in the lateral direction, the mouth is moved. A goal is set to improve the prototype so that it moves more vertically and horizontally. In this case, for example, the size of the prototype is increased, or the compounding ratio of materials and additives is adjusted to make it harder. Thus, after repeated trial and error, it is possible to design and produce an ideal food for mastication training.

〔effect〕
In addition, according to the configuration of the present disclosure, it is possible to train the masticatory power that may decline due to various factors such as aging and dysfunction, and to promote the development of the masticatory power of children in the process of growth, thereby improving the oral function. It can support maintenance and improvement. Chewing food, swallowing behavior, saliva secretion, etc. have a great influence on the brain and the whole body, and are factors that greatly affect mental and physical health and healthy life expectancy. Therefore, according to the configuration of the present disclosure, maintenance and improvement of oral functions are achieved, and it is expected to contribute to maintenance of mental and physical health, promotion of development, extension of healthy life expectancy, and the like. Ultimately, it contributes to the achievement of Goal 3 of the Sustainable Development Goals (SDGs), "Good Health and Well-Being for All."

[Example of realization by software]
The function of the information processing device 1 (hereinafter referred to as "device") is a program for causing a computer to function as the device, and the computer is used as each control block (especially each part included in the control unit 10) of the device. It can be realized by a program for functioning.

In this case, the apparatus comprises a computer having at least one control device (eg processor) and at least one storage device (eg memory) as hardware for executing the program. Each function described in each of the above embodiments is realized by executing the above program using the control device and the storage device.

The program may be recorded on one or more computer-readable recording media, not temporary. The recording medium may or may not be included in the device. In the latter case, the program may be supplied to the device via any transmission medium, wired or wireless.

Also, part or all of the functions of the above control blocks can be realized by logic circuits. For example, integrated circuits in which logic circuits functioning as the control blocks described above are formed are also included in the scope of the present invention. In addition, it is also possible to implement the functions of the control blocks described above by, for example, a quantum computer.

Further, each process described in each of the above embodiments may be executed by AI (Artificial Intelligence). In this case, the AI may operate on the control device, or may operate on another device (for example, an edge computer or a cloud server).

The present invention is not limited to the above-described embodiments, but can be modified in various ways within the scope of the claims, and can be obtained by appropriately combining technical means disclosed in different embodiments. is also included in the technical scope of the present invention.

1 information processing device 2 display device 3 imaging device 4 myoelectric potential measurement device 10 control unit 11 storage unit 101 image acquisition unit 102 detection unit 103 evaluation unit 104 analysis unit 105 output control unit 111 image 112 evaluation support information 1021 feature point detection unit 1022 Motion analysis part

Claims (13)

An information processing method executed by one or more information processing devices,
a detection step of detecting the magnitude of movement of the mouth of the subject during mastication based on images of the subject chewing the object to be masticated in chronological order;
an output step of outputting evaluation support information necessary for evaluating suitability of the object to be masticated for mastication training based on the magnitude of the detected movement of the mouth. 2. The method according to claim 1, wherein in said output step, at least one of an index value indicating the magnitude of the mouth movement detected in said detection step and visualization information visualizing said magnitude is output as said evaluation support information. Information processing method described. an evaluation step of evaluating suitability of the mastication object for mastication training based on the magnitude of mouth movement detected in the detection step;
3. The information processing method according to claim 1, wherein said outputting step outputs an evaluation result obtained by said evaluating step as said evaluation support information. In the detection step, based on the image, the magnitude of vertical and horizontal movements of the mouth of the subject during mastication is detected;
In the evaluation step, the longitudinal and lateral sizes of the mouth of the subject during mastication and the longitudinal and lateral sizes of the mouth during mastication required to obtain the effect of mastication training. 4. The information processing method according to claim 3, wherein the suitability is evaluated based on a difference from each threshold value indicating . In the detection step, based on the distance between predetermined feature points on the face of the subject whose position does not change due to mastication, detected from the image, the longitudinal direction and the lateral direction of the mouth during mastication of the subject are detected. 5. The information processing method according to claim 4, wherein the magnitude of directional movement is detected. In the evaluation step, an evaluation result of the vertical and horizontal movements of the mouth during mastication is used as an objective variable, and an image of the state of mastication taken in chronological order or a feature amount extracted from the image is described. As a variable, the suitability is evaluated by evaluating the vertical and horizontal movements of the mouth during mastication of the subject using an evaluation model that has learned the relationship between the objective variable and the explanatory variable. The information processing method according to claim 3, wherein an analysis step of analyzing the magnitude of the load applied to the masticatory muscles of the subject based on a signal indicating the activity of the masticatory muscles of the subject measured during mastication of the subject;
7. The information processing method according to any one of claims 3 to 6, wherein in said evaluation step, said suitability is evaluated based on the magnitude of said load and the magnitude of said movement. an adjustment step of adjusting the recipe of the object to be masticated so that the evaluation support information output by the information processing method according to any one of claims 1 to 7 satisfies a predetermined condition;
a manufacturing step of manufacturing the masticatory object based on the recipe adjusted in the adjusting step. the evaluation support information indicates the magnitude of the mouth movement detected in the detection step;
In the adjusting step, when the magnitude of the mouth movement indicated by the evaluation support information is less than a predetermined threshold, the recipe is adjusted such that the magnitude of the mouth movement during mastication is increased. Item 8. The manufacturing method according to item 8. the recipe includes parameters that define at least one of the shape, size, and composition of the chewable object;
10. The manufacturing method according to claim 9, wherein in said adjusting step, said parameter is adjusted such that the magnitude of mouth movement during mastication is increased. an adjustment step of adjusting the recipe of the mastication object so that the evaluation support information output by the information processing method according to claim 7 satisfies a predetermined condition;
a manufacturing step of manufacturing the masticatory object based on the recipe adjusted in the adjusting step;
The evaluation support information indicates the magnitude of the mouth movement detected in the detection step and the magnitude of the load applied to the masticatory muscles analyzed in the analysis step,
In the adjustment step,
adjusting the recipe to increase the magnitude of mouth movement during mastication when the magnitude of mouth movement indicated by the evaluation support information is less than a predetermined threshold;
A manufacturing method, wherein when the magnitude of the load indicated by the evaluation support information is less than a predetermined threshold, the recipe is adjusted so that the load becomes greater. a detection unit that detects the magnitude of movement of the mouth of the subject during mastication based on images of the subject chewing the object to be masticated in chronological order;
an output control unit that outputs evaluation support information necessary for evaluating suitability of the object to be masticated for mastication training based on the magnitude of the detected movement of the mouth. 13. A control program for causing a computer to function as the information processing apparatus according to claim 12, the control program for causing the computer to function as the detection section and the output control section.

Images