JP7090245B2 - Jaw evaluation system and jaw evaluation method - Google Patents

Description

One aspect of the present invention relates to a jaw evaluation system and a jaw evaluation method.

Techniques for evaluating the movement of a human jaw have been conventionally known. For example, Patent Document 1 describes a mastication detection device that detects a fluctuation amount of a detected portion and detects mastication. This device is a detection means that outputs a detection signal according to the fluctuation amount of the detected part in the head, and by comparing the value based on the detection signal with the threshold value, the detection section having a large fluctuation amount and the fluctuation amount can be compared. It is provided with a discriminating means for discriminating from a small detection section.

JP-A-2017-127511

Since the above-mentioned mastication detection device is a dedicated device provided with a detection means that outputs a detection signal according to the amount of fluctuation of the detected portion, it cannot be said that the movement of the jaw can be easily evaluated. Therefore, it is desired to easily evaluate the movement of a person's jaw.

In the jaw evaluation system according to one aspect of the present invention, an acquisition unit for acquiring image data obtained by capturing an image of a subject's face and at least one left feature point on the left side of the face in the image data are set in the image data. A setting unit that sets at least one right feature point on the right side of the face, and an evaluation unit that evaluates the movement of the subject's jaw based on the movement of at least one left feature point and the movement of at least one right feature point. And prepare.

The jaw evaluation method according to one aspect of the present invention is a jaw evaluation method executed by a computer system, in which an acquisition step of acquiring image data obtained by capturing an image of a subject's face and at least on the left side of the face in the image data. Based on the setting step of setting one left feature point and setting at least one right feature point on the right side of the face in the image data, and the movement of at least one left feature point and the movement of at least one right feature point. It includes an evaluation step to evaluate the movement of the subject's jaw.

In such an aspect, the movement of the subject's jaw is evaluated by analyzing the image data obtained by capturing the image of the subject's face. Since it is sufficient to prepare image data for evaluation, the movement of a person's jaw can be easily evaluated.

According to one aspect of the present invention, the movement of a human jaw can be easily evaluated.

It is a figure which shows the use example of the jaw evaluation system which concerns on embodiment. It is a figure which shows an example of the functional structure of the jaw evaluation system which concerns on embodiment. It is a flowchart which shows an example of the operation of the jaw evaluation system in an embodiment. It is a flowchart which shows an example of the process of measuring the movement of a face shown in FIG. It is a flowchart which shows an example of the process which evaluates the movement of a jaw shown in FIG. It is a flowchart which shows an example of the process of measuring the movement of a face shown in FIG. It is a figure which shows an example of a plurality of feature points set on a face in image data. It is a flowchart which shows an example of the process which evaluates the movement of a jaw shown in FIG. It is a figure which shows an example of the hardware composition of the computer used for the jaw evaluation system which concerns on embodiment.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. In the description of the drawings, the same or equivalent elements are designated by the same reference numerals, and duplicate description will be omitted.

The jaw evaluation system 10 according to the embodiment is a computer system that evaluates the movement of a human jaw. As an example, the jaw evaluation system 10 can evaluate the left-right movement of the jaw during chewing. Since both the left and right sides of a person's jaw do not always make the same movement, it may be useful to evaluate the left and right movements of the jaw. For example, the jaw evaluation system 10 can evaluate the degree of uneven chewing of a person. Unbalanced chewing is the habit of chewing food with only one tooth. The "degree of unbalanced mastication" is an index showing how strong the tendency of unbalanced mastication is. In this disclosure, a person to be evaluated for jaw movement is referred to as a "target person". "Evaluating jaw movement" means at least presenting information about jaw movement, including, for example, presenting data indicating jaw movement and presenting any determination result regarding jaw movement. It is a concept.

FIG. 1 is a diagram showing a usage example of the jaw evaluation system 10. The jaw evaluation system 10 acquires image data obtained by capturing an image of the face of the subject 80 who is chewing, and evaluates the movement of the jaw of the subject 80 by analyzing the image data. Specifically, the jaw evaluation system 10 sets a plurality of feature points on the face of the subject 80 in the image data, and evaluates the movement of the jaw based on the movement of the feature points. The evaluation results can be used for various purposes. For example, the evaluation result may be provided directly to the subject 80, or may be provided to an expert 90 such as a medical worker or a caregiver and then fed back from the expert 90 to the subject 80. good. When the image data is an image of a plurality of subjects 80, the jaw evaluation system 10 may evaluate the movement of the jaw for each subject 80.

FIG. 2 is a diagram showing an example of the functional configuration of the jaw evaluation system 10. The jaw evaluation system 10 includes an image acquisition unit 11, a feature point setting unit 12, an evaluation unit 13, and an output unit 14 as functional elements. The image acquisition unit 11 is a functional element that acquires image data obtained by capturing an image of the face of the subject 80 who is chewing. The image data is data indicating an image that can be visually recognized by being processed by a computer. The feature point setting unit 12 is a functional element for setting a plurality of feature points on the face of the target person 80 in the image data. The feature point is a virtual point set on the image data in order to specify an arbitrary part of the face of the target person 80. In the present disclosure, some feature points are used to identify jaw movements of subject 80. The evaluation unit 13 is a functional element that evaluates the movement of the jaw of the subject 80 based on the movement of the plurality of feature points. In one example, the evaluation unit 13 includes a measurement unit 131, a database 132, and a determination unit 133. The measuring unit 131 is a functional element that measures the movement of each feature point. The database 132 is a functional element that temporarily or permanently stores the measurement result. Judgment unit 133. It is a functional element that executes a determination regarding jaw movement based on the measurement result. The output unit 14 is a functional element that outputs the evaluation result by the evaluation unit 13.

As shown in FIG. 2, the image data of the face of the subject 80 is obtained by the image pickup apparatus 20. The image pickup device 20 is a device that captures an image of space and generates image data. The image pickup apparatus 20 may be constructed as a part of the jaw evaluation system 10 or may be constructed as a component different from the jaw evaluation system 10. The specific type of the image pickup apparatus 20 is not limited, and may be, for example, a camera or a computer including the camera. The computer provided with the camera may be a mobile terminal such as a high-performance mobile phone (smartphone), a mobile phone, a wearable terminal, or a laptop, or may be a stationary personal computer. The image pickup apparatus 20 has a function of being able to generate an image having a resolution capable of detecting the movement of the face of the subject 80. In one example, the image data is sent from the image pickup device 20 to the jaw evaluation system 10 via a communication network or a recording medium. Examples of communication networks include mobile communication networks, the Internet, and WAN (Wide Area Network). Examples of recording media include SD memory cards and USB memories. However, the communication network and the recording medium are not limited to these, and any method may be adopted.

An example of the operation of the jaw evaluation system 10 will be described with reference to FIGS. 3 to 5. FIG. 3 is a flowchart showing an example of a process for evaluating the movement of the jaw of the subject 80. FIG. 4 is a flowchart showing an example of the process of measuring the movement of the face. FIG. 5 is a flowchart showing an example of a process of evaluating the movement of the jaw based on the measurement result.

FIG. 3 shows the entire operation of the jaw evaluation system 10. In step S1, the image acquisition unit 11 acquires image data. The format of the image is not limited as long as the movement of the jaw of the subject 80 can be evaluated. For example, the image may be a moving image, a set of a plurality of still images obtained by continuous shooting, or a set of a plurality of frames of still images extracted from the moving image. The image acquisition unit 11 may extract a plurality of frames of still images from the moving image. The image acquisition unit 11 may acquire image data directly or indirectly from the image pickup apparatus 20.

In step S2, the feature point setting unit 12 sets a plurality of feature points on the face of the target person 80 in the image data, and the evaluation unit 13 measures the movement of the face using the feature points. In step S3, the evaluation unit 13 evaluates the jaw movement of the subject 80 based on the measurement result. Details of steps S2 and S3 will be described later.

In step S4, the output unit 14 outputs the evaluation result. The expression method of the evaluation result is not limited. For example, the evaluation result may be expressed at two or more levels, may be expressed by an arbitrary index, or may be expressed by using a graph or a graphic. The output method of the evaluation result is also not limited. For example, the output unit 14 may store the evaluation result in a predetermined database, send it to another computer such as a user terminal, or display it on a monitor. The user of the jaw evaluation system 10 is a person who uses the evaluation result output from the jaw evaluation system 10. The user is an arbitrary person, and may be, for example, an expert 90 such as a medical worker and a caregiver, the subject 80 himself, or the family of the subject 80. The user terminal is any kind of computer used by such a user. The user can confirm the tendency of the jaw movement of the subject 80 by referring to the evaluation result, and can further determine whether or not the subject 80 should be followed up or treated.

FIG. 4 shows one specific example of step S2. In step S201, the feature point setting unit 12 selects one frame to be processed from the image data. When the image data is a set of a plurality of still images, the feature point setting unit 12 may select one still image as one frame.

In step S202, the feature point setting unit 12 sets a plurality of feature points in the frame. Specifically, the feature point setting unit 12 sets one reference feature point, at least one left feature point, and at least one right feature point. The left feature point is a feature point set at an arbitrary place in the left half of the face of the subject 80. The right feature point is a feature point set at an arbitrary place in the right half of the face of the subject 80. The reference feature point is a feature point set for measuring the movement of the left feature point and the right feature point. The feature point setting unit 12 may set the feature point by using an arbitrary method. For example, the feature point setting unit 12 may access a learning model for extracting facial feature points, which is created in advance by machine learning, and set feature points by applying a frame to the model. .. The learning model may be constructed as a part of the jaw evaluation system 10 or as a component separate from the jaw evaluation system 10.

In step S203, the measuring unit 131 calculates the relative coordinates for each of the left feature point and the right feature point. These relative coordinates are coordinates obtained by assuming that the coordinates of the reference feature point are the origin (0,0) in the two-dimensional coordinate system obtained by assuming that the imaging range indicated by the image data is a plane.

In step S204, the measurement unit 131 stores the combination of the frame number and the relative coordinates of each feature point in the database 132 as a part of the time series data. The frame number is an identifier for uniquely identifying a frame in image data. The time-series data stored in the database 132 shows the movements of the left and right sides of the jaw of the subject 80 in a certain time width.

As shown in step S205, the jaw evaluation system 10 executes the processes of steps S201 to S204 for all frames (in one example, all still images) constituting the image data. By processing all the frames, time-series data for evaluating the jaw movement of the subject 80 is obtained.

FIG. 5 shows one specific example of step S3. In step S301, the determination unit 133 reads the time-series data obtained in step S2 from the database 132 and smoothes the time-series data. The determination unit 133 obtains a continuous change of each feature point by smoothing the history of relative coordinates for each of the left feature point and the right feature point. The history of relative coordinates refers to changes in relative coordinates over time.

In step S302, the determination unit 133 calculates the number of inflection points for each of the left feature point and the right feature point based on the smoothed time series data. This inflection corresponds to the timing at which the direction of reciprocating jaw movement changes.

In step S303, the determination unit 133 calculates the number of chews for each of the left and right faces of the subject 80 based on the number of inflection points. The determination unit 133 obtains the number of chews on the left side of the face by dividing the number of inflection points of the left feature point by 2, and the number of chews on the right side of the face by dividing the number of inflection points of the right feature point by 2. Ask for.

In step S304, the determination unit 133 evaluates the movement of the jaw of the subject 80 based on the number of chews on the left and right. For example, the determination unit 133 may calculate the degree of uneven chewing of the subject 80 based on the number of chewing on the left and right. The method for determining the degree of uneven chewing is not limited. For example, the determination unit 133 may calculate the ratio of the number of chews on the left and the number of chews on the right as the degree of uneven chewing. As an example, if the number of chews on the left is 15 and the number of chews on the right is 9, the determination unit 133 calculates the degree of uneven chewing as 15/9 ≈ 1.67 or 9/15 = 0.6. May be good. The determination unit 133 may set the calculated degree of uneven chewing as it is as an evaluation result of the degree of uneven chewing. Alternatively, the determination unit 133 compares the calculated degree of unbalanced chewing with a given standard to "no unbalanced chewing", "degree of unbalanced chewing = low", "degree of unbalanced chewing = medium", and "degree of unbalanced chewing =". You may set a level such as "high".

A more specific example of steps S2 and S3 will be shown with reference to FIGS. 6 to 8. FIG. 6 is a flowchart showing another example of the process of measuring the movement of the face. FIG. 7 is a diagram showing an example of a plurality of feature points set on the face. FIG. 8 is a flowchart showing another example of the process of evaluating the movement of the jaw based on the measurement result.

FIG. 6 shows an example of step S2. It can be said that steps S211 to S216 shown in FIG. 6 are more specific examples of steps S201 to S205 shown in FIG.

In step S211 the feature point setting unit 12 selects one frame to be processed from the image data. This process is the same as in step S201.

In step S212, the feature point setting unit 12 sets one reference feature point on the upper jaw, sets three left feature points on the left cheek, and sets three right feature points on the right cheek. As shown in FIG. 7, the feature point setting unit 12 can recognize a large number of feature points 40 for the face reflected in the frame 30. In this example, the feature point setting unit 12 sets one feature point corresponding to the upper jaw as a reference feature point 41 from among these feature points 40, and three feature points corresponding to the left cheek are set as the left feature point 42. And the three feature points corresponding to the right cheek are set as the right feature point 43. For example, the feature point setting unit 12 sets a reference feature point 41 at or near the center of the upper jaw and sets a left feature point 42 on or near the contour of the left cheek, and on or near the contour of the right cheek. The right feature point 43 is set near the contour.

In step S213, the measuring unit 131 sets the coordinates of the reference feature point as the origin (0,0), and then calculates the relative coordinates for each of the three left feature points and the three right feature points.

In step S214, the measuring unit 131 calculates the average coordinates of the left cheek and the right cheek based on their relative coordinates. The measuring unit 131 obtains the average of the relative coordinates of the three left feature points as the average coordinates of the left cheek, and the average of the relative coordinates of the three right feature points as the average coordinates of the right cheek.

In step S215, the measurement unit 131 stores the combination of the frame number and the left and right average coordinates in the database 132 as a part of the time series data.

As shown in step S216, the jaw evaluation system 10 executes the processes of steps S211 to S215 for all the frames (in one example, all the still images) constituting the image data. By processing all the frames, time-series data for evaluating the jaw movement of the subject 80 is obtained.

After that, the process proceeds to step S3 shown in FIG. It can be said that steps S311 to S314 shown in FIG. 8 are more specific examples of steps S301 to S304 shown in FIG.

In step S311, the determination unit 133 reads the time-series data obtained in step S2 from the database 132 and smoothes the time-series data. The determination unit 133 smoothes the coordinate history for each of the average coordinates of the left cheek and the right cheek, and obtains continuous changes of the left and right feature points.

In step S312, the determination unit 133 calculates the number of inflection points for each of the left cheek corresponding to the left feature point and the right cheek corresponding to the right feature point based on the smoothed time series data.

In step S313, the determination unit 133 calculates the number of chews for each of the left cheek and the right cheek based on the number of inflection points. The determination unit 133 obtains the number of chews on the left side of the face by dividing the number of inflection points on the left cheek by 2, and obtains the number of chews on the right side of the face by dividing the number of inflection points on the right cheek by 2. ..

In step S314, the determination unit 133 evaluates the movement of the jaw of the subject 80 based on the number of chews on the left and right. This process is the same as in step S304. Therefore, the determination unit 133 may calculate the degree of uneven chewing of the subject 80 based on the number of chewing on the left and right.

The jaw evaluation system 10 may be constructed by any method. For example, the jaw evaluation system 10 may be constructed as a client-server type system. Alternatively, a single device or computer may function as the jaw evaluation system 10, and for example, a mobile terminal equipped with a camera may function as the jaw evaluation system 10.

It is not essential to set a reference feature point. The jaw evaluation system may measure the movement of the left and right feature points without using the reference feature points.

The method of calculating the number of chews is not limited, and for example, the jaw evaluation system may calculate the number of chews without obtaining an inflection point. Calculation of the number of chews is not essential, and an index different from the number of chews may be used to evaluate the movement of the jaw. For example, the jaw evaluation system may calculate the movement distances of the left feature point and the right feature point, and evaluate the movement of the jaw (for example, the degree of uneven chewing) by comparing the movement distances of the left and right sides of the face. ..

The block diagram used in the description of the above embodiment shows a block of functional units. These functional blocks (components) are realized by any combination of at least one of hardware and software. Further, the method of realizing each functional block is not particularly limited. That is, each functional block may be realized using one physically or logically coupled device, or two or more physically or logically separated devices can be directly or indirectly (eg, for example). , Wired, wireless, etc.) and may be realized using these plurality of devices. The functional block may be realized by combining the software with the one device or the plurality of devices.

Functions include judgment, decision, judgment, calculation, calculation, processing, derivation, investigation, search, confirmation, reception, transmission, output, access, solution, selection, selection, establishment, comparison, assumption, expectation, and assumption. Broadcasting, notifying, communicating, forwarding, configuring, reconfiguring, allocating, mapping, assigning, etc., but limited to these I can't. For example, a functional block (component) that makes transmission function is called a transmitting unit or a transmitter. In each case, as described above, the realization method is not particularly limited.

For example, the jaw evaluation system in one embodiment of the present disclosure may function as a computer that processes the wireless communication methods of the present disclosure. FIG. 9 is a diagram showing an example of a hardware configuration of a computer 100 that functions as a jaw evaluation system 10. The computer 100 may physically include a processor 1001, a memory 1002, a storage 1003, a communication device 1004, an input device 1005, an output device 1006, a bus 1007, and the like.

In the following description, the word "device" can be read as a circuit, a device, a unit, or the like. The hardware configuration of the jaw evaluation system 10 may be configured to include one or more of the devices shown in the figure, or may be configured to include some of the devices.

For each function in the jaw evaluation system 10, by loading predetermined software (program) on hardware such as the processor 1001 and the memory 1002, the processor 1001 performs an operation, controls communication by the communication device 1004, and a memory. It is realized by controlling at least one of reading and writing of data in 1002 and storage 1003.

Processor 1001 operates, for example, an operating system to control the entire computer. The processor 1001 may be configured by a central processing unit (CPU) including an interface with peripheral devices, a control device, an arithmetic unit, a register, and the like.

Further, the processor 1001 reads a program (program code), a software module, data, and the like from at least one of the storage 1003 and the communication device 1004 into the memory 1002, and executes various processes according to these. As the program, a program that causes a computer to execute at least a part of the operations described in the above-described embodiment is used. For example, each functional element of the jaw evaluation system 10 may be realized by a control program stored in the memory 1002 and operating in the processor 1001. Although it has been described that the various processes described above are executed by one processor 1001, they may be executed simultaneously or sequentially by two or more processors 1001. Processor 1001 may be mounted by one or more chips. The program may be transmitted from the network via a telecommunication line.

The memory 1002 is a computer-readable recording medium, and is composed of at least one such as a ROM (Read Only Memory), an EPROM (Erasable Programmable ROM), an EEPROM (Electrically Erasable Programmable ROM), and a RAM (Random Access Memory). May be done. The memory 1002 may be referred to as a register, a cache, a main memory (main storage device), or the like. The memory 1002 can store a program (program code), a software module, or the like that can be executed to implement the wireless communication method according to the embodiment of the present disclosure.

The storage 1003 is a computer-readable recording medium, for example, an optical disk such as a CD-ROM (Compact Disc ROM), a hard disk drive, a flexible disk, an optical magnetic disk (for example, a compact disk, a digital versatile disk, a Blu-ray). It may consist of at least one (registered trademark) disk), smart card, flash memory (eg, card, stick, key drive), floppy (registered trademark) disk, magnetic strip, and the like. The storage 1003 may be referred to as an auxiliary storage device. The storage medium described above may be, for example, a database, server or other suitable medium containing at least one of the memory 1002 and the storage 1003.

The communication device 1004 is hardware (transmission / reception device) for communicating between computers via at least one of a wired network and a wireless network, and is also referred to as, for example, a network device, a network controller, a network card, a communication module, or the like. The communication device 1004 includes, for example, a high frequency switch, a duplexer, a filter, a frequency synthesizer, and the like in order to realize at least one of FDD (Frequency Division Duplex) and Time Division Duplex (TDD). It may be composed of.

The input device 1005 is an input device (for example, a keyboard, a mouse, a microphone, a switch, a button, a sensor, etc.) that receives an input from the outside. The output device 1006 is an output device (for example, a display, a speaker, an LED lamp, etc.) that outputs to the outside. The input device 1005 and the output device 1006 may have an integrated configuration (for example, a touch panel).

Further, each device such as the processor 1001 and the memory 1002 is connected by a bus 1007 for communicating information. The bus 1007 may be configured by using a single bus, or may be configured by using a different bus for each device.

Further, the computer 100 includes hardware such as a microprocessor, a digital signal processor (DSP: Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), a PLD (Programmable Logic Device), and an FPGA (Field Programmable Gate Array). The hardware may implement some or all of each functional block. For example, processor 1001 may be implemented using at least one of these hardware.

As described above, in the jaw evaluation system according to one aspect of the present invention, an acquisition unit for acquiring image data obtained by capturing an image of a subject's face and at least one left feature point are set on the left side of the face in the image data. Then, based on the setting unit that sets at least one right feature point on the right side of the face in the image data, and the movement of at least one left feature point and the movement of at least one right feature point, the subject's jaw It is equipped with an evaluation unit that evaluates movement.

The jaw evaluation method according to one aspect of the present invention is a jaw evaluation method executed by a computer system, in which an acquisition step of acquiring image data obtained by capturing an image of a subject's face and at least on the left side of the face in the image data. Based on the setting step of setting one left feature point and setting at least one right feature point on the right side of the face in the image data, and the movement of at least one left feature point and the movement of at least one right feature point. It includes an evaluation step to evaluate the movement of the subject's jaw.

In such an aspect, the movement of the subject's jaw is evaluated by analyzing the image data obtained by capturing the image of the subject's face. Since it is sufficient to prepare image data for evaluation, the movement of a person's jaw can be easily evaluated. Further, since feature points are provided on both the left and right sides of the face and the movements of these feature points are referred to, it becomes possible to evaluate the movement of the jaw in detail.

In the jaw evaluation system for other aspects, the evaluation unit calculates the number of left chews of the subject based on the movement of at least one left feature point, and the subject's movement is based on the movement of at least one right feature point. The number of chews on the right may be calculated and the movement of the jaw may be evaluated based on the number of chews on the left and the number of chews on the right. By considering the number of chews on the left and right, the movement of the jaw can be objectively and easily evaluated.

In the jaw evaluation system relating to the other aspect, the evaluation unit generates time-series data showing the change in the coordinates of at least one left feature point and the change in the coordinates of at least one right feature point, and is based on the time-series data. Then, the number of left inflections corresponding to at least one left feature point and the number of right inflections corresponding to at least one right feature point are calculated, and the left is based on the number of left inflections. The number of chews may be calculated, and the number of chews on the right may be calculated based on the number of right inflection points. By finding the inflection point from the time-series data showing the change in the coordinates of the feature point, the reciprocating motion of the jaw can be captured, and as a result, the number of chews can be found more accurately.

In the jaw evaluation system relating to other aspects, the setting unit sets a plurality of left feature points and a plurality of right feature points, and the evaluation unit changes the average coordinates of the plurality of left feature points and a plurality of right feature points. The number of left inflection points and the number of right inflection points may be calculated from the time-series data indicating the change in the average coordinates of. By finding the average coordinates of multiple feature points on each of the left and right sides of the face and finding the inflection points based on the changes in the average coordinates, the reciprocating motion of the jaw can be accurately captured, and therefore the number of chews can be determined. It will be possible to obtain more accurately.

In the jaw evaluation system relating to the other aspect, the setting unit further sets the reference feature point, and the evaluation unit sets the reference feature point as the origin of the coordinate system to change the relative coordinates of at least one left feature point. And the relative coordinates of at least one right feature point may be calculated to generate time-series data indicating changes in these relative coordinates. By introducing a reference feature point and expressing the positions of the left feature point and the right feature point in relative coordinates from the reference feature point, it is easier to change the positions of the left feature point and the right feature point over time. You can ask.

In the jaw evaluation system relating to other aspects, the setting unit sets a reference feature point at a position corresponding to the upper jaw of the subject, and sets at least one left feature point at a position corresponding to the left cheek of the subject. At least one right feature point may be set at a position corresponding to the subject's right cheek. The upper jaw hardly moves during mastication, while the left and right cheeks move significantly during mastication. Therefore, by setting reference points at these points, the movement of the jaw can be captured more accurately.

In the jaw evaluation system according to other aspects, the evaluation of jaw movement may include assessing the degree of uneven chewing of the subject. In this case, the degree of uneven chewing of the subject can be evaluated as long as the image data is prepared.

In the jaw evaluation system according to the other aspect, the evaluation unit may calculate the ratio of the number of chews on the left side of the subject and the number of times of chewing on the right side of the subject as the degree of uneven chewing. By calculating the ratio of the number of chewing times on the left and right, the degree of uneven chewing can be shown objectively and in an easy-to-understand manner.

Although the present disclosure has been described in detail above, it is clear to those skilled in the art that the present disclosure is not limited to the embodiments described in the present disclosure. The present disclosure may be implemented as amendments and modifications without departing from the spirit and scope of the present disclosure as determined by the description of the scope of claims. Therefore, the description of this disclosure is for purposes of illustration and does not have any limiting meaning to this disclosure.

The notification of information is not limited to the embodiments / embodiments described in the present disclosure, and may be performed by other methods. For example, information notification includes physical layer signaling (eg, DCI (Downlink Control Information), UCI (Uplink Control Information)), higher layer signaling (eg, RRC (Radio Resource Control) signaling, MAC (Medium Access Control) signaling, etc. It may be carried out by broadcast information (MIB (Master Information Block), SIB (System Information Block)), other signals, or a combination thereof. Further, the RRC signaling may be referred to as an RRC message, and may be, for example, an RRC Connection Setup message, an RRC Connection Reconfiguration message, or the like.

Each aspect / embodiment described in the present disclosure includes LTE (Long Term Evolution), LTE-A (LTE-Advanced), SUPER 3G, IMT-Advanced, 4G (4th generation mobile communication system), and 5G (5th generation mobile communication). system), FRA (Future Radio Access), NR (new Radio), W-CDMA (registered trademark), GSM (registered trademark), CDMA2000, UMB (Ultra Mobile Broadband), IEEE 802.11 (Wi-Fi (registered trademark)) )), LTE 802.16 (WiMAX®), IEEE 802.20, UWB (Ultra-WideBand), Bluetooth®, and other systems that utilize and extend based on these. It may be applied to at least one of the next generation systems. Further, a plurality of systems may be applied in combination (for example, a combination of at least one of LTE and LTE-A and 5G).

The order of the processing procedures, sequences, flowcharts, etc. of each aspect / embodiment described in the present disclosure may be changed as long as there is no contradiction. For example, the methods described in the present disclosure present elements of various steps using exemplary order, and are not limited to the particular order presented.

In some cases, the specific operation performed by the base station in the present disclosure may be performed by its upper node. In a network consisting of one or more network nodes having a base station, various operations performed for communication with the terminal are the base station and other network nodes other than the base station (eg, MME or). It is clear that it can be done by at least one of (but not limited to, S-GW, etc.). Although the case where there is one network node other than the base station is illustrated above, it may be a combination of a plurality of other network nodes (for example, MME and S-GW).

Information and the like can be output from the upper layer (or lower layer) to the lower layer (or upper layer). Input / output may be performed via a plurality of network nodes.

The input / output information and the like may be stored in a specific place (for example, a memory), or may be managed by using a management table. Information to be input / output may be overwritten, updated, or added. The output information and the like may be deleted. The input information or the like may be transmitted to another device.

The determination may be made by a value represented by 1 bit (0 or 1), by a boolean value (Boolean: true or false), or by comparing numerical values (for example, a predetermined value). It may be done by comparison with the value).

Each aspect / embodiment described in the present disclosure may be used alone, in combination, or may be switched and used according to the execution. Further, the notification of predetermined information (for example, the notification of "being X") is not limited to the explicit one, but is performed implicitly (for example, the notification of the predetermined information is not performed). May be good.

Software, whether called software, firmware, middleware, microcode, hardware description language, or other names, instructions, instruction sets, codes, code segments, program codes, programs, subprograms, software modules. , Applications, software applications, software packages, routines, subroutines, objects, executable files, execution threads, procedures, features, etc. should be broadly interpreted.

Further, software, instructions, information and the like may be transmitted and received via a transmission medium. For example, the software may use at least one of wired technology (coaxial cable, fiber optic cable, twist pair, Digital Subscriber Line (DSL), etc.) and wireless technology (infrared, microwave, etc.) to create a website. When transmitted from a server or other remote source, at least one of these wired and wireless technologies is included within the definition of transmission medium.

The information, signals, etc. described in the present disclosure may be represented using any of a variety of different techniques. For example, data, instructions, commands, information, signals, bits, symbols, chips, etc. that may be referred to throughout the above description are voltages, currents, electromagnetic waves, magnetic fields or magnetic particles, light fields or photons, or any of these. It may be represented by a combination of.

The terms described in the present disclosure and the terms necessary for understanding the present disclosure may be replaced with terms having the same or similar meanings. For example, at least one of a channel and a symbol may be a signal (signaling). Also, the signal may be a message. Further, the component carrier (CC: Component Carrier) may be referred to as a carrier frequency, a cell, a frequency carrier, or the like.

The terms "system" and "network" used in this disclosure are used interchangeably.

Further, the information, parameters, etc. described in the present disclosure may be expressed using an absolute value, a relative value from a predetermined value, or another corresponding information. It may be represented. For example, the radio resource may be indexed.

The names used for the parameters mentioned above are not limited in any respect. Further, mathematical formulas and the like using these parameters may differ from those expressly disclosed in this disclosure. Since the various channels (eg, PUCCH, PDCCH, etc.) and information elements can be identified by any suitable name, the various names assigned to these various channels and information elements are in any respect limited names. is not.

In the present disclosure, "Base Station (BS)", "Wireless Base Station", "Fixed Station", "NodeB", "eNodeB (eNB)", "gNodeB (gNB)", " "Access point", "transmission point", "reception point", "transmission / reception point", "cell", "sector", "cell group", "cell group" Terms such as "carrier" and "component carrier" may be used interchangeably. Base stations are sometimes referred to by terms such as macrocells, small cells, femtocells, and picocells.

The base station can accommodate one or more (eg, three) cells. When a base station accommodates multiple cells, the entire base station coverage area can be divided into multiple smaller areas, each smaller area being a base station subsystem (eg, a small indoor base station (RRH:)). Communication services can also be provided by (Remote Radio Head). The term "cell" or "sector" is used in this coverage as part or all of the coverage area of at least one of the base stations and base station subsystems that provide communication services. Point to.

In the present disclosure, terms such as "mobile station (MS)", "user terminal", "user equipment (UE)", and "terminal" may be used interchangeably. ..

Mobile stations can be used by those skilled in the art as subscriber stations, mobile units, subscriber units, wireless units, remote units, mobile devices, wireless devices, wireless communication devices, remote devices, mobile subscriber stations, access terminals, mobile terminals, wireless. It may also be referred to as a terminal, remote terminal, handset, user agent, mobile client, client, or some other suitable term.

At least one of a base station and a mobile station may be referred to as a transmitting device, a receiving device, a communication device, or the like. At least one of the base station and the mobile station may be a device mounted on the mobile body, a mobile body itself, or the like. The moving body may be a vehicle (eg, car, airplane, etc.), an unmanned moving body (eg, drone, self-driving car, etc.), or a robot (manned or unmanned). ) May be. It should be noted that at least one of the base station and the mobile station includes a device that does not necessarily move during communication operation. For example, at least one of a base station and a mobile station may be an IoT (Internet of Things) device such as a sensor.

Further, the base station in the present disclosure may be read by the user terminal. For example, the communication between the base station and the user terminal is replaced with the communication between a plurality of user terminals (for example, it may be called D2D (Device-to-Device), V2X (Vehicle-to-Everything), etc.). Each aspect / embodiment of the present disclosure may be applied to the configuration. In this case, the user terminal may have the function of the base station. Further, words such as "up" and "down" may be read as words corresponding to communication between terminals (for example, "side"). For example, the upstream channel, the downstream channel, and the like may be read as a side channel.

Similarly, the user terminal in the present disclosure may be read as a base station. In this case, the base station may have the functions of the user terminal.

The terms "determining" and "determining" used in this disclosure may include a wide variety of actions. "Judgment" and "decision" are, for example, judgment, calculation, computing, processing, deriving, investigating, looking up, search, inquiry. It may include (eg, searching in a table, database or another data structure), ascertaining as "judgment" or "decision". Also, "judgment" and "decision" are receiving (for example, receiving information), transmitting (for example, transmitting information), input (input), output (output), and access. It may include (for example, accessing data in memory) to be regarded as "judgment" or "decision". In addition, "judgment" and "decision" are considered to be "judgment" and "decision" when the things such as solving, selecting, choosing, establishing, and comparing are regarded as "judgment" and "decision". Can include. That is, "judgment" and "decision" may include considering some action as "judgment" and "decision". Further, "judgment (decision)" may be read as "assuming", "expecting", "considering" and the like.

The terms "connected", "coupled", or any variation thereof, mean any direct or indirect connection or connection between two or more elements and each other. It can include the presence of one or more intermediate elements between two "connected" or "combined" elements. The connection or connection between the elements may be physical, logical, or a combination thereof. For example, "connection" may be read as "access". As used in the present disclosure, the two elements use at least one of one or more wires, cables and printed electrical connections, and as some non-limiting and non-comprehensive examples, the radio frequency domain. Can be considered to be "connected" or "coupled" to each other using electromagnetic energy having wavelengths in the microwave and light (both visible and invisible) regions.

The phrase "based on" as used in this disclosure does not mean "based on" unless otherwise stated. In other words, the statement "based on" means both "based only" and "at least based on".

Any reference to elements using designations such as "first" and "second" as used in this disclosure does not generally limit the quantity or order of those elements. These designations can be used in the present disclosure as a convenient way to distinguish between two or more elements. Therefore, references to the first and second elements do not mean that only two elements can be adopted, or that the first element must somehow precede the second element.

When "include", "including" and variations thereof are used in the present disclosure, these terms are as inclusive as the term "comprising". Is intended. Moreover, the term "or" used in the present disclosure is intended not to be an exclusive OR.

In the present disclosure, if articles are added by translation, for example a, an and the in English, the disclosure may include the plural nouns following these articles.

In the present disclosure, the term "A and B are different" may mean "A and B are different from each other". The term may mean that "A and B are different from C". Terms such as "separate" and "combined" may be interpreted in the same way as "different".

10 ... jaw evaluation system, 11 ... image acquisition unit, 12 ... feature point setting unit, 13 ... evaluation unit, 131 ... measurement unit, 132 ... database, 133 ... judgment unit, 14 ... output unit, 20 ... image pickup device, 30 ... Frame (image data), 41 ... reference feature point, 42 ... left feature point, 43 ... right feature point, 80 ... subject.

Claims (7)

An acquisition unit that acquires image data of the subject's face,
A setting unit for setting at least one left feature point on the left side of the face in the image data and setting at least one right feature point on the right side of the face in the image data.
An evaluation unit for evaluating the movement of the jaw of the subject based on the movement of the at least one left feature point and the movement of the at least one right feature point is provided .
The evaluation unit
Time-series data showing the change in the coordinates of the at least one left feature point and the change in the coordinates of the at least one right feature point is generated.
Based on the time series data, the number of left inflection points corresponding to the at least one left feature point and the number of right inflection points corresponding to the at least one right feature point are calculated.
The number of chews on the left is calculated based on the number of left inflection points.
The number of chews on the right is calculated based on the number of inflection points on the right.
The movement of the jaw is evaluated based on the number of chews on the left and the number of chews on the right.
Jaw rating system. The setting unit sets a plurality of the left feature points and a plurality of the right feature points.
From the time-series data showing the change in the average coordinates of the plurality of left feature points and the change in the average coordinates of the plurality of right feature points, the evaluation unit determines the number of the left inflection points and the right inflection points. Calculate the number of
The jaw evaluation system according to claim 1 . The setting unit further sets the reference feature point,
The evaluation unit sets the reference feature point as the origin of the coordinate system, calculates the change in the relative coordinates of the at least one left feature point and the relative coordinates of the at least one right feature point, and makes them relative to each other. Generate the time series data showing the change of coordinates,
The jaw evaluation system according to claim 1 or 2 . The setting unit
The reference feature point is set at a position corresponding to the upper jaw of the subject, and the reference feature point is set.
At least one left feature point is set at a position corresponding to the left cheek of the subject.
At least one right feature point is set at a position corresponding to the right cheek of the subject.
The jaw evaluation system according to claim 3 . The evaluation of the jaw movement includes evaluating the degree of uneven chewing of the subject.
The jaw evaluation system according to any one of claims 1 to 4 . The evaluation unit calculates the ratio of the number of chews on the left side of the subject and the number of chews on the right side of the subject as the degree of uneven chewing.
The jaw evaluation system according to claim 5 . A jaw evaluation method performed by a computer system,
The acquisition step to acquire the image data of the subject's face,
A setting step of setting at least one left feature point on the left side of the face in the image data and setting at least one right feature point on the right side of the face in the image data.
It comprises an evaluation step of evaluating the movement of the subject's jaw based on the movement of the at least one left feature point and the movement of the at least one right feature point.
In the evaluation step,
Time-series data showing the change in the coordinates of the at least one left feature point and the change in the coordinates of the at least one right feature point is generated.
Based on the time series data, the number of left inflection points corresponding to the at least one left feature point and the number of right inflection points corresponding to the at least one right feature point are calculated.
The number of chews on the left is calculated based on the number of left inflection points.
The number of chews on the right is calculated based on the number of inflection points on the right.
The movement of the jaw is evaluated based on the number of chews on the left and the number of chews on the right.
Jaw evaluation method.

Images