TWI695699B - Unilateral chewing monitoring apparatus and method thereof - Google Patents

Abstract

The present invention provides a unilateral chewing monitoring apparatus and a method thereof, wherein the unilateral chewing monitoring apparatus comprises: a first module for acquiring continuous left image frames generated by a left masticatory muscle for a period of time; a second module for acquiring continuous right image frames generated by the right masticatory muscle in the time period; a processing module for receiving the left image frame of the first module and the right image frame of the second module. The processing module obtains a left side change value of the left image frame according to the continuous left image frames, and obtains a right side change value of the right image frame according to the continuous right image frames, and the processing module at least determines whether it is an unilateral chewing in the time period according to the result of comparison between the change value of the left image frame and the change value of the right image frame. The present invention can determine whether to chew on one side. Thus, it is possible to automatically monitor whether unilateral chewing is performed.

Description

Unilateral chewing monitoring equipment and method

The invention relates to the field of medical monitoring equipment, in particular to a unilateral chewing monitoring equipment and method thereof.

When eating food, the masticatory muscles on the left and right sides should be chewed in a balanced manner, so that the gums on both sides can be functionally stimulated evenly, which is conducive to the normal development of teeth, jaws and corresponding muscles. If you chew unilaterally for a long time, it will cause muscle development on one side and muscle atrophy on the other, and the left and right asymmetry of the face will lead to developmental deformity.

On the side that has not chewed for a long time, due to lack of functional stimulation, a large amount of food debris and tartar are deposited around the teeth, causing changes in the periodontal supporting bone, bleeding gums, bad breath and other symptoms.

Long-term stress on one side, unbalanced left and right strength, and easy to induce temporomandibular joint disease, the most common manifestations are the presence of snapping sounds in the joints, joint pain, or restricted mouth opening. Therefore, those with unilateral chewing habits should correct it in time.

In view of the defects in the prior art, the object of the present invention is to provide a unilateral chewing monitoring device and method thereof, which overcome the difficulties in the prior art and can determine whether it is unilateral chewing. Thus, it is possible to automatically monitor whether unilateral chewing is performed.

According to an aspect of the present invention, there is provided a unilateral chewing monitoring device, including: a first module, the first module acquiring a continuous left image frame generated by a left chewing muscle during a period of time; a second module Group, the second module obtains a continuous right image frame generated by the right masticatory muscle during the time period; a processor, the processor receives the left image frame and all the image frames of the first module The right image frame of the second module; the processor obtains the left change value of a left image frame from the continuous left image frame, and the right side of a right image frame from the continuous right image frame Change value; the processor determines whether the time period is one-sided chewing according to a result of comparison between a change value of the left image frame and a change value of the right image frame.

Preferably, the processor obtains the difference between the change value of the left image frame and the change value of the right image frame according to the difference between the change value and the change difference threshold To determine whether it is one-sided chewing within the time period.

Preferably, the processor acquires the chewing duration of one side chewing in multiple time periods, obtains the cumulative duration of left chewing and the cumulative duration of right chewing, and obtains the cumulative duration of left chewing and the cumulative time of chewing on the right The absolute value of the long difference, and at least according to the absolute value of the difference and the difference threshold, determine whether the multiple time periods are one-sided chewing.

Preferably, the first module, the second module, and the processor are disposed on a wearable device.

Preferably, the processor obtains the change value of the left or right image frame as the square root of the sum of squared differences of the RGB values of the pixel points of the same position of two consecutive image frames.

Preferably, the first module, the second module, and the processor are disposed in a pair of smart glasses, and the first module is a first camera disposed under the left lens of the smart glasses, the first A camera focuses on the left chewing muscle; the second module is a second camera disposed under the right lens of the smart glasses, and the second camera focuses on the right chewing muscle.

Preferably, it further comprises: a prompter, which is used to perform a unilateral chewing prompt according to the result determined by the processor as unilateral chewing.

Preferably, the reminder is provided on a wearable device or a mobile terminal.

According to another aspect of the present invention, there is also provided a unilateral chewing monitoring method, comprising the steps of: acquiring a continuous left image frame generated by a left masticatory muscle within a period of time, and obtaining a left image according to the continuous left image frame The change value of the frame; and obtain the continuous right image frame generated by the right masticatory muscle during the time period, and obtain the change value of a right image frame according to the continuous right image frame; at least according to the left image frame The result of the comparison of the change value of and the change value of the right image frame determines whether the time period is one-sided chewing.

Preferably, the change value of the left or right image frame is the square root of the sum of squared differences of the RGB values of the pixel points of the same position of two consecutive image frames, and n is the number of image element points of the image frame.

Preferably, the step of determining whether the time period is one-sided chewing according to the result of the comparison includes: obtaining a change value of both the change value of the left image frame and the change value of the right image frame The difference; according to the difference between the change value and the change difference threshold, determine whether the time period is one-sided chewing.

Preferably, the method further includes the following steps: acquiring the length of chewing of one side chewing in multiple time periods, obtaining the cumulative length of chewing on the left side and the cumulative length of chewing on the right side; acquiring the cumulative length of left side chewing and the cumulative amount of chewing on the right side The absolute value of the difference in duration, and at least according to the absolute value of the difference and the difference threshold, determine whether the multiple time periods are one-sided chewing.

In view of this, the unilateral chewing monitoring device and method of the present invention can determine whether it is unilateral chewing, so that it can automatically monitor whether unilateral chewing is performed.

In order to have a better understanding of the above and other aspects of the present invention, the following examples are specifically described in conjunction with the accompanying drawings as follows:

8: Smart glasses

81: left lens

82: right lens

83: left chewing muscle

84: right chewing muscle

110: First collector (first camera)

120: Second collector (second camera)

130: processor

200: monitoring equipment

240: prompter

400: monitoring equipment

410: The first module

420: second module

430: processor

500: monitoring equipment

540: Prompt module

700: monitoring equipment

710: processor

720: Communication interface

730: Memory

732: Program

740: Communication bus

Figure 1 is a schematic structural diagram of a monitoring device according to an embodiment of the present application.

FIG. 2 is a schematic structural diagram of a monitoring device according to another embodiment of this application.

Figure 3 is a schematic diagram of the first collector and the second collector of the monitoring device are cameras, which are installed in smart glasses.

FIG. 4 is a schematic structural diagram of a monitoring device according to another embodiment of this application.

FIG. 5 is a schematic structural diagram of a monitoring device according to another embodiment of this application.

FIG. 6 is a schematic flowchart of a monitoring method according to an embodiment of the application.

FIG. 7 is a schematic flowchart of a monitoring method according to another embodiment of this application.

FIG. 8 is a schematic structural diagram of yet another monitoring device provided by an embodiment of the present application.

Example embodiments will now be described more fully with reference to the drawings. However, the example embodiments can be implemented in various forms, and should not be construed as being limited to the embodiments set forth herein; on the contrary, providing these embodiments makes the present invention comprehensive and complete, and fully conveys the concept of the example embodiments to Those with ordinary knowledge in this field. The same reference numerals in the drawings denote the same or similar structures, and thus their repeated description will be omitted.

The described features, structures, or characteristics may be combined in one or more embodiments in any suitable manner. In the following description, many specific details are provided to give a full understanding of the embodiments of the invention. However, those with ordinary knowledge in the art should realize that without one or more of the specific details, or using other methods, components, materials, etc., the technical solution of the present invention can also be practiced. In some cases, well-known structures, materials, or operations are not shown or described in detail to avoid obscuring the present invention.

The specific implementation of the present application will be further described in detail below with reference to the drawings and embodiments. The following embodiments are used to illustrate this application, but are not used to limit the protection scope of this application.

Those with ordinary knowledge in the art can understand that the terms “first” and “second” in this application are only used to distinguish different devices, modules or parameters, etc., and neither represent any specific technical meaning nor represent their Necessary logical order.

FIG. 1 is a schematic structural diagram of a monitoring device 100 according to an embodiment of the present application, including: a first collector 110, a second collector 120, and a processor 130, where:

The first collector is used to collect the continuous left image frame generated by the left masticatory muscle within a period of time, and send the continuous left image frame to the processor; wherein, in the embodiment of the present application, the first collector may It is an imaging device, such as a camera, which captures a continuous left image frame generated by the user's mastication muscles at a certain time interval t. Of course, those with ordinary knowledge in the field should be able to understand, this is only an example, and should not be construed as a limitation to the embodiments of the present application.

The second collector is used to collect the continuous right image frame generated by the right masticatory muscle during the time period and send the continuous right image frame to the processor; wherein, in the embodiment of the present application, the second collector may be An imaging device, such as a camera, captures a continuous right image frame generated by the user's mastication muscles at a certain time interval t. Of course, those with ordinary knowledge in the field should be able to understand, this is only an example, and should not be construed as a limitation to the embodiments of the present application.

The processor is used to obtain the continuous left image frame generated by the left masticatory muscle and the continuous right image frame generated by the right masticatory muscle during the time period, and obtain a change value of a left image frame according to the continuous left image frame, A change value of a right image frame is obtained according to consecutive right image frames, and at least according to a change value of a left image frame and a change value of a right image frame, it is determined whether it is one-sided chewing within a time period.

The inventor of the present application found that when the user chews with the left chewing muscle, the movement range of the left chewing muscle is greater than that of the right chewing muscle, resulting in the left chewing muscle The change value of the continuous image frame produced by chewing is greater than the change value of the continuous image frame produced by mastication of the right masticatory muscle. Similarly, when the user chews with the right masticatory muscle, the movement amplitude of the right masticatory muscle is greater than that of the left masticatory muscle, resulting in the continuous image frame generated by the right masticatory muscle is greater than that of the left masticatory muscle. The change value of the image frame. Similarly, when the user chews with the left and right masticatory muscles, the movement amplitude of the left masticatory muscle is equal to or almost equal to the movement amplitude of the right masticatory muscle, resulting in the change value of the continuous image frame produced by the right masticatory muscle is equal to or almost equal to The change value of the continuous image frame produced by mastication of the left chewing muscle. Therefore, according to the change value of the continuous image frame generated by the left masticatory muscles and the change value of the continuous image frame generated by the right masticatory muscles, it can be determined as the left side chewing or the right side chewing, similarly determined as non-unilateral chewing Or chew on both sides.

The processor determines that the left image is chewed according to the change value of the continuous image frame generated by the left masticatory muscle mastication is greater than the change value of the image frame generated by the right masticatory muscle; or, based on the continuous image generated by the left masticatory muscle mastication The change value of the image frame is smaller than the change value of the continuous image frame produced by the chewing muscle of the right side, and it is determined as the right chew. Alternatively, it can be determined as a double-sided chew or a non-unilateral chew based on the change value of the continuous image frame generated by the right masticatory muscle mastication being equal to or almost equal to the change value of the continuous image frame generated by the left masticatory muscle chew.

Wherein, the processor obtains the change value V of the continuous image frame generated by mastication of the left or right masticatory muscle to calculate the R(red)G(green)B(blue) value of the pixel points of the same position of two consecutive image frames The square root of the difference sum of squares, n is the number of primitive points in the image frame. In the above manner, the present invention counts the R(red), Change of G(green) and B(blue) values, Thus, the change value V of the reaction image frame is obtained. The change value V of the image frame reflects the movement of the masticatory muscles. When the change value V is larger, it can be judged that the amount of movement of the masticatory muscle on that side is large; when the change value V is smaller, the side can be judged The amount of exercise in the masticatory muscles is very small. Therefore, by comparing the change value V of the image frames of the left and right masticatory muscles, the comparison parameters of the movement conditions of the left and right masticatory muscles can be obtained. By comparing the comparison parameters of the two, the movement of the left masticatory muscles can be learned more (One-sided chewing, focusing on the left chewing) or more movement of the right chewing muscle (one-sided chewing, focusing on the right chewing).

Those of ordinary skill in the art should understand that the above examples of obtaining the change value of the continuous image frame caused by the left masticatory muscle mastication and the change value of the continuous image frame caused by the right masticatory muscle mastication are only for illustration. It should not be understood as a limitation to the embodiments of the present application. Those with ordinary knowledge in this field should make corresponding changes and modifications accordingly, and they should be covered by the protection scope of the patent scope of this application.

Wherein, in the embodiment of the present application, the processor may be provided on the wearable device. The wearable devices mentioned here are, for example, smart glasses and earphones. The processor can communicate with the first collector and the second collector through Bluetooth, infrared and other methods. Of course, those with ordinary knowledge in this field should be able to understand, this is only for example, and should not be construed as a limitation of this application. Other ways to communicate with each other are also included in the scope of protection of this application.

In this embodiment of the present application, the change value of the continuous image frame generated by the left masticatory muscle and the change value of the continuous image frame generated by the right masticatory muscle respectively reflect the left chewing situation and the right chewing situation, such as whether to chew or not , The range of motion of chewing, etc. Therefore, according to the change value of the continuous image frame produced by the left masticatory muscle and the right masticatory muscle production The change value of the continuous image frame can be determined whether it is one-sided chewing. Thus, it is possible to automatically monitor whether unilateral chewing is performed.

In a possible implementation manner, the processor obtains the difference between the change value of the left image frame and the change value of the right image frame, and determines the time period according to the change value difference and the change difference threshold Whether the inside is chewed unilaterally.

In a possible implementation manner, the processor obtains the chewing duration of one side chewing in multiple time periods to obtain the cumulative duration of chewing on the left side and the cumulative duration of chewing on the right side, and the cumulative duration of left chewing and the chewing time on the right side The absolute value of the difference of the accumulated time, and at least according to the absolute value of the difference and the difference threshold, it is determined whether it is one-sided chewing in multiple time periods. The difference threshold can be set according to the situation.

For n time periods (n is a natural number), for example, paragraph 1, paragraph 2, paragraph 3... paragraph n, get the chewing duration of each period of time, and accumulate n chewing duration to get Cumulative duration. The way to record the length of time is, for example, to calculate the length of chewing based on the time of starting chewing and the time of ending chewing. Of course, those with ordinary knowledge in the art can also calculate the chewing duration in other ways. This is only for illustration, and should not be construed as limiting the protection scope of the present application.

In this embodiment of the present application, if the absolute value of the difference between the cumulative length of chewing on the left side and the cumulative length of chewing on the right side is greater than the difference threshold, it means that the user chews on one side for a longer period of time. Thus, it is possible to automatically monitor whether unilateral chewing is performed.

FIG. 2 is a schematic structural diagram of a monitoring device 200 according to another embodiment of the present application. The monitoring device may further include: a prompter 240, configured to perform a single-sided chewing prompt based on a result determined by the processor as a single-sided chewing.

Wherein, the processor determines that it is unilateral chewing, and sends a trigger signal for unilateral chewing to the prompter, and the prompter performs a unilateral chewing prompt according to the trigger signal.

For example, the absolute value of the difference between the cumulative duration of left chewing and the cumulative duration of right chewing is greater than the difference threshold, and the cumulative duration of left chewing is greater than the cumulative duration of chewing on the right. Prompt to chew on the left side for a long time, or remind users to chew on the right side. Or the user is reminded that the length of chewing on the left side and the length of chewing on the right side are for the user's reference to adjust the chewing side. Similarly, a single-side chewing prompt can be made for the case where the user uses the right side to chew.

In a possible embodiment, there is one prompter, that is, the processor sends a trigger signal for one-sided chewing to the prompter, and the prompter prompts according to the trigger signal. The user can know that he has used one-sided chewing for a long time, and can adjust the chewing side accordingly.

In another possible embodiment, the prompter includes a left prompter and a right prompter. For the case of determining that the left side is chewing, the processor may send a trigger signal for unilateral chewing to the left prompter, and the left prompter according to the trigger signal For prompting, the user can know that he has used the left side for chewing for a long time, and can be adjusted accordingly to use the right side for chewing; similarly, for the case of determining the right side to chew, the processor can send a trigger signal for unilateral chewing to the right prompter The right prompter prompts according to the trigger signal, and the user can know that he has used the right side for chewing for a long time, and can be adjusted to use the left side for chewing accordingly. Of course, those with ordinary knowledge in the art can understand that this is only for illustration, and should not be construed as a limitation to the embodiments of the present application. For example, when the user uses chewing on the left for a long time, the processor may also send a trigger signal to the right reminder, and prompt the user to chew on the right according to the trigger information.

Among them, the reminder is provided on the wearable device or the mobile terminal. The wearable devices mentioned here are, for example, smart glasses and earphones. The reminder may be an indicator light, an alarm, or a spectacle lens of smart glasses, or the like.

A possible embodiment is that an indicator light or an alarm is respectively provided on the left or right side of the smart glasses. For a user to chew on the left side for a long time, the indicator light on the left flashes to remind, or the alarm is prompted by the alarm. The reminder can be set on the left or right spectacle frame of the smart glasses; it can also be set on the left or right temple.

A possible embodiment is that, by displaying the caption content on the smart glasses, that is, using the display screen of the smart glasses as a reminder, the processor sends the prompt information of the unilateral chewing results to the glasses, for example, the left side has been used for a long time Chew and display the reminder information on the smart glasses. The user can see the prompt information and adjust the chew side accordingly.

A possible embodiment is to set an indicator light or an alarm on the left or right side of the earphone respectively, for a user to chew on the left side for a long time, the indicator light on the left flashes to remind, or an alarm is reminded through the alarm. Here, the left side is, for example, the left earpiece of the earphone, and the right side is, for example, the right earpiece of the earphone.

One possible embodiment is that the prompter is provided on the mobile terminal. The processor determines that it is unilateral chewing, and sends a trigger signal for unilateral chewing to the mobile terminal, and the mobile terminal performs unilateral chewing prompts in at least one manner of sound prompt, indicator flashing, alarm sound, and subtitle content display according to the trigger signal.

In this embodiment of the present application, according to the result determined to be one-sided chewing, a one-sided chewing prompt is performed, and the user may be recommended to use one side for a short time to chew, thereby reminding the user to chew evenly.

Figure 3 is a schematic diagram of the first collector and the second collector of the monitoring device are cameras, which are installed in smart glasses. In the following, an example will be described in which both the processor and the prompter are installed in smart glasses. Those with ordinary knowledge in this field should be able to understand that this is just for example and should not be construed as limiting the patent scope of this application. In this embodiment, the first module, the second module, and the processor are disposed in a smart glasses 8, and the smart glasses 8 include a left lens 81 and a right lens 82. The first module and the second module continuously take pictures of the masticatory muscles on both sides of the mouth. The first module is a first camera 110 disposed under the left lens 81 of the smart glasses 8. The first camera 110 focuses on the left masticatory muscle 83, that is, the first camera 110 continuously takes pictures of the area where the left masticatory muscle 83 is located. The second module is a second camera 120 disposed under the right lens 82 of the smart glasses 8, and the second camera 120 focuses on the right masticatory muscle 84, that is, the second camera 120 continuously takes pictures of the area where the right masticatory muscle 84 is located. The processor is set in the smart glasses to obtain the change value of the continuous image frame generated by the left masticatory muscle and the change value of the continuous image frame generated by the right masticatory muscle within a period of time, according to the continuous The change value of the image frame is greater than the change value of the continuous image frame produced by the right masticatory muscles. It is determined to be one-sided chewing and the left-side chewing. Record the left-side chewing duration and obtain the left-side chewing duration in n time periods .

Similarly, the length of chewing on the right side in n time periods is obtained.

The cumulative duration of chewing on the left side is 3 hours, and the cumulative duration of chewing on the right side is 1 hour. The absolute value of the difference between the cumulative length of left-hand chewing and the cumulative time of right-hand chewing is 2 hours, which is greater than the difference threshold of 30 minutes.

The processor sends reminder information to the left reminder set on the left frame of the smart glasses (can also be the temple), triggering the left reminder to remind you to use the left side for a long time to chew. The left reminder is for example a sound prompt , Or indicator lights, alarms. Alternatively, the processor sends a prompt message to the right prompter provided on the right frame (or the temple) of the smart glasses, and triggers the right prompter to prompt "please chew on the right side". The right prompter is a voice prompt, for example. Or indicator lights, alarms. Alternatively, the content of the subtitles can be displayed on the smart glasses, that is, the display screen of the smart glasses can be used as a prompter, and the processor can send prompt information to the prompter to display the prompt information on the smart glasses. Chew on the left side, or use the right side for chewing, or display the chewing length on the left side and the chewing length on the right side on the smart glasses."

The embodiment of the present application can enable the user to chew evenly, thereby avoiding asymmetrical facial muscles of the user, bleeding gums, bad breath, and temporomandibular joint disease caused by uneven chewing.

FIG. 4 is a schematic structural diagram of a monitoring device 400 according to yet another embodiment of the present application, including: a first module 410, a second module 420, and a processor 430, wherein: the first module is used to obtain a The continuous left image frame generated by the left chewing muscle during the time period, and the continuous left image frame are sent to the processor; The second module is used to obtain the continuous right image frame generated by the right masticatory muscle in the time period and send the continuous right image frame to the processor; the processor is used to obtain a left image according to the continuous left image frame The change value of the picture frame, according to the continuous right image frame to obtain a right image frame change value, at least according to a left image frame change value and a right image frame change value, to determine whether the time period is single Side chew.

The inventor of the present application found that when the user chews with the left masticatory muscle, the movement amplitude of the left masticatory muscle is greater than that of the right masticatory muscle, resulting in the change value of the continuous image frame produced by the left masticatory muscle is greater than that of the right masticatory muscle. Changes in the resulting continuous image frame. Similarly, when the user chews with the right masticatory muscle, the movement amplitude of the right masticatory muscle is greater than that of the left masticatory muscle, resulting in the continuous image frame generated by the right masticatory muscle is greater than that of the left masticatory muscle. The change value of the image frame. Similarly, when the user chews with the left and right masticatory muscles, the movement amplitude of the left masticatory muscle is equal to or almost equal to the movement amplitude of the right masticatory muscle, resulting in the change value of the continuous image frame produced by the right masticatory muscle is equal to or almost equal to The change value of the continuous image frame produced by mastication of the left chewing muscle. Therefore, according to the change value of the continuous image frame generated by the left masticatory muscles and the change value of the continuous image frame generated by the right masticatory muscles, it can be determined as the left side chewing or the right side chewing, similarly determined as non-unilateral chewing Or chew on both sides.

The processor determines that the left image is chewed according to the change value of the continuous image frame generated by the left masticatory muscle mastication is greater than the change value of the image frame generated by the right masticatory muscle; or, based on the continuous image generated by the left masticatory muscle mastication The change value of the image frame is smaller than the change value of the continuous image frame produced by the chewing muscle of the right side, and it is determined as the right chew. or According to the change value of the continuous image frame caused by mastication of the right masticatory muscle is equal to or almost equal to the change value of the continuous image frame caused by mastication of the left masticatory muscle, it is determined to be bilateral chewing or non-unilateral chewing.

In the embodiments of the present application, the first module, the second module, and the processor may be disposed on the wearable device. The wearable devices mentioned here are, for example, smart glasses and earphones.

The first module and the second module here can be implemented by a transceiver, for example, and the processor can be implemented by a calculator and a comparator. The calculator calculates the change value of the left image frame and the change value of the right image frame, The change value of the left image frame and the change value of the right image frame are respectively output to the two input terminals of the comparator, and the comparator performs comparison according to the above two values, and outputs a signal indicating the comparison result. For example, the change value of the left image frame is greater than the change value of the right image frame, and a high potential signal is output, otherwise, a low potential signal is output. In another embodiment, if a prompt module is also connected, the high-potential or low-potential signal is sent to the prompt module, and the prompt module prompts according to the high-potential signal or the low-potential signal. Of course, the first module, the second module, and the processor can also be implemented by software programs and stored in a device or storage medium, which will be described in the following embodiments.

Wherein, the processor obtains the change value V of the continuous image frame generated by mastication of the left or right masticatory muscle to calculate the R(red)G(green)B(blue) value of the pixel points of the same position of two consecutive image frames The square root of the sum of squared differences.

Those of ordinary skill in the art should understand that the above-mentioned acquisition of the change value of the continuous image frame generated by the left masticatory muscles and the continuous value generated by the right masticatory muscles during mastication The example of the change value of the image frame is only for illustration, and should not be construed as a limitation to the embodiment of the present application. Those with ordinary knowledge in this field should make corresponding changes and modifications accordingly, and they should be covered by the protection scope of the patent scope of this application.

In this embodiment of the present application, the change value of the continuous image frame caused by the left masticatory muscle mastication and the change value of the continuous image frame caused by the right masticatory muscle mastication respectively reflect the left chewing situation and the right chewing situation, for example Whether to chew, the range of motion of chewing, etc. Therefore, according to the change value of the continuous image frame caused by the left chewing muscle chewing and the change value of the continuous image frame caused by the right chewing muscle chewing, it can be determined whether it is unilateral chewing . Thus, it is possible to automatically monitor whether unilateral chewing is performed.

In a possible implementation manner, the processor obtains the difference between the change value of the left image frame and the change value of the right image frame, and determines the time period according to the change value difference and the change difference threshold Whether the inside is chewed unilaterally.

In a possible implementation manner, the processor obtains the chewing duration of one side chewing in multiple time periods to obtain the cumulative duration of chewing on the left side and the cumulative duration of chewing on the right side, and the cumulative duration of left chewing and the chewing time on the right side The absolute value of the difference of the accumulated time, and at least according to the absolute value of the difference and the difference threshold, it is determined whether it is one-sided chewing in multiple time periods.

Wherein, the absolute value of the difference between the cumulative duration of left-hand chewing and the cumulative duration of right-hand chewing is greater than or equal to the difference threshold, and is determined to be unilateral chewing. For the determination of unilateral chewing, and the cumulative length of chewing on the left side is greater than the cumulative length of chewing on the right side, determine the left chewing; for the determination of unilateral chewing, and the cumulative length of chewing on the left side is less than the right The cumulative length of side chewing is determined as the right side chewing. The absolute value of the difference between the cumulative duration of left-hand chewing and the cumulative duration of right-hand chewing is less than the difference threshold, and is determined to be bilateral chewing.

For n time periods (n is a natural number), for example, paragraph 1, paragraph 2, paragraph 3... paragraph n, get the chewing duration of each period of time, and accumulate n chewing duration to get Cumulative duration. The way to record the length of time is, for example, to calculate the length of chewing based on the time of starting chewing and the time of ending chewing. Of course, those with ordinary knowledge in the art can also calculate the chewing duration in other ways. This is only for illustration, and should not be construed as limiting the protection scope of the present application.

In this embodiment of the present application, if the absolute value of the difference between the cumulative length of chewing on the left side and the cumulative length of chewing on the right side is greater than the difference threshold, it means that the user chews on one side for a longer period of time. Thus, it is possible to automatically monitor whether unilateral chewing is performed.

FIG. 5 is a schematic structural diagram of a monitoring device 500 according to another embodiment of the present application. Based on the embodiment shown in FIG. 4, the monitoring device may further include: a prompt module 540, which is used to determine As a result of one-sided chewing, one-sided chewing tips are given.

Wherein, the processor determines that it is one-sided chewing and sends a trigger signal for one-sided chewing to the prompt module, and the prompt module performs a unilateral chewing prompt according to the trigger signal.

For example, the absolute value of the difference between the cumulative duration of left chewing and the cumulative duration of right chewing is greater than the difference threshold, and the cumulative duration of left chewing is greater than the cumulative duration of chewing on the right. Prompt to chew on the left side for a long time, or remind users to chew on the right side. Or to remind the user how long to chew on the left and The length of chewing on the right side is for the user's reference to adjust the chewing side. Similarly, a single-side chewing prompt can be made for the case where the user uses the right side to chew.

In a possible embodiment, there is one prompting module, that is, the processor sends a trigger signal for one-sided chewing to the prompter, and the prompting module performs prompting according to the triggering signal. The user can know that he has used one-sided chewing for a long time, and can adjust the chewing side accordingly.

In another possible embodiment, the prompting module includes a left prompting module and a right prompting module. For a case where the left side is chewed, the processor may send a trigger signal for unilateral chewing to the left prompting module, and the left prompting module The group prompts according to the trigger signal, the user can know that he has used the left side for chewing for a long time, and can be adjusted to use the right side for chewing accordingly; similarly, for the case of determining the right side to chew, the processor can send to the right prompt module For the trigger signal of unilateral chewing, the prompting module on the right side prompts according to the triggering signal. The user can know that he has used the right side for chewing for a long time, and can be adjusted to use the left side for chewing accordingly. Of course, those with ordinary knowledge in the art can understand that this is only for illustration, and should not be construed as a limitation to the embodiments of the present application. For example, when the user uses chewing on the left side for a long time, the processor may also send a trigger signal to the right prompter to prompt the user to chew on the right side according to the trigger information.

Among them, the prompt module is provided in the wearable device or the mobile terminal. The wearable devices mentioned here are, for example, smart glasses and earphones. The prompt module may be an indicator light, an alarm, or a spectacle lens of smart glasses, etc.

A possible embodiment is to set an indicator light or an alarm on the left or right side of the smart glasses, for the user to chew on the left side for a long time, the left side indicates The lamp flashes to remind, or through the siren to alert. The reminder can be set on the left or right spectacle frame of the smart glasses; it can also be set on the left or right temple.

A possible embodiment is that, by displaying the caption content on the smart glasses, that is, using the display screen of the smart glasses as a reminder, the processor sends the prompt information of the unilateral chewing results to the glasses, for example, the left side has been used for a long time Chew and display the reminder information on the smart glasses. The user can see the prompt information and adjust the chew side accordingly.

A possible embodiment is to set an indicator light or an alarm on the left or right side of the earphone respectively, for a user to chew on the left side for a long time, the indicator light on the left flashes to remind, or an alarm is reminded through the alarm. Here, the left side is, for example, the left earpiece of the earphone, and the right side is, for example, the right earpiece of the earphone.

One possible embodiment is that the prompt module is provided in the mobile terminal. The processor determines that it is unilateral chewing, and sends a trigger signal for unilateral chewing to the mobile terminal, and the mobile terminal performs unilateral chewing prompts in at least one manner of sound prompt, indicator flashing, alarm sound, and subtitle content display according to the trigger signal.

In this embodiment of the present application, according to the result determined to be one-sided chewing, a one-sided chewing prompt is performed, and the user may be recommended to use one side for a short time to chew, thereby reminding the user to chew evenly.

FIG. 6 is a schematic flowchart of a monitoring method according to an embodiment of the present application. The method includes the following steps: S610: Obtain the continuous left image frame generated by the left masticatory muscle in a period of time, and obtain the change value of a left image frame according to the continuous left image frame; S620. Acquire the continuous right of the right masticatory muscle in the period of time Image frame, obtain the change value of a right image frame according to the continuous right image frame; S630, at least according to the change value of a left image frame and the change value of a right image frame, determine whether the time period is single Side chew.

The inventor of the present application found that when the user chews with the left masticatory muscle, the movement amplitude of the left masticatory muscle is greater than that of the right masticatory muscle, resulting in the change value of the continuous image frame produced by the left masticatory muscle is greater than that of the right masticatory muscle. Changes in the resulting continuous image frame. Similarly, when the user chews with the right masticatory muscle, the movement amplitude of the right masticatory muscle is greater than that of the left masticatory muscle, resulting in the continuous image frame generated by the right masticatory muscle is greater than that of the left masticatory muscle. The change value of the image frame. Similarly, when the user chews with the left and right masticatory muscles, the movement amplitude of the left masticatory muscle is equal to or almost equal to the movement amplitude of the right masticatory muscle, resulting in the change value of the continuous image frame produced by the right masticatory muscle is equal to or almost equal to The change value of the continuous image frame produced by mastication of the left chewing muscle. Therefore, according to the change value of the continuous image frame generated by the left masticatory muscles and the change value of the continuous image frame generated by the right masticatory muscles, it can be determined as the left side chewing or the right side chewing, similarly determined as non-unilateral chewing Or chew on both sides.

According to the change value of the continuous image frame produced by the left masticatory muscle mastication greater than the change value of the continuous image frame produced by the right masticatory muscle mastication, it is determined as the left chewing; or, the continuous image frame produced by the left masticatory muscle mastication The change value of is less than the change value of the continuous image frame produced by the right chewing muscle chewing, and it is determined as the right chewing. Or, root According to the change value of the continuous image frame produced by the right masticatory muscle mastication is equal to or almost equal to the change value of the continuous image frame produced by the left masticatory muscle mastication, it is determined to be bilateral chewing or non-unilateral chewing.

Wherein, the processor obtains the change value V of the continuous image frame generated by mastication of the left or right masticatory muscle as the R(red)G(green)B(blue) value of the pixel points of the same position of two consecutive image frames The square root of the sum of squared differences.

Those of ordinary skill in the art should understand that the above examples of obtaining the change value of the continuous image frame caused by the left masticatory muscle mastication and the change value of the continuous image frame caused by the right masticatory muscle mastication are only for illustration. It should not be understood as a limitation to the embodiments of the present application. Those with ordinary knowledge in this field should make corresponding changes and modifications accordingly, and they should be covered by the protection scope of the patent scope of this application.

In this embodiment of the present application, the change value of the continuous image frame caused by the left masticatory muscle mastication and the change value of the continuous image frame caused by the right masticatory muscle mastication respectively reflect the left chewing situation and the right chewing situation, for example Whether to chew, the range of motion of chewing, etc. Therefore, according to the change value of the continuous image frame caused by the left chewing muscle chewing and the change value of the continuous image frame caused by the right chewing muscle chewing, it can be determined whether it is unilateral chewing . Thus, it is possible to automatically monitor whether unilateral chewing is performed.

FIG. 7 is a schematic flowchart of a monitoring method according to another embodiment of this application. In another possible implementation manner, as shown in FIG. 7, the following steps are included:

S810. Acquire a continuous left image frame generated by the left masticatory muscle within a period of time, and obtain a change value of a left image frame according to the continuous left image frame.

S820. Acquire a continuous right image frame generated by the right masticatory muscle in the time period, and obtain a change value of a right image frame according to the continuous right image frame.

S830. Determine, according to at least a change value of a left image frame and a change value of a right image frame, whether it is one-sided chewing within a time period.

S840. Obtain the chewing duration of one-sided chewing in multiple time periods to obtain the cumulative duration of chewing on the left side and the cumulative duration of chewing on the right side; obtain the absolute value of the difference between the cumulative length of left chewing and the cumulative time of right chewing Value, and at least according to the absolute value of the difference and the difference threshold, determine whether it is one-sided chewing in multiple time periods. The difference threshold can be set according to the situation.

In step S840, the absolute value of the difference between the cumulative duration of left-hand chewing and the cumulative duration of right-hand chewing is greater than or equal to the difference threshold, and is determined to be unilateral chewing. For the determination of unilateral chewing, and the cumulative length of chewing on the left side is greater than the cumulative length of chewing on the right side, determine the left chewing; for the determination of unilateral chewing, and the cumulative length of chewing on the left side is less than the cumulative length of chewing on the right side , Determined to chew on the right. The absolute value of the difference between the cumulative duration of left-hand chewing and the cumulative duration of right-hand chewing is less than the difference threshold, and is determined to be bilateral chewing.

For n time periods (n is a natural number), for example, paragraph 1, paragraph 2, paragraph 3... paragraph n, get the chewing duration of each period of time, and accumulate n chewing duration to get Cumulative duration. The way to record the length of time is, for example, to calculate the length of chewing based on the time of starting chewing and the time of ending chewing. Of course, those with ordinary knowledge in the art can also calculate the chewing duration in other ways. This is only for illustration, and should not be construed as limiting the protection scope of the present application.

In this embodiment of the present application, if the absolute value of the difference between the cumulative length of chewing on the left side and the cumulative length of chewing on the right side is greater than the difference threshold, it means that the user chews on one side for a longer period of time. Thus, it is possible to automatically monitor whether unilateral chewing is performed.

In the embodiment of the present application, it may further include: performing a unilateral chewing prompt according to the result determined to be unilateral chewing.

For example, the absolute value of the difference between the cumulative duration of left chewing and the cumulative duration of right chewing is greater than the difference threshold, and the cumulative duration of left chewing is greater than the cumulative duration of chewing on the right. Prompt to chew on the left side for a long time, or remind users to chew on the right side. Or the user is reminded that the length of chewing on the left side and the length of chewing on the right side are for the user's reference to adjust the chewing side. Similarly, a single-side chewing prompt can be made for the case where the user uses the right side to chew.

In this embodiment of the present application, a unilateral chewing prompt is given to the user according to the result that the absolute value of the difference is greater than the difference threshold, and the user can be recommended to chew unilaterally for a short time to remind the user to chew evenly.

Those of ordinary knowledge in the art can understand that in the above method of the embodiments of the present application, the sequence number size of each step does not mean the order of execution, and the execution order of each step should be determined by its function and internal logic, and should not be dealt with. The implementation process of the embodiments of the present application constitutes any limitation.

FIG. 8 is a schematic structural diagram of yet another monitoring device 700 provided by an embodiment of the present application. This embodiment does not limit the specific implementation of the monitoring device 700. As shown in FIG. 8, the monitoring device 700 may include: A processor (processor) 710, a communication interface (Communications Interface) 720, a memory (memory) 730, and a communication bus 740. among them:

The processor 710, the communication interface 720, and the memory 730 communicate with each other through the communication bus 740.

The communication interface 720 is used to communicate with network elements such as users.

The processor 710 is used to execute the program 732.

The program 732 may include code, and the code includes computer operation instructions.

The processor 710 may be a central processing unit CPU, or an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application.

The memory 730 is used to store the program 732. The memory 730 may include a high-speed RAM memory, or may further include a non-volatile memory (non-volatile memory), for example, at least one magnetic disk memory. The program 732 may specifically include: a first module for acquiring a continuous left image frame generated by the left masticatory muscle within a period of time and sending the continuous left image frame to the processor; a second module for acquiring time The continuous right image frame generated by the right masticatory muscle in the segment sends the continuous right image frame to the processor; the processor is used to obtain the change value of a left image frame according to the continuous left image frame, according to the continuous The right image frame obtains the change value of a right image frame, and at least according to the change value of a left image frame and the change value of a right image frame, it is determined whether it is one-sided chewing during the time period.

In this embodiment of the present application, it can be determined whether it is unilateral mastication according to the change value of the continuous image frame caused by the left masticatory muscle mastication and the change value of the continuous image frame caused by the right masticatory muscle mastication. Thus, it is possible to automatically monitor whether unilateral chewing is performed.

For specific implementation of each module in the program 732, reference may be made to the corresponding description in the corresponding module in the foregoing embodiment, and details are not described herein. Those with ordinary knowledge in the art can clearly understand that for the convenience and conciseness of the description, the specific working processes of the devices and modules described above can refer to the corresponding process descriptions in the foregoing method embodiments, which will not be repeated here.

Those with ordinary knowledge in the art can clearly understand that for the convenience and conciseness of the description, the specific working processes of the above-described devices and modules can refer to the corresponding descriptions in the foregoing device embodiments, which will not be repeated here.

Those of ordinary skill in the art may realize that the example modules and method steps described in conjunction with the embodiments disclosed in this application can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application of the technical solution and design constraints. Professional technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application essentially or part of the contribution to the original technology or part of the technical solution can be reflected in the form of a software product, the computer software product is stored in a storage medium, including Several commands are used to make a computer device (may It is a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods of the embodiments of the present application. The aforementioned computer-readable storage media include physical volatile and non-volatile, removable and removable physical storage devices implemented by any method or technology that stores information such as computer-readable instructions, data structures, program modules, or other data. Non-removable media. Computer-readable storage media include, but are not limited to, flash drives, removable hard drives, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), erasable and removable Programming read-only memory (EPROM), electrically erasable and programmable read-only memory (EEPROM), flash memory or other solid-state memory technology, CD-ROM, digital versatile disc (DVD), HD-DVD , Blue-Ray or other optical storage devices, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store the required information and can be accessed by a computer.

In summary, the unilateral chewing monitoring device and method of the present invention can determine whether it is unilateral chewing, and thus, can automatically monitor whether unilateral chewing is performed.

In summary, although the present invention has been disclosed as above with examples, it is not intended to limit the present invention. Those with ordinary knowledge in the technical field to which the present invention belongs can make various modifications and retouching without departing from the spirit and scope of the present invention. Therefore, the scope of protection of the present invention shall be deemed as defined by the scope of the attached patent application.

400: monitoring equipment

410: The first module

420: second module

430: processor

Claims (12)

A unilateral chewing monitoring device includes: a first module, the first module acquiring a continuous left image frame generated by a left chewing muscle in a period of time; a second module, the second module Acquiring a continuous right image frame generated by the right masticatory muscle in the time period; a processor, the processor receiving the left image frame of the first module and the second module of the second module Right image frame; the processor obtains a left change value of a left image frame from the continuous left image frame, and obtains a right change value of a right image frame from the continuous right image frame; the processor at least According to the comparison result of the change value of the left image frame and the change value of the right image frame, it is determined whether the time period is one-sided chewing. The monitoring device according to item 1 of the patent application scope, wherein the processor obtains the difference between the change value of the left image frame and the change value of the right image frame, according to the The difference between the change value and the change difference threshold determines whether it is one-sided chewing within the time period. The monitoring device as described in item 1 of the patent application scope, wherein the processor obtains the chewing duration of one side chew in multiple time periods, obtains the cumulative duration of left chewing and the cumulative duration of right chewing, and obtains the left chewing The absolute value of the difference between the cumulative duration of and the cumulative duration of chewing on the right side, and at least according to the absolute value of the difference and the difference threshold, determine whether the multiple time periods are unilateral chewing. The monitoring device according to item 1 of the patent application scope, wherein the first module, the second module, and the processor are provided on a wearable device. For example, the monitoring device according to any one of items 1 to 4 of the patent application scope, wherein the processor acquires the change value of the left or right image frame to calculate the RGB value of the pixel points of the same position of two consecutive image frames The square root of the sum of squared differences. The monitoring device according to item 5 of the patent application scope, wherein the first module, the second module, and the processor are disposed in a pair of smart glasses, and the first module is a left lens disposed in the smart glasses A first camera on the lower side, the first camera focuses on the left chewing muscle; the second module is a second camera disposed under the right lens of the smart glasses, and the second camera focuses on the right chewing muscle. The monitoring device as described in item 5 of the patent application scope further includes: a prompter, which is used to perform a single-side chewing prompt based on the result determined by the processor as one-sided chewing. The monitoring device according to item 7 of the patent application scope, wherein the prompter is provided on the wearable device or the mobile terminal. A unilateral chewing monitoring method includes the following steps: obtaining a continuous left image frame generated by a left masticatory muscle within a period of time, and obtaining a change value of a left image frame according to the continuous left image frame; and obtaining the time A continuous right image frame generated by the right masticatory muscle in the segment, a change value of a right image frame is obtained according to the continuous right image frame; at least according to a change value of the left image frame and a right image The result of the comparison of the changed values of the boxes determines whether the side is chewed unilaterally within the time period. The unilateral chew monitoring method as described in item 9 of the patent application scope, wherein the change value of the left or right image frame is the square root of the sum of the square of the difference between the RGB values of the pixel points of the same position of two consecutive image frames . The unilateral chewing monitoring method as described in item 9 of the patent application scope, wherein the step of determining whether the side is chewed in one side according to the result of the comparison includes obtaining the change value and the value of the left image frame The difference between the change value of the change value of the right image frame; according to the difference value of the change value and the change difference threshold, it is determined whether the time period is one-sided chewing. The unilateral chewing monitoring method as described in item 9 of the patent application scope further includes the following steps: obtaining the chewing duration of one side chewing in multiple time periods, obtaining the cumulative duration of chewing on the left side and the cumulative duration of chewing on the right side; Obtaining the absolute value of the difference between the cumulative duration of left-side chewing and the cumulative time of right-side chewing, and at least according to the absolute value of the difference and the difference threshold, determining whether the multiple time periods are unilateral chewing.

Images