CN112213134A - Electric toothbrush oral cavity cleaning quality detection system and detection method based on acoustics - Google Patents
Electric toothbrush oral cavity cleaning quality detection system and detection method based on acoustics Download PDFInfo
- Publication number
- CN112213134A CN112213134A CN202011033308.8A CN202011033308A CN112213134A CN 112213134 A CN112213134 A CN 112213134A CN 202011033308 A CN202011033308 A CN 202011033308A CN 112213134 A CN112213134 A CN 112213134A
- Authority
- CN
- China
- Prior art keywords
- signal
- module
- tooth
- signals
- acoustic
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01M—TESTING STATIC OR DYNAMIC BALANCE OF MACHINES OR STRUCTURES; TESTING OF STRUCTURES OR APPARATUS, NOT OTHERWISE PROVIDED FOR
- G01M99/00—Subject matter not provided for in other groups of this subclass
- G01M99/008—Subject matter not provided for in other groups of this subclass by doing functionality tests
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/25—Fusion techniques
- G06F18/253—Fusion techniques of extracted features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2218/00—Aspects of pattern recognition specially adapted for signal processing
- G06F2218/12—Classification; Matching
Abstract
The application discloses electric toothbrush oral cavity cleaning quality detection system and method based on acoustics, wearable equipment using a low-cost throat microphone and a Bluetooth earphone is constructed based on acoustics principle, and a general electric toothbrush on the market is subjected to a tooth brushing behavior recognition model provided by the method by analyzing the audio characteristics of the electric toothbrush, so that the tooth brushing quality of an application user can be detected in real time, and a long-term oral cavity health file of the application user is established.
Description
Technical Field
The application relates to the technical field of mobile computing and artificial intelligence, in particular to an electric toothbrush oral cavity cleaning quality detection system and method based on acoustics.
Background
Brushing is an important and fundamental method of protecting dental health, and the quality of brushing has a significant impact on dental health. However, with the continuous development of society, the pace of life of modern urban residents is increasing, and especially in the big cities with tense and full competition lives, no matter students in middle and primary schools or young office workers, the daily dental care belongs to one of the students who neglect their health status to a different extent because of busy daily work and study tasks. In fact, daily brushing is still the most effective method of maintaining oral health compared to rinsing or rinsing teeth, taking into account the frequent occurrence of oral health problems and the consequent high cost of dental care.
However, most people still use the nonstandard tooth brushing mode at present, and the tooth brushing quality is difficult to ensure. Although there are many standard methods of brushing teeth proposed in the oral medical community, there is a lack of effective supervision and feedback when using such methods, and when cleaning the oral cavity, the time, force, coverage, etc. of each oral cavity area is not evaluated quantitatively, especially the inner front gingival surface, the inner side of molars, etc. in the past, the accumulated oral bacteria can produce dental plaque, calculus, caries and more serious oral diseases in these areas. In recent years, in order to solve the problem that the general manual toothbrush is difficult to be thoroughly cleaned, the electric toothbrush has been popularized, which can more thoroughly clean general tooth surfaces, but also in the case that there is no good evaluation and detection of the tooth brushing process, the user easily ignores the inner surface of the tooth. At present, some high-end electric toothbrushes provided with sensors such as IMUs have the function of identifying the tooth cleaning surface, but the manufacturing cost is high, and the price of the high-end electric toothbrushes is 4-5 times higher than that of common electric toothbrushes.
Disclosure of Invention
In order to solve the above technical problems, embodiments of the present application provide an acoustic-based oral cleaning quality detection system and detection method for an electric toothbrush, which construct a wearable device using a low-cost throat microphone and a bluetooth headset based on an acoustic principle, and apply a tooth brushing behavior recognition model provided by the method to a general electric toothbrush on the market by analyzing audio characteristics thereof, so as to detect tooth brushing quality of an application in real time and establish a long-term oral health file thereof.
A first aspect of embodiments of the present application provides an acoustic-based oral cleaning quality detection system for an electric toothbrush, which may include:
the system comprises an electric toothbrush sound signal acquisition module, a multi-channel signal synchronization module, an SIMO system identification module, a feature extraction and dimension reduction module, a tooth brushing behavior classification module and a real-time detection module;
the electric toothbrush sound signal acquisition module is used for acquiring observation signals; the observation signals are generated at the neck and the ear after the acoustic signals generated by the friction between the brush head and the teeth are transmitted through facial tissues;
the multichannel signal synchronization module is used for synchronizing and aligning time delay signals acquired by the throat microphone and the Bluetooth headset;
the SIMO system identification module is used for calculating the frequency domain response function of the oral cavity system corresponding to each tooth surface after acquiring the synchronous observation signals and extracting the stable characteristics of the tooth surfaces;
the feature extraction and dimension reduction module is used for carrying out dimension reduction processing on the feature after the frequency domain response function and the MFCC are fused to form a fused dimension reduction feature;
the tooth brushing behavior classification module is used for comparing and using various machine learning models, classifying and identifying the tooth brushing behaviors corresponding to the feature sets, and preferably selecting the model with better accuracy and calculation performance to be integrated into real-time detection application;
and the real-time detection module is used for identifying the tooth brushing behavior based on the fusion dimensionality reduction characteristics.
Further, the electric toothbrush sound signal acquisition module comprises:
the two-channel throat microphone module comprises two microphones as signal acquisition hardware, wherein one microphone is close to the skin and used for acquiring an observation signal generated by an acoustic signal generated by the friction between the toothbrush head and the surface of the tooth after being transmitted through an esophagus, and the other microphone faces the outside and is used for acquiring the signal difference of the rotation of the motor of the toothbrush motor when the toothbrush motor cleans different tooth surfaces;
the Bluetooth headset with the microphone is used for collecting asymmetric brushing field signals and increasing the identification accuracy of tooth surfaces symmetrically distributed on the left side and the right side of the oral cavity;
and the processing module is used for performing sliding window processing on the acquired signals.
Further, the multi-channel signal synchronization module includes:
the correlation calculation module is used for calculating a cross-correlation function of the acquired signals of the throat microphone and the acquired signals of the Bluetooth headset in the same window;
the sampling point acquisition module analyzes the cross-correlation function step by step and finds out the sampling point corresponding to the extreme value closest to the zero point;
and the alignment module is used for shifting the signal with time delay forward to the sampling point number obtained by calculation in the sampling point acquisition module and aligning the signal.
Further, the SIMO system identification module includes:
the idle running event input signal module is used for extracting the first K main frequency components of the idle running sound signal of the electric toothbrush collected in the idle running event and estimating an input signal;
the two-path output module is used for calculating an input self-power spectrum aiming at the throat microphone signal and the Bluetooth headset signal in a sampling window;
and the frequency domain response function module is used for calculating a cross-power spectrum between the idle event input signal module and the two output modules and obtaining frequency domain response function distribution corresponding to the two output signals respectively through Hv estimation.
Further, the feature extraction and dimension reduction module comprises:
the feature fusion module is used for obtaining a frequency domain response vector by utilizing histogram estimation based on the function distribution of the frequency domain response function module and then fusing the frequency domain response vector with the MFCC features of the original audio;
and the characteristic dimension reduction module performs variance analysis on the whole characteristic set by adopting an ANOVA method and performs characteristic dimension reduction so as to meet the calculation requirement of real-time monitoring on the mobile equipment.
Further, the real-time detection module comprises:
the sampling classification module extracts and fuses characteristics for each sampling window signal and classifies the tooth surface through an optimal classifier;
the tooth brushing state display module accumulates the cleaning time and the coverage rate of each tooth surface according to a reference tooth brushing method, and gives a graphic and sound prompt to a user for the tooth surface which is not cleaned in place in application; and scoring feedback is carried out on the tooth brushing according to the cleaning quality of each tooth surface, and the scoring feedback is stored in a cloud server to record the long-term tooth health data of the user.
In a second aspect, the present embodiments provide an acoustic-based method for detecting oral cleaning quality of an electric toothbrush, including:
collecting observation signals and asymmetrical tooth brushing field signals generated at the neck and the ear after being transmitted by facial tissues;
synchronizing and aligning based on the observed signal and the asymmetrical brushing field signal;
acquiring a frequency domain response function of each tooth surface corresponding to an oral cavity system, and extracting stable characteristics of the tooth surfaces;
reducing the dimension of the feature after the frequency domain response function and the MFCC are fused;
and (3) identifying the tooth brushing behavior based on the fusion dimensionality reduction features by using a machine learning algorithm, and forming tooth brushing health data.
Further, collecting observation signals and asymmetrical brushing field signals generated in the neck and ear after transmission through facial tissue comprises:
collecting observation signals generated at the neck and the ear after acoustic signals generated by the friction of the brush head and the teeth are transmitted through facial tissues by using a throat microphone;
the method adopts a Bluetooth earphone with a microphone to collect asymmetrical tooth brushing field signals, and increases the identification accuracy of tooth surfaces symmetrically distributed on the left side and the right side of the oral cavity;
the throat microphone adopts a dual-channel throat microphone, one microphone is close to the skin and used for collecting an observation signal generated by the friction of the toothbrush head and the surface of the tooth after an acoustic signal is transmitted through the esophagus, and the other microphone faces the outside and collects the signal difference of the rotation of the motor of the toothbrush motor when different tooth surfaces are cleaned.
Further, synchronizing and aligning based on the observed signal and the asymmetrical brushing field signal includes:
acquiring a cross-correlation function of a throat microphone acquisition signal and a Bluetooth headset acquisition signal in the same window;
analyzing step by step based on the cross-correlation function to find out the sampling point corresponding to the extreme value closest to the zero point;
and forward shifting the sampling points obtained by calculation in the previous step aiming at the signals with time delay, and aligning the signals.
Further, obtaining a frequency domain response function of the oral cavity system corresponding to each tooth surface, and extracting the stable characteristics of the tooth surface comprises:
extracting the first K main frequency components of an idle sound signal of the electric toothbrush collected in the idle event, and estimating an input signal;
calculating an input self-power spectrum for the throat microphone signal and the Bluetooth headset signal in a sampling window;
acquiring a cross-power spectrum between an input signal and two paths of outputs;
and acquiring frequency domain response function distribution corresponding to the two output signals respectively through Hv estimation.
In the embodiment of the application, the scheme provides an acoustic-based tooth brushing behavior recognition monitoring method, wearable equipment is constructed by using a low-cost throat microphone and a Bluetooth earphone, acoustic signals generated by an electric toothbrush can be asymmetrically acquired and analyzed, and the accuracy of recognizing bilaterally symmetrical oral cavity regions is improved; the scheme is based on a modeling method of the SIMO system, different oral cavity regions are regarded as different linear time-invariant systems, characteristics are extracted from frequency domain response functions of the systems, compared with statistical characteristics, the stability is higher, and the systems corresponding to tooth surfaces can be effectively distinguished when disturbance changes exist in the electric quantity or gears of the electric toothbrush; the dimension reduction processing is carried out on the original data by a variance analysis method, the calculated amount is effectively reduced, and the endurance time of the mobile smart phone is prolonged; according to the scheme, different machine learning models and different window lengths and step length settings are tested through comprehensive comparison and experiments, the conditions of the optimal accuracy and the lower calculated amount are considered, the optimal machine learning model is selected, parameters such as the corresponding window and step length are integrated, and the detection precision of real-time tooth brushing behavior recognition application is improved.
Drawings
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present application, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.
FIG. 1 is a schematic flow chart of the detection method of the present invention;
FIG. 2 is a flow chart of the present invention for brushing behavior recognition;
FIG. 3 is a schematic representation of the tooth surface division of the present invention;
FIG. 4 is a schematic view of the wearing of the multichannel acoustic signal acquisition device of the present invention;
FIG. 5 is a schematic diagram of the contribution to recognition accuracy under different sensor combinations;
FIG. 6 is a schematic diagram showing the effect on recognition accuracy at different window lengths and step sizes;
FIG. 7(a) is a schematic diagram of the throat microphone signal of the present invention;
FIG. 7(b) is a signal diagram of a Bluetooth headset of the present invention;
FIG. 7(c) is a schematic diagram of the synchronization of the throat microphone signal and the Bluetooth headset signal of the present invention;
FIGS. 8(a) - (b) are schematic diagrams of frequency domain response function distributions of different tooth surface correspondence systems according to the present invention;
FIG. 9 is a comparison of classifier model performance for the present invention;
FIGS. 10(a) - (c) are schematic diagrams of confusion matrices for identifying different tooth surfaces according to the present invention.
Detailed Description
In order to make the purpose, features and advantages of the present application more obvious and understandable, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the embodiments described below are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
The invention is further elucidated with reference to the drawings and the embodiments.
In the description of the present application, it is to be understood that the terms "upper", "lower", "top", "bottom", "inner", "outer", and the like, indicate orientations or positional relationships based on the orientations or positional relationships shown in the drawings, are only for convenience in describing the present application and simplifying the description, and do not indicate or imply that the referenced devices or elements must have a particular orientation, be constructed and operated in a particular orientation, and thus, are not to be construed as limiting the present application.
The invention aims to identify the tooth brushing behavior of a user of an electric toothbrush so as to achieve the purposes of detecting and ensuring the tooth brushing quality of the user and monitoring the long-term tooth brushing quality, and the invention is composed of an electric toothbrush sound signal acquisition module, a multi-channel signal synchronization module, a SIMO system identification module, a feature extraction and dimension reduction module, a tooth brushing behavior classification module and a real-time detection module, wherein the flow chart of the system is shown in figure 2.
In the present application, the electric toothbrush sound signal collecting module is used for collecting observation signals, wherein the observation signals are generated at the neck and the ear after the sound signals generated by the friction between the brush head and the teeth are transmitted through facial tissues.
Electric toothbrush sound signal acquisition module includes:
the dual-channel throat microphone module comprises two microphones as signal acquisition hardware, wherein one microphone is close to the skin and used for acquiring observation signals of sound signals generated by friction between the toothbrush head and the surface of teeth after being transmitted through an esophagus, and the other microphone faces the outside and is used for acquiring the signal difference of the rotation of the motor of the toothbrush motor when different teeth are cleaned.
The Bluetooth earphone with the microphone is used for collecting asymmetric brushing field signals and increasing the identification accuracy of tooth surfaces symmetrically distributed on the left side and the right side of the oral cavity.
And the processing module is used for performing sliding window processing on the acquired signals and performing smoothing processing.
In the application, the multichannel signal synchronization module is mainly used for synchronizing and aligning signals collected by the throat microphone and the Bluetooth headset, and eliminating the influence of signal time delay between channels on feature extraction and tooth surface identification. The method specifically comprises the following steps:
and the correlation calculation module is used for calculating a cross-correlation function of the acquired signals of the throat microphone and the acquired signals of the Bluetooth headset in the same window.
And the sampling point acquisition module analyzes the cross-correlation function step by step and finds the sampling point corresponding to the extreme value closest to the zero point.
And the alignment module is used for shifting the signal with time delay forward to the sampling point number obtained by calculation in the sampling point acquisition module and aligning the signal.
In the present application, the SIMO system recognition module regards each oral cavity part through which the electric toothbrush motor signal is transmitted to the throat microphone and the bluetooth headset as a linear time invariant system, and extracts its stable characteristics. The method specifically comprises the following steps:
the idle running event input signal module is used for extracting the first K main frequency components of the idle running sound signal of the electric toothbrush collected in the idle running event and estimating an input signal;
the two-path output module is used for calculating an input self-power spectrum aiming at the throat microphone signal and the Bluetooth headset signal in a sampling window;
and the frequency domain response function module is used for calculating a cross-power spectrum between the idle event input signal module and the two output modules and obtaining frequency domain response function distribution corresponding to the two output signals respectively through Hv estimation.
In the application, the feature extraction and dimension reduction module is configured to perform dimension reduction processing on the feature after the frequency domain response function and the MFCC are fused, so as to form a fused dimension reduction feature. The method specifically comprises the following steps:
the feature fusion module is used for obtaining a frequency domain response vector by utilizing histogram estimation based on the function distribution of the frequency domain response function module and then fusing the frequency domain response vector with the MFCC features of the original audio;
and the characteristic dimension reduction module performs variance analysis on the whole characteristic set by adopting an ANOVA method and performs characteristic dimension reduction so as to meet the calculation requirement of real-time monitoring on the mobile equipment.
In the application, the tooth brushing behavior classification module is used for comparing and using various machine learning models and classifying and identifying the tooth brushing behaviors corresponding to the feature sets, and preferably, the models with better accuracy and calculation performance are integrated into real-time detection application;
in the application, the real-time detection module identifies the tooth brushing behavior based on the fusion dimensionality reduction features. The method specifically comprises the following steps:
the sampling classification module extracts and fuses characteristics for each sampling window signal and classifies the tooth surface through an optimal classifier;
the tooth brushing state display module accumulates the cleaning time and the coverage rate of each tooth surface according to a reference tooth brushing method, and gives a graphic and sound prompt to a user for the tooth surface which is not cleaned in place in application; and scoring feedback is carried out on the tooth brushing according to the cleaning quality of each tooth surface, and the scoring feedback is stored in a cloud server to record the long-term tooth health data of the user.
When this application specifically gathers the signal, uses the asymmetric binary channels larynx microphone after the repacking, combines bluetooth headset's microphone, constitutes wearable acoustic signal collection system, gathers the multichannel signal after the source signal propagates through different transfer paths when clean of electric toothbrush. According to the baseline brushing method, the oral cavity is divided into 16 different zones, as shown in fig. 3, while increasing the rinse event and the electric toothbrush idle event for 18 events to be identified. Regarding audio signals collected by different tooth surfaces, the corresponding tooth surfaces are regarded as different linear time-invariant systems, a frequency domain response function of each system is calculated by using a system modeling mode, the stable characteristics of the systems are extracted, and the difference between the systems is excavated by using a machine learning algorithm, so that the tooth brushing behavior of a user can be identified.
The embodiment of the application also provides an acoustic-based electric toothbrush oral cavity cleaning quality detection method, which is used for being embedded in any one of the detection systems.
The invention provides an electric toothbrush oral cavity cleaning quality detection method based on acoustics, which specifically comprises the following steps:
s101: and acquiring and labeling multi-channel audio data.
When the marked data are collected and the real-time detection is carried out, the throat microphone is connected with the mobile phone through the 3.5mm audio interface and is worn on the neck; the bluetooth headset establishes wireless connection with the mobile phone through bluetooth and is worn on one side ear of the user, as shown in schematic diagram 4.
In order to construct a universal tooth brushing behavior recognition model, tooth brushing acoustic signals of a plurality of volunteers are collected, and acoustic signals corresponding to all tooth surfaces are labeled by using a labeling tool matched with real-time monitoring application to form an original training data set.
S102: in the real-time detection process, the acoustic signals collected by the throat microphone and the Bluetooth headset are finally transmitted to the mobile phone through two different links, and the respective propagation delays are different; and the problem of asynchronous starting of the signal acquisition threads of the throat microphone and the Bluetooth headset respectively is solved, the signal time difference between the Bluetooth headset and the throat microphone is caused by the two points, and if two paths of signals with the time difference are directly analyzed, the more serious characteristic extraction deviation and identification error can be caused. Therefore, the method uses a mode based on the cross-correlation function to synchronously process the signals acquired in real time. The signal delay is determined by calculating the offset of the extreme points of the cross-correlation function between the multichannel signals. The method specifically comprises the following steps:
and S102, 102A, calculating the cross-correlation function of the acquired signals of the laryngeal microphone and the Bluetooth headset in the same window.
S102B, analyzing the cross-correlation function step by step, and finding out the sampling point corresponding to the extreme value closest to the zero point.
And S102C, shifting the time-delayed signal forwards by the number of the sampling points calculated in the step S102B, and aligning the signal.
S103: and carrying out system modeling and feature fusion on the oral cavity region. In order to identify the tooth surfaces that the user is cleaning, the present invention uses a method based on SIMO system modeling, considering the facial tissues corresponding to 16 regions of the oral cavity as respective different linear time-invariant systems. The sound signal generated by the rotation of the motor of the electric toothbrush is used as the input signal of the system, the observation signals at the throat microphone and the Bluetooth earphone are used as the output signal of the system, and the frequency domain response function (FRF) of the system is worked out by digging the transformation relation between the input and the output so as to represent the stable characteristics of the oral cavity area. The method comprises the following specific steps:
S103A, extracting the first K main frequency components of the idle running sound signals of the electric toothbrush collected in the idle running event, and estimating input signals;
S103B, calculating an input self-power spectrum for the throat microphone signal and the Bluetooth headset signal in a sampling window;
S103C, calculating the cross power spectrum between the input signal and the two paths of outputs;
and S103D, obtaining frequency domain response function distribution corresponding to the two output signals respectively through Hv estimation.
S104: after fusion of FRF distribution of the two channels and MFCC characteristics of original audio, dimension reduction is carried out on the characteristics by using an analysis of variance method in order to guarantee calculation real-time performance on mobile equipment, and finally tooth surface classification and tooth brushing behavior recognition are carried out by using a feature descriptor set after dimension reduction. The method specifically comprises the following steps:
S104A, performing histogram estimation on the frequency domain response function obtained in S103D to obtain a frequency domain response vector;
S104B, fusing frequency domain response vectors respectively corresponding to the throat microphone and the Bluetooth headset with the MFCC characteristics of the original audio;
and S104C, performing variance analysis on the whole feature set by using an ANOVA method and performing feature dimension reduction to meet the calculation requirement of real-time monitoring on the mobile equipment.
S105: and (5) constructing a tooth brushing behavior recognition model and monitoring in real time.
Comparing and optimizing a plurality of machine learning classifier models for the feature descriptors of each oral cavity position acquired in the step S104C, and selecting the model with the most suitable accuracy and calculation complexity to integrate into the real-time detection application. In the process of detecting the tooth brushing behavior of the user in real time, for each sampling window signal, the characteristics are extracted and fused through the method of the steps S102-104, and the tooth surface is classified through an optimal classifier. After the tooth surfaces brushed at each moment are obtained, the cleaning time and the coverage rate of each tooth surface are accumulated according to a standard tooth brushing method, and graphic and sound prompts are given to a user for the tooth surfaces which are not cleaned in place in application. After the process of brushing teeth each time is finished, scoring feedback is carried out on the brushing teeth according to the cleaning quality of each tooth surface, and the scoring feedback is stored in the cloud server to record the long-term tooth health data of the user. The method specifically comprises the following steps:
S105A: various machine learning models are used for comparison, the tooth brushing behaviors corresponding to the feature sets are classified and identified, and models with better accuracy and calculation performance are preferably integrated into real-time detection application;
S105B: during real-time detection, for each sampling window signal, extracting and fusing characteristics through the processes of the steps 2-6, and classifying the tooth surface through an optimal classifier;
S105C: the cleaning time and the coverage rate of each tooth surface are accumulated according to a standard tooth brushing method, and graphic and sound prompts are given to a user for the tooth surfaces which are not cleaned in place in application;
S105D: and scoring feedback is carried out on the tooth brushing according to the cleaning quality of each tooth surface, and the scoring feedback is stored in a cloud server to record the long-term tooth health data of the user.
According to the method for detecting the oral cleaning quality of the electric toothbrush based on the acoustics, the brushing behavior of a user of the electric toothbrush is identified, so that the purposes of detecting and ensuring the brushing quality of the user and monitoring the brushing quality for a long time are achieved.
The present application will be described with reference to specific examples.
Firstly, the traditional two-channel throat microphone is modified, one microphone is close to the skin, so that an observation signal generated by the friction of a toothbrush head and the surface of teeth after an acoustic signal is transmitted through an esophagus can be collected, the other microphone faces the outside, and the signal difference of the rotation of the motor of the toothbrush motor when different tooth surfaces are cleaned is mainly collected. Data of the two microphones are fused into single-channel audio data through hardware, and then combined with the microphone of the Bluetooth headset to construct a wearable acoustic signal acquisition system, wherein contributions of acoustic signals acquired by the channels to the identification accuracy are shown in fig. 5. According to the basic tooth brushing method, the oral cavity is divided into 16 different regions, wherein the upper (denoted by U) and lower (denoted by D) incisor regions each include an outer region (denoted by O) and an inner region (denoted by I), the four posterior tooth regions each include an inner side, an outer side and a chewing surface, and the left and right tooth surfaces are divided by L and R for a total of 16 regions. At the same time, a rinse event is added to improve the robustness of the identification method in distinguishing between brushing and non-brushing events; and an idle event of the electric toothbrush, calculating and rectifying the input signal of the system by extracting its main frequency. Regarding audio signals collected by different tooth surfaces, the corresponding tooth surfaces are regarded as different linear time-invariant systems, a frequency domain response function of each system is calculated by using a system modeling mode, the stable characteristics of the systems are extracted, and the difference between the systems is excavated by using a machine learning algorithm, so that the tooth brushing behavior of a user can be identified.
The detection method specifically comprises the following steps:
step 1: and acquiring and labeling multi-channel audio data.
When the marked data are collected and the real-time detection is carried out, the throat microphone is connected with the mobile phone through the 3.5mm audio interface and is worn on the neck; the Bluetooth headset is in wireless connection with the mobile phone through Bluetooth and is worn on one side ear of a user. The collected signal is divided into two channels of a microphone and a Bluetooth headset by using a 44100Hz sampling rate and a 16bit width. For further acoustic signal processing, the method sets the sliding window sizes of the sampling points of 128, 512, 1024, 2048, 4096, 8192 and 10240 and the step sizes of 30%, 50%, 70% and 100%, and selects the optimal size and step size to implement in real-time monitoring, wherein the performances of different combinations are shown in fig. 6. In order to construct a universal tooth brushing behavior recognition model, tooth brushing acoustic signals of a plurality of volunteers are collected, and acoustic signals corresponding to all tooth surfaces are labeled by using a labeling tool matched with real-time monitoring application to form an original training data set.
Step 2: the audio signals are synchronized.
In the real-time detection process, the signal of the throat microphone is transmitted to the mobile phone by a link of a microphone-audio line-3.5 mm interface; signals collected by the Bluetooth earphone are transmitted to the mobile phone by a microphone-DSP coding-Bluetooth wireless transmission link, and the propagation time delay of the two links is different; and the respective signal acquisition threads of the throat microphone and the Bluetooth headset have the problem of asynchronous starting. The two points cause signal time difference between the Bluetooth headset and the throat microphone, and if two paths of signals with the time difference are directly analyzed, more serious characteristic extraction deviation and identification errors can be caused. Therefore, the method uses a mode based on the cross-correlation function to synchronously process the signals acquired in real time. And calculating the distribution of the cross-correlation function of the Bluetooth and Hour signals in a window, selecting a sampling point corresponding to an extreme value point closest to the point 0 in the cross-correlation coefficient distribution as the offset sampling point number of the Bluetooth signal and the Hour signal, and shifting the delayed signal forwards by the corresponding sampling point number to form a synchronized dual-channel signal. The schematic diagram of the synchronization method is shown in fig. 7, and for two signals in the diagram, the cross-correlation function reaches an extreme value at 3710 sampling points, which indicates that the acquired signal of the headset lags behind the acquired signal of the bluetooth headset by 3710 sampling points.
And step 3: the oral region is modeled systematically.
In order to identify the tooth surfaces that the user is cleaning, the present invention uses a method based on SIMO system modeling, considering the facial tissues corresponding to 16 regions of the oral cavity as respective different linear time-invariant systems. Because the electric toothbrush has the characteristic of stable rotation, a signal generated by the rotation of a motor of the electric toothbrush can be regarded as the input of the system; after the signal passes through a certain specific transmission path and reaches the Bluetooth and throat microphone observation points, the observation signal is taken as an output signal of the system, and the system corresponding to the path can cause the signal to generate specific change, and the transmission paths of the signal are different for different tooth surfaces. For example, when brushing the inner surface of the left posterior tooth, the signal transmission paths to the two observation points are respectively a tooth-inside-jaw-inside-esophagus-throat microphone left side microphone, a tooth-inside-outside-jaw-air-throat microphone right side microphone, and a tooth-right oral-right side face-bluetooth headset microphone, and when brushing the outer surface of the right posterior tooth, the input signal is transmitted through different paths. Since the user's facial structure is relatively stable, the features obtained by system modeling have greater stability than statistical features. When the rotating speed of the electric toothbrush is reduced due to low electric quantity or the rotating mode is changed due to gear switching, the input signal of the system is changed, and the statistical characteristic mode of the output signal is greatly changed, so that different tooth surfaces cannot be distinguished; the facial structure system corresponding to the input-output signals is not changed, and the characteristics of the system model still have good discrimination for each tooth surface.
The invention obtains the oral system model parameters by the following modes: firstly, extracting the first K main frequency components of an idle sound signal of the electric toothbrush collected in an idle event, and estimating an input signal I; then, for the throat microphone signal T and the Bluetooth headset signal B in a sampling window, calculating an input self-power spectrum GTAnd GB(ii) a Calculating the cross-power spectrum G between the input and the two outputsIT,GIB(ii) a And finally, obtaining frequency domain response function (FRF) distribution corresponding to the two output signals respectively through Hv estimation, wherein the frequency domain response function (FRF) distribution is used as a descriptor of the oral system parameters. There is a better distinction between FRF distributions corresponding to different oral positions, as shown in fig. 8.
And 4, step 4: and extracting and fusing system characteristics.
The whole feature set consists of three parts: FRF distribution of the input-throat microphone system, FRF distribution of the input-bluetooth headset system, and 12-orderfmcc characteristics of the raw output signal. MFCC features are added to the feature set to increase feature richness. After the histogram estimation is carried out on the FRF distribution respectively corresponding to the larynx microphone and the Bluetooth headset, three parts of features are fused to form a complete feature descriptor of a sampling window. And performing variance analysis on all the feature sets by using an ANOVA method so as to perform feature dimension reduction to meet the calculation requirement of real-time detection, and finally retaining M-dimensional features, wherein the M value is optimized according to the results of modeling and testing on the calibration data set.
And 5: and (5) constructing a tooth brushing behavior recognition model and monitoring in real time.
For the feature descriptors of each oral cavity position acquired in step 4, compared with the classification results of the features by using classifiers such as XGBoost, SVM, RandomForest, Bagging, NaiveBayes, deep neural network and the like, the accuracy, recall rate and F1 value performance metrics are as shown in fig. 9 (black vertical lines in the figure represent error bars corresponding to histograms), and finally, an XGBoost model with higher accuracy and lower computational complexity than the deep neural network is integrated into real-time detection application. In the process of detecting the tooth brushing behavior of the user in real time, for each sampling window signal, the characteristics are extracted and fused through the method of the steps 2-4, the tooth surfaces are classified through the optimal classifier, and a confusion matrix (reflecting the evaluation quality and the wrong classification of the actual value by the detector) for wrong classification of different tooth surfaces is shown in fig. 10. After the tooth surfaces brushed at each moment are obtained, the cleaning time and the coverage rate of each tooth surface are accumulated according to a standard tooth brushing method, and graphic and sound prompts are given to a user for the tooth surfaces which are not cleaned in place in application. After the process of brushing teeth each time is finished, scoring feedback is carried out on the brushing teeth according to the cleaning quality of each tooth surface, and the scoring feedback is stored in the cloud server to record the long-term tooth health data of the user.
Although the preferred embodiments of the present invention have been described in detail, the present invention is not limited to the details of the foregoing embodiments, and various equivalent changes (such as number, shape, position, etc.) may be made to the technical solution of the present invention within the technical spirit of the present invention, and the equivalents are protected by the present invention.
Claims (10)
1. An electric toothbrush oral cavity cleaning quality detection system based on acoustics is characterized by comprising an electric toothbrush acoustic signal acquisition module, a multi-channel signal synchronization module, a SIMO system identification module, a feature extraction and dimension reduction module, a tooth brushing behavior classification module and a real-time detection module;
the electric toothbrush sound signal acquisition module is used for acquiring observation signals; the observation signals are generated at the neck and the ear after acoustic signals generated by the friction between the brush head and the teeth are transmitted through facial tissues;
the multichannel signal synchronization module is used for synchronizing and aligning time delay signals acquired by the throat microphone and the Bluetooth headset;
the SIMO system identification module is used for calculating the frequency domain response function of the oral cavity system corresponding to each tooth surface after acquiring the synchronous observation signals and extracting the stable characteristics of the tooth surfaces;
the feature extraction and dimension reduction module is used for carrying out dimension reduction processing on the feature after the frequency domain response function and the MFCC are fused to form a fused dimension reduction feature;
the tooth brushing behavior classification module is used for performing classification and identification on tooth brushing behaviors corresponding to the feature set by comparing and using various machine learning models, and preferably, models with better accuracy and calculation performance are integrated into real-time detection application;
and the real-time detection module identifies the tooth brushing behavior based on the fusion dimensionality reduction characteristics.
2. The acoustic-based electric toothbrush oral cleaning quality detection system of claim 1, wherein the electric toothbrush acoustic signal acquisition module comprises:
the two-channel throat microphone module comprises two microphones as signal acquisition hardware, wherein one microphone is close to the skin and used for acquiring an observation signal generated by an acoustic signal generated by the friction between the toothbrush head and the surface of the tooth after being transmitted through an esophagus, and the other microphone faces the outside and is used for acquiring the signal difference of the rotation of the motor of the toothbrush motor when the toothbrush motor cleans different tooth surfaces;
the Bluetooth headset with the microphone is used for collecting asymmetric brushing field signals and increasing the identification accuracy of tooth surfaces symmetrically distributed on the left side and the right side of the oral cavity;
and the processing module is used for performing sliding window processing on the acquired signals.
3. The acoustic-based power toothbrush oral cleaning quality detection system of claim 2, wherein the multi-channel signal synchronization module comprises:
the correlation calculation module is used for calculating a cross-correlation function of the acquired signals of the throat microphone and the acquired signals of the Bluetooth headset in the same window;
the sampling point acquisition module analyzes the cross-correlation function step by step and finds out the sampling point corresponding to the extreme value closest to the zero point;
and the alignment module is used for shifting the signal with time delay forward to the sampling point number obtained by calculation in the sampling point acquisition module and aligning the signal.
4. The acoustic-based power toothbrush oral cleaning quality detection system of claim 3, wherein the SIMO system identification module comprises:
the idle running event input signal module is used for extracting the first K main frequency components of the idle running sound signal of the electric toothbrush collected in the idle running event and estimating an input signal;
the two-path output module is used for calculating an input self-power spectrum aiming at the throat microphone signal and the Bluetooth headset signal in a sampling window;
and the frequency domain response function module is used for calculating a cross-power spectrum between the idle event input signal module and the two output modules and obtaining frequency domain response function distribution corresponding to the two output signals respectively through Hv estimation.
5. The acoustic-based power toothbrush oral cleaning quality detection system of claim 4, wherein the feature extraction and dimension reduction module comprises:
the feature fusion module is used for obtaining a frequency domain response vector by utilizing histogram estimation based on the function distribution of the frequency domain response function module and then fusing the frequency domain response vector with the MFCC features of the original audio;
and the characteristic dimension reduction module performs variance analysis on the whole characteristic set by adopting an ANOVA method and performs characteristic dimension reduction so as to meet the calculation requirement of real-time monitoring on the mobile equipment.
6. The acoustic-based power toothbrush oral cleaning quality detection system of claim 5, wherein the real-time detection module comprises:
the sampling classification module extracts and fuses characteristics for each sampling window signal and classifies the tooth surface through an optimal classifier;
the tooth brushing state display module accumulates the cleaning time and the coverage rate of each tooth surface according to a reference tooth brushing method, and gives a graphic and sound prompt to a user for the tooth surface which is not cleaned in place in application; and scoring feedback is carried out on the tooth brushing according to the cleaning quality of each tooth surface, and the scoring feedback is stored in a cloud server to record the long-term tooth health data of the user.
7. A method of testing the acoustic-based electric toothbrush oral cleaning quality detection system of any of claims 1-6, comprising:
collecting observation signals and asymmetrical tooth brushing field signals generated at the neck and the ear after being transmitted by facial tissues;
synchronizing and aligning based on the observed signal and the asymmetrical brushing field signal;
acquiring a frequency domain response function of each tooth surface corresponding to an oral cavity system, and extracting stable characteristics of the tooth surfaces;
reducing the dimension of the feature after the frequency domain response function and the MFCC are fused;
and (3) identifying the tooth brushing behavior based on the fusion dimensionality reduction features by using a machine learning algorithm, and forming tooth brushing health data.
8. The acoustic-based quality of oral cleaning with an electric toothbrush of claim 7, wherein the collecting of the observation signals and the asymmetrical brushing field signals generated in the neck and ear after the transmission through the facial tissue comprises:
collecting observation signals generated at the neck and the ear after acoustic signals generated by the friction of the brush head and the teeth are transmitted through facial tissues by using a throat microphone;
the method adopts a Bluetooth earphone with a microphone to collect asymmetrical tooth brushing field signals, and increases the identification accuracy of tooth surfaces symmetrically distributed on the left side and the right side of the oral cavity;
the throat microphone is a dual-channel throat microphone, one microphone is close to the skin and used for collecting observation signals generated by the friction of the toothbrush head and the surface of teeth after acoustic signals are transmitted through the esophagus, and the other microphone faces the outside and collects the signal difference of the rotation of the motor of the toothbrush motor when different tooth surfaces are cleaned.
9. The acoustic-based power toothbrush oral cleaning quality detection method of claim 7, wherein the synchronizing and aligning based on the observed signal and the asymmetric brushing field signal comprises:
acquiring a cross-correlation function of a throat microphone acquisition signal and a Bluetooth headset acquisition signal in the same window;
analyzing step by step based on the cross-correlation function to find out the sampling point corresponding to the extreme value closest to the zero point;
and forward shifting the sampling points obtained by calculation in the previous step aiming at the signals with time delay, and aligning the signals.
10. The acoustic-based method for detecting oral cleaning quality of an electric toothbrush according to claim 7, wherein the obtaining of the frequency domain response function of the oral system corresponding to each tooth surface and the extracting of the stable characteristics of the tooth surface comprises:
extracting the first K main frequency components of an idle sound signal of the electric toothbrush collected in the idle event, and estimating an input signal;
calculating an input self-power spectrum for the throat microphone signal and the Bluetooth headset signal in a sampling window;
acquiring a cross-power spectrum between an input signal and two paths of outputs;
and acquiring frequency domain response function distribution corresponding to the two output signals respectively through Hv estimation.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011033308.8A CN112213134B (en) | 2020-09-27 | 2020-09-27 | Electric toothbrush oral cavity cleaning quality detection system and detection method based on acoustics |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011033308.8A CN112213134B (en) | 2020-09-27 | 2020-09-27 | Electric toothbrush oral cavity cleaning quality detection system and detection method based on acoustics |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112213134A true CN112213134A (en) | 2021-01-12 |
| CN112213134B CN112213134B (en) | 2022-09-27 |
Family
ID=74051977
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011033308.8A Active CN112213134B (en) | 2020-09-27 | 2020-09-27 | Electric toothbrush oral cavity cleaning quality detection system and detection method based on acoustics |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112213134B (en) |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107167133A (en) * | 2017-05-24 | 2017-09-15 | 北京慧联科技有限公司 | A kind of brushing evaluation method and system |
| CN108354315A (en) * | 2017-12-05 | 2018-08-03 | 牛家祺 | A kind of brush teeth quality detecting system and method based on the asymmetric sound field of double units |
| CN109172016A (en) * | 2018-07-11 | 2019-01-11 | 上海斐讯数据通信技术有限公司 | A kind of brush teeth points-scoring system and method based on intelligent electric toothbrush |
| CN109700170A (en) * | 2018-12-28 | 2019-05-03 | 北京小米移动软件有限公司 | Method, system, electronic equipment and the storage medium that data report of brushing teeth generates |
| CN210541899U (en) * | 2019-05-20 | 2020-05-19 | 福建美丽宝科技有限公司 | Intelligent tooth cleaner with monitoring device |
| CN111189572A (en) * | 2020-01-13 | 2020-05-22 | 深圳市东至西科技有限公司 | Electric toothbrush scouring force detection circuit and detection method |
| CN111397664A (en) * | 2020-04-22 | 2020-07-10 | 好维股份有限公司 | Test equipment for testing durability and cleanness of toothbrush bristles |
| US20200268141A1 (en) * | 2019-02-27 | 2020-08-27 | The Procter & Gamble Company | Voice Assistant in an Electric Toothbrush |
| CN211433432U (en) * | 2019-09-09 | 2020-09-08 | 佛山市知而行信息科技有限公司 | Electric toothbrush capable of detecting breath |
-
2020
- 2020-09-27 CN CN202011033308.8A patent/CN112213134B/en active Active
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107167133A (en) * | 2017-05-24 | 2017-09-15 | 北京慧联科技有限公司 | A kind of brushing evaluation method and system |
| CN108354315A (en) * | 2017-12-05 | 2018-08-03 | 牛家祺 | A kind of brush teeth quality detecting system and method based on the asymmetric sound field of double units |
| CN109172016A (en) * | 2018-07-11 | 2019-01-11 | 上海斐讯数据通信技术有限公司 | A kind of brush teeth points-scoring system and method based on intelligent electric toothbrush |
| CN109700170A (en) * | 2018-12-28 | 2019-05-03 | 北京小米移动软件有限公司 | Method, system, electronic equipment and the storage medium that data report of brushing teeth generates |
| US20200268141A1 (en) * | 2019-02-27 | 2020-08-27 | The Procter & Gamble Company | Voice Assistant in an Electric Toothbrush |
| CN210541899U (en) * | 2019-05-20 | 2020-05-19 | 福建美丽宝科技有限公司 | Intelligent tooth cleaner with monitoring device |
| CN211433432U (en) * | 2019-09-09 | 2020-09-08 | 佛山市知而行信息科技有限公司 | Electric toothbrush capable of detecting breath |
| CN111189572A (en) * | 2020-01-13 | 2020-05-22 | 深圳市东至西科技有限公司 | Electric toothbrush scouring force detection circuit and detection method |
| CN111397664A (en) * | 2020-04-22 | 2020-07-10 | 好维股份有限公司 | Test equipment for testing durability and cleanness of toothbrush bristles |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112213134B (en) | 2022-09-27 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN104765952B (en) | One kind is brushed teeth attitude detection and evaluation system | |
| JP7105318B2 (en) | Tooth brushing recommendation method, device, device, and storage medium based on artificial intelligence | |
| CN108354315B (en) | A kind of brush teeth quality detecting system and method based on the asymmetric sound field of double units | |
| CN107832719B (en) | Tooth brushing analysis method and device and tooth brushing system | |
| CN105590007B (en) | The analysis method and analysis system for posture of brushing teeth | |
| CN106388771B (en) | A kind of method and motion bracelet of automatic detection human body physiological state | |
| Vaidyanathan et al. | Tongue-movement communication and control concept for hands-free human–machine interfaces | |
| US20200178883A1 (en) | Method and system for articulation evaluation by fusing acoustic features and articulatory movement features | |
| Huang et al. | Toothbrushing monitoring using wrist watch | |
| EP3777761A1 (en) | Smart toothbrush-based tooth-brushing evaluation method, apparatus, device and storage medium | |
| CN110265135A (en) | A kind of stamping quality testing assessment system and method based on inertial sensor | |
| CN108805390B (en) | Intelligent tooth brushing process evaluation method | |
| CN104142583A (en) | Intelligent glasses with blinking detection function and implementation method thereof | |
| Akther et al. | mORAL: An mHealth model for inferring oral hygiene behaviors in-the-wild using wrist-worn inertial sensors | |
| CN112107386A (en) | System and method for collecting and processing user health data | |
| WO2016082784A1 (en) | Child teeth brushing smart training system | |
| CN112184010A (en) | Tooth brushing effect evaluation method, device, system, electronic device and storage medium | |
| CN108958482A (en) | A kind of similitude action recognition device and method based on convolutional neural networks | |
| Ouyang et al. | An asymmetrical acoustic field detection system for daily tooth brushing monitoring | |
| CN114358194A (en) | Gesture tracking based detection method for abnormal limb behaviors of autism spectrum disorder | |
| CN112213134B (en) | Electric toothbrush oral cavity cleaning quality detection system and detection method based on acoustics | |
| CN108919962B (en) | Auxiliary piano training method based on brain-computer data centralized processing | |
| CN109271889A (en) | A kind of action identification method based on the double-deck LSTM neural network | |
| CN106096544A (en) | Contactless nictation and heart rate combined detection system and method based on second-order blind identification | |
| WO2023127504A1 (en) | Evaluation device, evaluation method, and evaluation program |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |