Disclosure of Invention
In order to solve the above technical problems, embodiments of the present application provide an acoustic-based oral cavity cleaning quality detection system and method for an electric toothbrush, which are based on the acoustic principle, a wearable device using a low-cost throat microphone and a bluetooth headset is constructed, and a tooth brushing behavior recognition model provided by the method is applied to a general electric toothbrush in the market by analyzing the audio characteristics thereof, so that the tooth brushing quality of an application can be detected in real time, and a long-term oral cavity health file can be established.
A first aspect of embodiments of the present application provides an acoustic-based oral cleaning quality detection system for an electric toothbrush, which may include:
the system comprises an electric toothbrush sound signal acquisition module, a multi-channel signal synchronization module, an SIMO system identification module, a feature extraction and dimension reduction module, a tooth brushing behavior classification module and a real-time detection module;
the electric toothbrush sound signal acquisition module is used for acquiring observation signals; the observation signals are generated at the neck and the ear after the acoustic signals generated by the friction between the brush head and the teeth are transmitted through facial tissues;
the multichannel signal synchronization module is used for synchronizing and aligning time delay signals acquired by the throat microphone and the Bluetooth headset;
the SIMO system identification module is used for calculating the frequency domain response function of the oral cavity system corresponding to each tooth surface after acquiring the synchronous observation signals and extracting the stable characteristics of the tooth surfaces;
the feature extraction and dimension reduction module is used for carrying out dimension reduction processing on the feature after the frequency domain response function and the MFCC are fused to form a fusion dimension reduction feature;
the tooth brushing behavior classification module is used for comparing and using various machine learning models, classifying and identifying the tooth brushing behaviors corresponding to the feature sets, and preferably selecting the model with better accuracy and calculation performance to be integrated into real-time detection application;
and the real-time detection module is used for identifying the tooth brushing behavior based on the fusion dimensionality reduction characteristics.
Further, the electric toothbrush sound signal acquisition module comprises:
the two-channel throat microphone module comprises two microphones as signal acquisition hardware, wherein one microphone is close to the skin and used for acquiring an observation signal generated by an acoustic signal generated by the friction between the toothbrush head and the surface of the tooth after being transmitted through an esophagus, and the other microphone faces the outside and is used for acquiring the signal difference of the rotation of the motor of the toothbrush motor when the toothbrush motor cleans different tooth surfaces;
the Bluetooth headset with the microphone is used for collecting asymmetric brushing field signals and increasing the identification accuracy of tooth surfaces symmetrically distributed on the left side and the right side of the oral cavity;
and the processing module is used for performing sliding window processing on the acquired signals.
Further, the multi-channel signal synchronization module includes:
the correlation calculation module is used for calculating a cross-correlation function of the acquired signals of the throat microphone and the acquired signals of the Bluetooth headset in the same window;
the sampling point acquisition module analyzes the cross-correlation function step by step to find out a sampling point corresponding to an extreme value closest to a zero point;
and the alignment module is used for shifting the signal with time delay forward to the sampling point number obtained by calculation in the sampling point acquisition module and aligning the signal.
Further, the SIMO system identification module includes:
the idle running event input signal module is used for extracting the first K main frequency components of the idle running sound signal of the electric toothbrush collected in the idle running event and estimating an input signal;
the two-path output module is used for calculating an input self-power spectrum aiming at the throat microphone signal and the Bluetooth headset signal in a sampling window;
and the frequency domain response function module is used for calculating a cross-power spectrum between the idle event input signal module and the two output modules and obtaining frequency domain response function distribution corresponding to the two output signals respectively through Hv estimation.
Further, the feature extraction and dimension reduction module comprises:
the feature fusion module is used for obtaining a frequency domain response vector by utilizing histogram estimation based on the function distribution of the frequency domain response function module and then fusing the frequency domain response vector with the MFCC features of the original audio;
and the characteristic dimension reduction module performs variance analysis on the whole characteristic set by adopting an ANOVA method and performs characteristic dimension reduction so as to meet the calculation requirement of real-time monitoring on the mobile equipment.
Further, the real-time detection module comprises:
the sampling classification module extracts and fuses characteristics for each sampling window signal and classifies the tooth surface through an optimal classifier;
the tooth brushing state display module accumulates the cleaning time and the coverage rate of each tooth surface according to a reference tooth brushing method, and gives a user graphic and sound prompt aiming at the tooth surfaces which are not cleaned in place in application; grading and feeding back the tooth brushing according to the cleaning quality of each tooth surface, storing the grading and feeding back to the cloud server, and recording the long-term tooth health data of the user.
In a second aspect, the present embodiments provide an acoustic-based method for detecting oral cleaning quality of an electric toothbrush, including:
collecting observation signals and asymmetrical tooth brushing field signals generated at the neck and the ear after being transmitted by facial tissues;
synchronizing and aligning based on the observed signal and the asymmetrical brushing field signal;
acquiring a frequency domain response function of each tooth surface corresponding to an oral cavity system, and extracting stable characteristics of the tooth surfaces;
reducing the dimension of the feature after the frequency domain response function and the MFCC are fused;
and (3) identifying the tooth brushing behavior based on the fusion dimensionality reduction features by using a machine learning algorithm, and forming tooth brushing health data.
Further, collecting observation signals and asymmetric brushing field signals generated in the neck and ears after transmission through facial tissue comprises:
collecting observation signals generated at the neck and the ear after acoustic signals generated by the friction of the brush head and the teeth are transmitted through facial tissues by using a throat microphone;
the method adopts a Bluetooth earphone with a microphone to collect asymmetrical tooth brushing field signals, and increases the identification accuracy of tooth surfaces symmetrically distributed on the left side and the right side of the oral cavity;
the throat microphone is a dual-channel throat microphone, one microphone is close to the skin and used for collecting observation signals of sound signals generated by friction between the toothbrush head and the surface of teeth after being transmitted through the esophagus, and the other microphone faces the outside and collects signal difference of motor rotation of the toothbrush motor when different tooth surfaces are cleaned.
Further, synchronizing and aligning based on the observed signal and the asymmetrical brushing field signal includes:
acquiring a cross-correlation function of a throat microphone acquisition signal and a Bluetooth headset acquisition signal in the same window;
analyzing step by step based on the cross-correlation function to find out a corresponding sampling point of an extreme value closest to a zero point;
and forward shifting the sampling points obtained by calculation in the previous step aiming at the signals with time delay, and aligning the signals.
Further, obtaining a frequency domain response function of the oral cavity system corresponding to each tooth surface, and extracting the stable characteristics of the tooth surface comprises:
extracting the first K main frequency components of an idle sound signal of the electric toothbrush collected in the idle event, and estimating an input signal;
calculating an input self-power spectrum for the throat microphone signal and the Bluetooth headset signal in a sampling window;
acquiring a cross-power spectrum between an input signal and two paths of outputs;
and acquiring frequency domain response function distribution corresponding to the two output signals respectively through Hv estimation.
In the embodiment of the application, the scheme provides an acoustic-based tooth brushing behavior recognition monitoring method, wearable equipment is constructed by using a low-cost throat microphone and a Bluetooth earphone, acoustic signals generated by an electric toothbrush can be asymmetrically acquired and analyzed, and the accuracy of recognizing bilaterally symmetrical oral cavity regions is improved; the scheme is based on a modeling method of the SIMO system, different oral cavity regions are regarded as different linear time-invariant systems, characteristics are extracted from frequency domain response functions of the systems, compared with statistical characteristics, the stability is higher, and the systems corresponding to tooth surfaces can be effectively distinguished when disturbance changes exist in the electric quantity or gears of the electric toothbrush; the dimension reduction processing is carried out on the original data by a variance analysis method, the calculated amount is effectively reduced, and the endurance time of the mobile smart phone is prolonged; according to the scheme, different machine learning models and different window lengths and step length settings are tested through comprehensive comparison and experiments, the conditions of the optimal accuracy and the lower calculated amount are considered, the optimal machine learning model is selected, parameters such as the corresponding window and step length are integrated, and the detection precision of real-time tooth brushing behavior recognition application is improved.
Detailed Description
In order to make the purpose, features and advantages of the present application more obvious and understandable, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the embodiments described below are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
The invention is further elucidated with reference to the drawings and the embodiments.
In the description of the present application, it is to be understood that the terms "upper", "lower", "top", "bottom", "inner", "outer", and the like, indicate orientations or positional relationships based on the orientations or positional relationships shown in the drawings, are only for convenience in describing the present application and simplifying the description, and do not indicate or imply that the referenced devices or elements must have a particular orientation, be constructed and operated in a particular orientation, and thus, are not to be construed as limiting the present application.
The invention aims to identify the tooth brushing behavior of a user of an electric toothbrush so as to achieve the purposes of detecting and ensuring the tooth brushing quality of the user and monitoring the long-term tooth brushing quality, and the invention is composed of an electric toothbrush sound signal acquisition module, a multi-channel signal synchronization module, a SIMO system identification module, a feature extraction and dimension reduction module, a tooth brushing behavior classification module and a real-time detection module, wherein the flow chart of the system is shown in figure 2.
In the present application, the electric toothbrush sound signal collecting module is used for collecting observation signals, wherein the observation signals are generated at the neck and the ear after the sound signals generated by the friction between the brush head and the teeth are transmitted through facial tissues.
The electric toothbrush sound signal acquisition module includes:
the two-channel throat microphone module comprises two microphones used as signal acquisition hardware, wherein one microphone is close to the skin and used for acquiring an observation signal of an acoustic signal generated by the friction between a toothbrush head and the surface of teeth after being transmitted through an esophagus, and the other microphone faces the outside and used for acquiring the signal difference of the rotation of a motor of the toothbrush motor when different tooth surfaces are cleaned.
The Bluetooth earphone with the microphone is used for collecting asymmetric brushing field signals and increasing the identification accuracy of tooth surfaces symmetrically distributed on the left side and the right side of the oral cavity.
And the processing module is used for performing sliding window processing on the acquired signals and performing smoothing processing.
In the application, the multichannel signal synchronization module is mainly used for synchronizing and aligning signals collected by the throat microphone and the Bluetooth headset, and eliminating the influence of signal time delay between channels on feature extraction and tooth surface identification. The method specifically comprises the following steps:
and the correlation calculation module is used for calculating a cross-correlation function of the acquired signals of the throat microphone and the acquired signals of the Bluetooth headset in the same window.
And the sampling point acquisition module analyzes the cross-correlation function step by step and finds the sampling point corresponding to the extreme value closest to the zero point.
And the alignment module is used for shifting the signal with time delay forward to the sampling point number obtained by calculation in the sampling point acquisition module and aligning the signal.
In the present application, the SIMO system recognition module regards each oral cavity part through which the electric toothbrush motor signal is transmitted to the throat microphone and the bluetooth headset as a linear time invariant system, and extracts its stable characteristics. The method specifically comprises the following steps:
the idle running event input signal module is used for extracting the first K main frequency components of the idle running sound signal of the electric toothbrush collected in the idle running event and estimating an input signal;
the two-path output module is used for calculating an input self-power spectrum aiming at the throat microphone signal and the Bluetooth headset signal in a sampling window;
and the frequency domain response function module is used for calculating a cross-power spectrum between the idle event input signal module and the two output modules and obtaining frequency domain response function distribution corresponding to the two output signals respectively through Hv estimation.
In the application, the feature extraction and dimension reduction module is configured to perform dimension reduction processing on the feature after the frequency domain response function and the MFCC are fused, so as to form a fused dimension reduction feature. The method specifically comprises the following steps:
the feature fusion module is used for obtaining a frequency domain response vector by utilizing histogram estimation based on the function distribution of the frequency domain response function module and then fusing the frequency domain response vector with the MFCC features of the original audio;
and the characteristic dimension reduction module performs variance analysis on the whole characteristic set by adopting an ANOVA method and performs characteristic dimension reduction so as to meet the calculation requirement of real-time monitoring on the mobile equipment.
In the application, the tooth brushing behavior classification module is used for carrying out classification and identification on tooth brushing behaviors corresponding to the feature set by comparing and using various machine learning models, and preferably, the models with better accuracy and calculation performance are integrated into real-time detection application;
in the application, the real-time detection module identifies the tooth brushing behavior based on the fusion dimensionality reduction features. The method specifically comprises the following steps:
the sampling classification module extracts and fuses characteristics for each sampling window signal and classifies the tooth surface through an optimal classifier;
the tooth brushing state display module accumulates the cleaning time and the coverage rate of each tooth surface according to a reference tooth brushing method, and gives a graphic and sound prompt to a user for the tooth surface which is not cleaned in place in application; and scoring feedback is carried out on the tooth brushing according to the cleaning quality of each tooth surface, and the scoring feedback is stored in a cloud server to record the long-term tooth health data of the user.
When this application specifically gathers the signal, uses the asymmetric binary channels larynx microphone after the repacking, combines bluetooth headset's microphone, constitutes wearable acoustic signal collection system, gathers the multichannel signal after the source signal propagates through different transfer paths when clean of electric toothbrush. According to the baseline brushing method, the oral cavity is divided into 16 different zones, as shown in fig. 3, while increasing the rinse event and the electric toothbrush idle event for 18 events to be identified. Regarding audio signals collected by different tooth surfaces, the corresponding tooth surfaces are regarded as different linear time-invariant systems, a frequency domain response function of each system is calculated by using a system modeling mode, the stable characteristics of the systems are extracted, and the difference between the systems is excavated by using a machine learning algorithm, so that the tooth brushing behavior of a user can be identified.
The embodiment of the application also provides an acoustic-based electric toothbrush oral cavity cleaning quality detection method, which is used for being embedded in any one of the detection systems.
The invention provides an electric toothbrush oral cavity cleaning quality detection method based on acoustics, which specifically comprises the following steps:
s101: and acquiring and labeling multi-channel audio data.
When the marked data are collected and the real-time detection is carried out, the throat microphone is connected with the mobile phone through the 3.5mm audio interface and is worn on the neck; the bluetooth headset establishes wireless connection with the mobile phone through bluetooth and is worn on one side ear of a user, as shown in an illustration 4.
In order to construct a universal tooth brushing behavior recognition model, tooth brushing acoustic signals of a plurality of volunteers are collected, and acoustic signals corresponding to all tooth surfaces are labeled by using a labeling tool matched with real-time monitoring application to form an original training data set.
S102: in the real-time detection process, the sound signals collected by the microphone and the Bluetooth earphone are finally transmitted to the mobile phone by two different links, and the propagation time delays of the sound signals are different; and the problem of asynchronous starting of the signal acquisition threads of the throat microphone and the Bluetooth headset respectively is solved, the signal time difference between the Bluetooth headset and the throat microphone is caused by the two points, and if two paths of signals with the time difference are directly analyzed, the more serious characteristic extraction deviation and identification error can be caused. Therefore, the method uses a mode based on the cross-correlation function to synchronously process the signals acquired in real time. The signal delay is determined by calculating the offset of the extreme points of the cross-correlation function between the multichannel signals. The method specifically comprises the following steps:
and S102, 102A, calculating the cross-correlation function of the acquired signals of the laryngeal microphone and the Bluetooth headset in the same window.
S102B, analyzing the cross-correlation function step by step, and finding out the sampling point corresponding to the extreme value closest to the zero point.
And S102C, shifting the time-delayed signal forwards by the number of the sampling points calculated in the step S102B, and aligning the signal.
S103: and carrying out system modeling and feature fusion on the oral cavity region. In order to identify the tooth surfaces that the user is cleaning, the present invention uses a method based on SIMO system modeling, considering the facial tissues corresponding to 16 regions of the oral cavity as respective different linear time-invariant systems. The sound signal generated by the rotation of the motor of the electric toothbrush is used as the input signal of the system, the observation signals at the throat microphone and the Bluetooth earphone are used as the output signal of the system, and the frequency domain response function (FRF) of the system is worked out by digging the transformation relation between the input and the output so as to represent the stable characteristics of the oral cavity area. The method comprises the following specific steps:
S103A, extracting the first K main frequency components of the idle running sound signals of the electric toothbrush collected in the idle running event, and estimating input signals;
S103B, calculating an input self-power spectrum for the throat microphone signal and the Bluetooth headset signal in a sampling window;
S103C, calculating the cross power spectrum between the input signal and the two paths of outputs;
and S103D, obtaining frequency domain response function distribution corresponding to the two output signals respectively through Hv estimation.
S104: after fusion of FRF distribution of the two channels and MFCC characteristics of original audio, dimension reduction is carried out on the characteristics by using an analysis of variance method in order to guarantee calculation real-time performance on mobile equipment, and finally tooth surface classification and tooth brushing behavior recognition are carried out by using a feature descriptor set after dimension reduction. The method specifically comprises the following steps:
S104A, performing histogram estimation on the frequency domain response function obtained in S103D to obtain a frequency domain response vector;
S104B, fusing frequency domain response vectors respectively corresponding to the throat microphone and the Bluetooth headset with the MFCC characteristics of the original audio;
and S104C, performing variance analysis on the whole feature set by using an ANOVA method and performing feature dimension reduction to meet the calculation requirement of real-time monitoring on the mobile equipment.
S105: and (4) constructing a tooth brushing behavior recognition model and monitoring in real time.
Comparing and optimizing a plurality of machine learning classifier models for the feature descriptors of each oral cavity position acquired in the step S104C, and selecting the model with the most suitable accuracy and calculation complexity to integrate into the real-time detection application. In the process of detecting the tooth brushing behavior of the user in real time, for each sampling window signal, extracting and fusing characteristics through the method of the steps S102-104, and classifying the tooth surface through an optimal classifier. After the tooth surfaces brushed at each moment are obtained, the cleaning time and the coverage rate of each tooth surface are accumulated according to a standard tooth brushing method, and graphic and sound prompts are given to a user for the tooth surfaces which are not cleaned in place in application. After the process of brushing teeth each time is finished, scoring feedback is carried out on the brushing teeth according to the cleaning quality of each tooth surface, and the scoring feedback is stored in the cloud server to record the long-term tooth health data of the user. The method specifically comprises the following steps:
S105A: various machine learning models are used for comparison, the tooth brushing behaviors corresponding to the feature sets are classified and identified, and models with better accuracy and calculation performance are preferably integrated into real-time detection application;
S105B: during real-time detection, for each sampling window signal, extracting and fusing characteristics through the processes of the steps 2-6, and classifying the tooth surface through an optimal classifier;
S105C: the cleaning time and the coverage rate of each tooth surface are accumulated according to a standard tooth brushing method, and graphic and sound prompts are given to a user for the tooth surfaces which are not cleaned in place in application;
S105D: grading and feeding back the tooth brushing according to the cleaning quality of each tooth surface, storing the grading and feeding back to the cloud server, and recording the long-term tooth health data of the user.
According to the method for detecting the oral cleaning quality of the electric toothbrush based on the acoustics, the brushing behavior of a user of the electric toothbrush is identified, so that the purposes of detecting and ensuring the brushing quality of the user and monitoring the brushing quality for a long time are achieved.
The present application will be described with reference to specific examples.
Firstly, the traditional two-channel throat microphone is modified, one microphone is close to the skin, so that an observation signal generated by the friction of a toothbrush head and the surface of teeth after an acoustic signal is transmitted through an esophagus can be collected, the other microphone faces the outside, and the signal difference of the rotation of the motor of the toothbrush motor when different tooth surfaces are cleaned is mainly collected. The data of the two microphones are fused into single-channel audio data through hardware, and then combined with the microphone of the Bluetooth headset to construct a wearable acoustic signal acquisition system, wherein the contribution of the acoustic signals acquired by each channel to the identification accuracy is shown in fig. 5. According to the basic tooth brushing method, the oral cavity is divided into 16 different regions, wherein the upper (denoted by U) and lower (denoted by D) incisor regions each include an outer region (denoted by O) and an inner region (denoted by I), the four posterior tooth regions each include an inner, outer and chewing surfaces, and the left and right tooth surfaces are divided by L and R for a total of 16 regions. At the same time, a rinse event is added to improve the robustness of the identification method in distinguishing between brushing and non-brushing events; and an idle event of the electric toothbrush, calculating and rectifying the input signal of the system by extracting its main frequency. Regarding the audio signals collected by different tooth surfaces, the corresponding tooth surfaces are regarded as different linear time-invariant systems, the frequency domain response function of each system is calculated by using a system modeling mode, the stable characteristics of the frequency domain response function are extracted, and the difference between the systems is excavated by using a machine learning algorithm, so that the tooth brushing behavior of a user can be identified.
The detection method specifically comprises the following steps:
step 1: and acquiring and labeling multi-channel audio data.
When the marked data are collected and the real-time detection is carried out, the throat microphone is connected with the mobile phone through the 3.5mm audio interface and is worn on the neck; the Bluetooth headset is in wireless connection with the mobile phone through Bluetooth and is worn on one side ear of a user. The collected signal is divided into two channels of a microphone and a Bluetooth headset by using a 44100Hz sampling rate and a 16bit width. For further acoustic signal processing, the method sets the sliding window sizes of the 128, 512, 1024, 2048, 4096, 8192 and 10240 sampling points and the step sizes of 30%, 50%, 70% and 100%, and selects the optimal size and step size for real-time monitoring, wherein the performances of different combinations are shown in fig. 6. In order to construct a universal tooth brushing behavior recognition model, tooth brushing acoustic signals of a plurality of volunteers are collected, and acoustic signals corresponding to all tooth surfaces are labeled by using a labeling tool matched with real-time monitoring application to form an original training data set.
Step 2: the audio signals are synchronized.
In the real-time detection process, the signal of the throat microphone is transmitted to the mobile phone by a link of a microphone-audio line-3.5 mm interface; signals collected by the Bluetooth earphone are transmitted to the mobile phone by a microphone-DSP coding-Bluetooth wireless transmission link, and the propagation time delay of the two links is different; and the respective signal acquisition threads of the throat microphone and the Bluetooth headset have the problem of asynchronous starting. The two points cause signal time difference between the Bluetooth headset and the throat microphone, and if two paths of signals with the time difference are directly analyzed, more serious characteristic extraction deviation and identification errors can be caused. Therefore, the method uses a mode based on the cross-correlation function to synchronously process the signals acquired in real time. And calculating the distribution of the cross-correlation function of the Bluetooth and Hour signals in a window, selecting a sampling point corresponding to an extreme value point closest to the point 0 in the cross-correlation coefficient distribution as the offset sampling point number of the Bluetooth signal and the Hour signal, and shifting the delayed signal forwards by the corresponding sampling point number to form a synchronized dual-channel signal. The schematic diagram of the synchronization method is shown in fig. 7, and for two signals in the diagram, the cross-correlation function reaches an extreme value at 3710 sampling points, which indicates that the acquired signal of the headset lags behind the acquired signal of the bluetooth headset by 3710 sampling points.
And step 3: the oral region is modeled systematically.
In order to identify the tooth surfaces that the user is cleaning, the present invention uses a method based on SIMO system modeling, considering the facial tissues corresponding to 16 regions of the oral cavity as respective different linear time-invariant systems. Because the electric toothbrush has the characteristic of stable rotation, a signal generated by the rotation of a motor of the electric toothbrush can be regarded as the input of the system; after the signal passes through a certain specific transmission path and reaches the Bluetooth and throat microphone observation points, the observation signal is taken as an output signal of the system, and the system corresponding to the path can cause the signal to generate specific change, and the transmission paths of the signal are different for different tooth surfaces. For example, when brushing the inner surface of the left posterior tooth, the signal transmission paths to the two observation points are respectively a tooth-inside-jaw-inside-esophagus-throat microphone left side microphone, a tooth-inside-outside-jaw-air-throat microphone right side microphone, and a tooth-right oral-right side face-bluetooth headset microphone, and when brushing the outer surface of the right posterior tooth, the input signal is transmitted through different paths. Since the user's facial structure is relatively stable, the features obtained by system modeling have greater stability than statistical features. When the rotating speed of the electric toothbrush is reduced due to low electric quantity or the rotating mode is changed due to gear switching, the input signal of the system is changed, and the statistical characteristic mode of the output signal is greatly changed, so that different tooth surfaces cannot be distinguished; the facial structure system corresponding to the input-output signals is not changed, and the characteristics of the system model still have good discrimination for each tooth surface.
The invention obtains the oral system model parameters by the following method: firstly, extracting the first K main frequency components of an idle sound signal of the electric toothbrush collected in an idle event, and estimating an input signal I; then, for the throat microphone signal T and the Bluetooth headset signal B in a sampling window, calculating an input self-power spectrum G T And G B (ii) a Calculating the cross-power spectrum G between the input and the two outputs IT ,G IB (ii) a And finally, obtaining frequency domain response function (FRF) distribution corresponding to the two output signals respectively through Hv estimation, wherein the frequency domain response function (FRF) distribution is used as a descriptor of the oral system parameters. There is a better distinction between FRF distributions corresponding to different oral positions, as shown in fig. 8.
And 4, step 4: and extracting and fusing system characteristics.
The whole feature set consists of three parts: FRF distribution of the input-throat microphone system, FRF distribution of the input-bluetooth headset system, and 12-orderfmcc characteristics of the raw output signal. MFCC features are added to the feature set to increase feature richness. After the histogram estimation is carried out on the FRF distribution respectively corresponding to the larynx microphone and the Bluetooth headset, three parts of features are fused to form a complete feature descriptor of a sampling window. And performing variance analysis on all the feature sets by using an ANOVA method so as to perform feature dimension reduction to meet the calculation requirement of real-time detection, and finally retaining M-dimensional features, wherein the M value is optimized according to the results of modeling and testing on the calibration data set.
And 5: and (5) constructing a tooth brushing behavior recognition model and monitoring in real time.
For the feature descriptors of each oral cavity position acquired in step 4, compared with the classification results of the features by using classifiers such as XGBoost, SVM, RandomForest, Bagging, NaiveBayes, deep neural network and the like, the accuracy, recall rate and F1 value performance metrics are as shown in fig. 9 (black vertical lines in the figure represent error bars corresponding to histograms), and finally, an XGBoost model with higher accuracy and lower computational complexity than the deep neural network is integrated into real-time detection application. In the process of detecting the tooth brushing behavior of the user in real time, for each sampling window signal, the characteristics are extracted and fused through the method of the steps 2-4, the tooth surfaces are classified through the optimal classifier, and a confusion matrix (reflecting the evaluation quality and the wrong classification of the actual value by the detector) for wrong classification of different tooth surfaces is shown in fig. 10. And after the tooth surfaces brushed at each moment are obtained, the cleaning time and the coverage rate of each tooth surface are accumulated according to a standard tooth brushing method, and graphic and sound prompts are given to a user for the tooth surfaces which are not cleaned in place in application. After the process of brushing teeth is finished every time, scoring feedback is carried out on the current brushing teeth according to the cleaning quality of each tooth surface, the scoring feedback is stored in the cloud server, and long-term tooth health data of a user are recorded.
Although the preferred embodiments of the present invention have been described in detail, the present invention is not limited to the details of the foregoing embodiments, and various equivalent changes (such as number, shape, position, etc.) may be made to the technical solution of the present invention within the technical spirit of the present invention, and the equivalents are protected by the present invention.