WO2018223489A1

WO2018223489A1 - Intelligent input method and system based on bone conduction vibration signal propagation

Info

Publication number: WO2018223489A1
Application number: PCT/CN2017/092769
Authority: WO
Inventors: 伍楷舜; 陈文强; 王璐; 蔡素到
Original assignee: 深圳大学
Priority date: 2017-06-09
Filing date: 2017-07-13
Publication date: 2018-12-13
Also published as: CN107300971A; CN107300971B

Abstract

An intelligent input method and system based on bone conduction vibration signal propagation. The intelligent input method comprises: S1. a receiving step, an intelligent device receiving a vibration signal by using a vibration sensor, and performing noise reduction processing on the vibration signal; S2. an extraction step, detecting, by means of a double-threshold endpoint detection method, and extracting a vibration signal segment generated by knocking a specified part of a human body; and S3. a processing step, extracting signal features, and classifying signal positions based on an RNM algorithm. The beneficial effects are that the problem of a text input mode of an intelligent watch can not only be solved, but also a high recognition rate can be achieved, and rapid inputting can be carried out without losing the battery life of the intelligent watch.

Description

Intelligent input method and system based on bone conduction vibration signal propagation

Technical field

The present invention relates to the field of smart wearable device technologies, and in particular, to an intelligent input method and system based on bone conduction vibration signal propagation.

Background technique

In recent years, we have witnessed the rapid development of smart wearable devices. Wearable devices such as smart bracelets, smart headphones, smart glasses, smart helmets and smart watches are becoming more and more popular in people's daily lives. As the innovation space of smartphones continues to shrink and the market is approaching saturation, smart wearable devices have been widely recognized as the next hot spot in the mobile terminal industry, and are predicted to be the inventions that will replace mobile phones.

Smart watches are popular as a portable device. However, its small size and lightness also face unavoidable technical problems. Since the screen is small, only a few keys can be displayed at a time, and the display of other content is blocked, which is inefficient. At present, there are three main ways for smart watches to implement text input: traditional keyboard, text prediction and speech recognition. But the above methods are either not convenient enough or flexible enough. In the case of environmental noise, the speech recognition rate is difficult to achieve the desired effect, and in terms of protecting the user's password and other privacy, it is obviously not suitable to use the voice input in public. Although multi-country research teams have developed finger tracking recognition technology, the user experience of keyboard use has not been able to compare with the fast and comfortable text input of large-screen smartphones. For example, the research team at the University of Washington in 2016 realized millimeter-level precise finger tracking. Technology allows users to implement handwriting input based on sonic positioning on mobile devices, but handwriting input is still too slow to meet people's needs. In order to expand the market demand for smart watches, it is necessary to dig deep into their applications and solve text input problems.

Summary of the invention

The invention provides an intelligent input method based on bone conduction vibration signal propagation, comprising the following steps:

S1. receiving step, the smart device receives the vibration signal by using the vibration sensor, and performs noise reduction processing on the vibration signal;

S2. an extraction step of detecting and extracting a vibration signal segment generated by striking a designated part of the human body by using a double threshold endpoint detection method;

S3. Processing steps, extracting signal features, and classifying signal locations based on the RMN algorithm.

As a further improvement of the present invention, in the S2. extraction step, the method comprises:

Step S21, setting two thresholds of high and low for the processed signal;

Step S22, when the energy or zero-crossing rate of the signal exceeds the low threshold, the starting point of the tapping signal is initially determined, and when the energy or zero-crossing rate of the signal exceeds the high threshold, the true starting point of the tapping signal is determined; when the signal energy and the zero-crossing When the rate is lower than the low threshold, the signal end point is determined;

In step S23, the data from the start point to the end point is retained, and a vibration signal segment generated by the back of the taper is obtained.

As a further improvement of the present invention, the S3. processing step includes:

Step S31, normalizing the segmented signal, extracting the Mel frequency cepstral coefficient, and obtaining a signal characteristic;

Step S32, classifying the signal features using an RMN algorithm based on a random subspace and a nearest center point algorithm, thereby determining a tap position;

The step S32 includes:

Step S321, collecting the signal features obtained in step S31 as training samples according to different tap positions, and classifying them;

Step S322, calculating a center point of each type of the training sample by using a nearest center point algorithm;

Step S323, based on the random subspace, the class center points of the test sample and the training sample are compared in the subspace multiple times to obtain a plurality of classification results;

Step S324, using a simple majority voting principle for the classification result, and setting a certain number of votes, and obtaining a final number of votes while obtaining a majority of the votes, and obtaining a final classification result;

In step S325, the successfully classified samples are classified into new training samples, and the new class center points of the new training samples are recalculated.

As a further improvement of the present invention, in the step S321, the opponent's back is tapped at different positions, and the signal feature of step S31 is collected as a training sample, and the position is marked and classified into n types according to the position, and each type is tapped m times. Wherein n and m are greater than or equal to 1;

In the step S323, the class centers generated in each step S322 are randomly sampled T attributes, and the above operations are repeated Q times to obtain Q subspaces, and the test samples and the training sample sample centers are compared one by one in the subspace. The Euclidean distance of the point, find the nearest center point, that is, the result of Q subspace classification, where T and Q are greater than 1.

As a further improvement of the present invention, in the S2. extraction step, the designated part of the human body is the back of the hand;

The vibration sensor is a piezoelectric ceramic vibration sensor;

In the S1. receiving step, the noise reduction processing on the vibration signal includes:

Step S11, filtering the DC component and the low frequency noise by using a 20 Hz Butterworth high-pass filter;

In step S12, high frequency noise is filtered out using 800 Hz low pass filtering.

The invention also provides an intelligent input system based on bone conduction vibration signal propagation, comprising:

Receiving module, the smart device receives the vibration signal by using the vibration sensor, and performs noise reduction processing on the vibration signal;

The extraction module detects and extracts a vibration signal segment generated by striking a designated part of the human body by using a double threshold end point detection method;

The processing module extracts signal features and classifies signal locations based on the RMN algorithm.

As a further improvement of the present invention, the extraction module includes:

a first extraction module, configured to set two thresholds of high and low for the processed signal;

The second extraction module is configured to determine the starting point of the tapping signal when the energy or zero-crossing rate of the signal exceeds the low threshold, and determine the true starting point of the tapping signal when the energy or zero-crossing rate of the signal exceeds the high threshold; The end of the signal is determined when the energy and zero-crossing rate are simultaneously below the low threshold;

The third extraction module is configured to retain data from the start point to the end point, and obtain a vibration signal segment generated by the back of the taper.

As a further improvement of the present invention, the processing module includes:

a first processing module, configured to normalize the segmented signal, extract a Mel frequency cepstral coefficient, and obtain a signal characteristic;

a second processing module, configured to classify the signal feature using an RMN algorithm based on a random subspace and a nearest center point algorithm, thereby determining a tap position;

The second processing module includes:

a first processing unit, configured to collect the signal features obtained in step S31 as training samples according to different tap positions, and perform classification;

a second processing unit, configured to calculate each type of center point of the training sample using a nearest center point algorithm;

a third processing unit, configured to compare the class center points of the test sample and the training sample in the subspace multiple times based on the random subspace, to obtain a plurality of classification results;

a fourth processing unit, configured to use a simple majority voting principle for the classification result, and set a certain number of votes, and obtain a final classification result if a majority of votes are obtained while a certain number of votes are obtained;

The fifth processing unit is configured to classify the successfully classified samples into new training samples, and recalculate the new class center points of the new training samples.

As a further improvement of the present invention, in the first processing unit, the opponent's back is tapped at different positions, and the signal feature of step S31 is acquired as a training sample, and the position is marked and according to the position. The difference is divided into n categories, each type is struck m times, where n and m are greater than or equal to 1;

In the third processing unit, the class centers generated in each of the second processing units are randomly sampled by T attributes, and the above operations are repeated Q times to obtain Q subspaces, and the test samples and the training samples are compared one by one in the subspace. The Euclidean distance of each class center point, find the nearest center point, that is, the result of Q subspace classification, where T and Q are greater than 1.

As a further improvement of the present invention,

In the extraction module, the designated part of the human body is the back of the hand;

The vibration sensor is a piezoelectric ceramic vibration sensor;

In the receiving module, performing noise reduction processing on the vibration signal includes:

a first receiving module for filtering DC components and low frequency noise using a 20 Hz Butterworth high pass filter;

A second receiving module for filtering high frequency noise using 800 Hz low pass filtering.

The invention has the beneficial effects that the invention not only solves the problem of the text input mode of the smart watch, but also achieves a high recognition rate, and can input quickly without losing the battery life of the smart watch.

DRAWINGS

Figure 1 is a schematic diagram of the user typing on the virtual nine-square grid in the back of the hand;

2 is a schematic diagram of a piezoelectric ceramic vibration sensor;

3 is a structural view of a piezoelectric ceramic vibration sensor;

Figure 4 is a waveform diagram of the original signal;

Figure 5 is an adaptive filtering diagram;

Figure 6 is a low pass filter diagram;

Figure 7 is a flow chart of the RNM algorithm.

detailed description

As shown in FIG. 1, the present invention discloses an intelligent input method based on bone conduction vibration signal propagation, which includes the following steps:

In the S2. extraction step, the designated part of the human body is the back of the hand;

The vibration sensor is a piezoelectric ceramic vibration sensor, and the smart device includes a smart watch, piezoelectric ceramic The porcelain vibration sensor is built into the smart watch. Figures 2 and 3 show the schematic and structure of the piezoelectric ceramic vibration sensor. Due to the piezoelectric effect, the internal polarity changes, and the voltage change is displayed externally, allowing the operator to strike the back of the hand and collect the vibration signal generated by the tap.

Figure 4 shows the original signal waveform. It can be seen that the original signal collected has strong anti-interference ability and less noise. Figure 5 uses adaptive filtering and Figure 6 waveform diagram after low pass filtering. Low-pass filtering preserves more of the characteristics of the signal and works better for subsequent classification of slightly different vibration signals.

In the S2. extraction step, the method includes:

Step S21, setting two thresholds of high and low for the processed signal;

In step S23, only the data from the start point to the end point is retained, and a vibration signal segment generated by the back of the taper is obtained.

In the S3. processing step, extracting the signal feature includes: initializing the length of the starting point and the ending point in the training sample as the segmented signal, and normalizing the signal with the same length after the segmentation, using the formula:

Where x is the vibration signal and n is the dimension of the signal. The normalized signal is extracted, the calculation amount is reduced, and most of the original signal is retained. The feature extracts the Mel frequency cepstral coefficient of the signal while retaining the characteristics of the time domain and the frequency domain.

The S3. processing step includes:

As shown in FIG. 7, in the step S32, classifying the signal location based on the RRM algorithm includes: including:

Step S321, according to different tap positions, the signal feature training obtained in step S31 is collected. Practice the samples and classify them into 9 categories according to the position of Jiugongge;

In the step S321, the opponent's back is tapped at different positions, and the signal feature of step S31 is collected as a training sample, and the position is marked and divided into n categories according to the position, each type of tapping m times, wherein n and m are greater than or equal to 1;

The invention also discloses an intelligent input system based on bone conduction vibration signal propagation, comprising:

The extraction module includes:

The third extraction module is configured to retain only the data from the start point to the end point, and the rest is processed as a segment to obtain a vibration signal segment generated by the back of the taper.

The processing module includes:

The second processing module includes:

In the first processing unit, the opponent's back is tapped at different positions, and the signal feature of step S31 is collected as a training sample, and the position is marked and divided into n categories according to different positions, each type of tapping m times, where n and m are greater than Or equal to 1;

In the extraction module, the designated part of the human body is the back of the hand; the vibration sensor is a piezoelectric ceramic vibration sensor.

The invention embeds a tiny thin piezoelectric sensor on a smart watch, which can realize the conversion of mechanical energy to electric energy. On the back of the hand, a nine-square grid keyboard is virtualized. When the finger taps on the grid at different positions, the mechanical waves will spread out in all directions, and will be reflected back after touching the object. Therefore, based on the broadcast nature of mechanical waves, the piezoelectric sensor receives mechanical wave signals with different multipath propagations at a time. This mechanical wave signal is spread on the one hand by the voice signal into the air, on the other hand in the interior of the hand, and the so-called bone conduction. This part of the mechanical wave signal is not affected by the environmental noise, and can be better accepted by the piezoelectric sensor, and converted into an electrical signal processed by the controller of the smart watch. Because the multipath effects of mechanical waves generated by different grids are different, the letter received by the smart watch The number is different. Using this difference, combined with the classification algorithm of machine learning, you can classify each button of the nine squares. Thereby, a smart watch text input method and system based on the hand back bone conduction technology can be realized.

The invention adopts a piezoelectric ceramic vibration sensor built in a smart watch, and the vibration of the back of the tapper is used as the text input mode of the smart watch for the first time, and the back of the hand is used as a virtual large screen of the small screen of the smart watch, which facilitates text input; It is the vibration signal after multipath propagation in the human hand behind the percussion, strong anti-interference, and noise reduction, segmentation, normalization, extraction of the Mel frequency cepstral coefficient, etc. The RNM algorithm classifies and its recognition rate reaches 92%. The algorithm complexity used here is also linear, so fast input of text can be achieved. In addition, the piezoelectric ceramic vibration sensor consumes a very low amount of electricity and does not significantly reduce the life time of the smart watch.

The invention not only solves the problem of the text input mode of the smart watch, but also achieves a high recognition rate, and can input quickly without losing the life time of the smart watch.

The hardware of the invention is low, the system is simple, the use is convenient, and the input of the smart watch based on the bone conduction vibration signal propagation of the back of the hand can be realized simply and quickly.

The above is a further detailed description of the present invention in connection with the specific preferred embodiments, and the specific embodiments of the present invention are not limited to the description. It will be apparent to those skilled in the art that the present invention may be made without departing from the spirit and scope of the invention.

Claims

An intelligent input method based on bone conduction vibration signal propagation, characterized in that it comprises the following steps:

S1. receiving step, the smart device receives the vibration signal by using the vibration sensor, and performs noise reduction processing on the vibration signal;

S2. an extraction step of detecting and extracting a vibration signal segment generated by striking a designated part of the human body by using a double threshold endpoint detection method;

S3. Processing steps, extracting signal features, and classifying signal locations based on the RMN algorithm.
The intelligent input method according to claim 1, wherein in the S2. extracting step, the method comprises:

Step S21, setting two thresholds of high and low for the processed signal;

Step S22, when the energy or zero-crossing rate of the signal exceeds the low threshold, the starting point of the tapping signal is initially determined, and when the energy or zero-crossing rate of the signal exceeds the high threshold, the true starting point of the tapping signal is determined; when the signal energy and the zero-crossing When the rate is lower than the low threshold, the signal end point is determined;

In step S23, the data from the start point to the end point is retained, and a vibration signal segment generated by the back of the taper is obtained.
The intelligent input method according to claim 1, wherein the S3. processing step comprises:

Step S31, normalizing the segmented signal, extracting the Mel frequency cepstral coefficient, and obtaining a signal characteristic;

Step S32, classifying the signal features using an RMN algorithm based on a random subspace and a nearest center point algorithm, thereby determining a tap position;

The step S32 includes:

Step S321, collecting the signal features obtained in step S31 as training samples according to different tap positions, and classifying them;

Step S322, calculating a center point of each type of the training sample by using a nearest center point algorithm;

Step S323, based on the random subspace, the class center points of the test sample and the training sample are compared in the subspace multiple times to obtain a plurality of classification results;

Step S324, using a simple majority voting principle for the classification result, and setting a certain number of votes, The final classification result is obtained when a majority of votes are obtained and a certain number of votes are reached;

In step S325, the successfully classified samples are classified into new training samples, and the new class center points of the new training samples are recalculated.
The intelligent input method according to claim 3, wherein:

In the step S321, the opponent's back is tapped at different positions, and the signal feature of step S31 is collected as a training sample, and the position is marked and divided into n categories according to the position, each type of tapping m times, wherein n and m are greater than or equal to 1;

In the step S323, the class centers generated in each step S322 are randomly sampled T attributes, and the above operations are repeated Q times to obtain Q subspaces, and the test samples and the training sample sample centers are compared one by one in the subspace. The Euclidean distance of the point, find the nearest center point, that is, the result of Q subspace classification, where T and Q are greater than 1.
The intelligent input method according to claim 1, wherein:

In the S2. extraction step, the designated part of the human body is the back of the hand;

The vibration sensor is a piezoelectric ceramic vibration sensor;

In the S1. receiving step, the noise reduction processing on the vibration signal includes:

Step S11, filtering the DC component and the low frequency noise by using a 20 Hz Butterworth high-pass filter;

In step S12, high frequency noise is filtered out using 800 Hz low pass filtering.
An intelligent input system based on bone conduction vibration signal propagation, characterized in that it comprises:

Receiving module, the smart device receives the vibration signal by using the vibration sensor, and performs noise reduction processing on the vibration signal;

The extraction module detects and extracts a vibration signal segment generated by striking a designated part of the human body by using a double threshold end point detection method;

The processing module extracts signal features and classifies signal locations based on the RMN algorithm.
The intelligent input system according to claim 6, wherein the extraction module comprises:

a first extraction module, configured to set two thresholds of high and low for the processed signal;

a second extraction module, configured to initially determine a tapping signal when the energy or zero-crossing rate of the signal exceeds a low threshold Number starting point, and when the energy or zero-crossing rate of the signal breaks through the high threshold, the true starting point of the tapping signal is determined; when the signal energy and the zero-crossing rate are simultaneously lower than the low threshold, the signal end point is determined;

The third extraction module is configured to retain data from the start point to the end point, and obtain a vibration signal segment generated by the back of the taper.
The intelligent input system according to claim 6, wherein the processing module comprises:

a first processing module, configured to normalize the segmented signal, extract a Mel frequency cepstral coefficient, and obtain a signal characteristic;

a second processing module, configured to classify the signal feature using an RMN algorithm based on a random subspace and a nearest center point algorithm, thereby determining a tap position;

The second processing module includes:

a first processing unit, configured to collect the signal features obtained in step S31 as training samples according to different tap positions, and perform classification;

a second processing unit, configured to calculate each type of center point of the training sample using a nearest center point algorithm;

a third processing unit, configured to compare the class center points of the test sample and the training sample in the subspace multiple times based on the random subspace, to obtain a plurality of classification results;

a fourth processing unit, configured to use a simple majority voting principle for the classification result, and set a certain number of votes, and obtain a final classification result if a majority of votes are obtained while a certain number of votes are obtained;

The fifth processing unit is configured to classify the successfully classified samples into new training samples, and recalculate the new class center points of the new training samples.
The intelligent input system of claim 8 wherein:

In the first processing unit, the opponent's back is tapped at different positions, and the signal feature of step S31 is collected as a training sample, and the position is marked and divided into n categories according to different positions, each type of tapping m times, where n and m are greater than Or equal to 1;

In the third processing unit, the class centers generated in each of the second processing units are randomly sampled by T attributes, and the above operations are repeated Q times to obtain Q subspaces, and the test samples and the training samples are compared one by one in the subspace. The Euclidean distance of each class center point, find the nearest center point, that is, the result of Q subspace classification, where T and Q are greater than 1.
The intelligent input system of claim 6 wherein:

In the extraction module, the designated part of the human body is the back of the hand;

The vibration sensor is a piezoelectric ceramic vibration sensor;

In the receiving module, performing noise reduction processing on the vibration signal includes:

a first receiving module for filtering DC components and low frequency noise using a 20 Hz Butterworth high pass filter;

A second receiving module for filtering high frequency noise using 800 Hz low pass filtering.