WO2020186477A1 - 一种基于骨传导的智能输入方法和系统 - Google Patents

一种基于骨传导的智能输入方法和系统 Download PDF

Info

Publication number
WO2020186477A1
WO2020186477A1 PCT/CN2019/078870 CN2019078870W WO2020186477A1 WO 2020186477 A1 WO2020186477 A1 WO 2020186477A1 CN 2019078870 W CN2019078870 W CN 2019078870W WO 2020186477 A1 WO2020186477 A1 WO 2020186477A1
Authority
WO
WIPO (PCT)
Prior art keywords
signal
vibration signal
vibration
feature extraction
input
Prior art date
Application number
PCT/CN2019/078870
Other languages
English (en)
French (fr)
Inventor
伍楷舜
黄勇志
王璐
蔡少填
张健浩
Original Assignee
深圳大学
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 深圳大学 filed Critical 深圳大学
Priority to PCT/CN2019/078870 priority Critical patent/WO2020186477A1/zh
Publication of WO2020186477A1 publication Critical patent/WO2020186477A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology

Definitions

  • the present invention relates to the technical field of wearable devices, in particular to an intelligent input method and system based on bone conduction.
  • smart devices have begun to play an important role in accelerating the transmission of information and enhancing communication between people.
  • smart devices are designed to be smaller and smaller, such as smart watches, Google glasses, etc.
  • the screen projection technology, augmented reality technology and wristband holographic projection technology of smart watches have gradually become research hotspots.
  • smart devices are also very important to solve the technical problems of interaction.
  • voice input has very high requirements on the user's accent, speech speed, and the network environment where the smart device is located. As long as one of these three conditions is not met, the user needs to re-enter many times. Moreover, the voice input is very unsatisfactory in its resistance to external noise and its ability to protect privacy.
  • the purpose of the present invention is to overcome the above-mentioned defects of the prior art and provide an intelligent input method and input system based on bone conduction.
  • an intelligent input method based on bone conduction includes the following steps:
  • Step S1 receiving a bone conduction vibration signal from the button and performing feature extraction to obtain a vibration signal after feature extraction;
  • Step S2 input the time-series signal sequence of the vibration signal after the feature extraction into the trained neural network classification model, and identify the type of button corresponding to the vibration signal of bone conduction;
  • Step S3 Determine the text information input by the user based on the recognized key type.
  • step S1 includes the first feature extraction
  • the first physical sign extraction includes: collecting the received vibration signal through the vibration patch sensor; for the collected vibration signal, using a Pap low-pass filter Remove high-frequency electrical noise; use a signal amplifier to amplify the vibration signal that removes high-frequency noise; convert the amplified vibration signal into a corresponding digital signal through an analog-to-digital converter.
  • the method further includes: using a notch filter to remove high-frequency and low-frequency signal noise; for the vibration signal that removes high-frequency and low-frequency signal noise, determine whether it is noise Signal; for the vibration signal judged to be a non-noise signal, determine the cutting start point and the cutting end point, and extract the signal between the cutting start point and the cutting end point as the target signal to be analyzed.
  • a signal whose amplitude of the vibration signal exceeds the amplitude threshold and whose duration exceeds the time threshold is determined as a non-noise signal.
  • the cutting start point and the cutting end point are determined according to the following steps:
  • the cutting end point is selected after M consecutive sampling points of the vibration signal are lower than the first point of the second amplitude threshold, and N2 sampling points are selected, where N1, M, and N2 are positive integers.
  • step S1 further includes performing the following steps for the target signal to be analyzed:
  • the normalization method is used to perform amplitude normalization processing on the target signal to be analyzed.
  • it further includes performing a second feature extraction on the normalized signal.
  • the second feature extraction includes: after the signal is processed by Haar wavelet, the trend sub-signal is retained; The variance of, filter out the feature dimensions whose variance is greater than the variance threshold.
  • step S3 further includes: comparing the recognized letter combination input by the user with the letter combination selected by the user, marking different letters as misjudged letters; multiple times when the same letter is marked as misjudged In the case of letters, it is determined whether to retrain the neural network classification model according to the similarity of the corresponding vibration signals when they are marked.
  • the similarity is determined according to the following steps: calculate the Pearson correlation coefficient ⁇ of the two groups of vibration signals, and when ⁇ is greater than the threshold ⁇ threshold , it is determined that the two groups of vibration signals are similar. Sex, where 0 ⁇ threshold ⁇ 1.
  • the neural network classification model is a neural network classification model based on radial basis functions.
  • an intelligent input system based on bone conduction includes:
  • Vibration signal processing unit used to receive the bone conduction vibration signal from the button and perform feature extraction to obtain the vibration signal after feature extraction;
  • Classification and recognition unit used to input the time sequence signal sequence of the vibration signal after the feature extraction into the trained neural network classification model, and identify the type of button corresponding to the vibration signal of bone conduction;
  • Output unit used to determine the text information input by the user based on the recognized key type.
  • the present invention has the beneficial effect of: using the characteristics of human anatomy to propose a more convenient and lower-cost intelligent input method, which uses inexpensive sensors and uses human bones to achieve low consumption. Capable, fast and high-precision text input.
  • the present invention also proposes data processing methods such as normalization and feature screening to process the vibration signal, which further amplifies the corresponding characteristics of the signal; and the present invention uses a radial basis neural network to classify the vibration signal, The type of the input key can be accurately recognized; further, the present invention also includes detecting and correcting the misjudgment data, which improves the stability and robustness of the key recognition.
  • Fig. 1 is a flowchart of an intelligent input method based on bone conduction according to an embodiment of the present invention
  • Figure 2 (a) is a keyboard layout diagram according to an embodiment of the present invention.
  • Figure 2(b) is an experimental layout diagram according to an embodiment of the present invention.
  • Figure 3 is a schematic diagram of the radial basis function neural network structure
  • FIG. 4 is a schematic structural diagram of an intelligent input system based on bone conduction according to an embodiment of the present invention.
  • Fig. 5 is a structural diagram of an electronic device according to an embodiment of the present invention.
  • an intelligent input method based on bone conduction is provided, which can be applied to the realization of a virtual keyboard of a wearable device.
  • the method includes receiving the bone-conducted vibration signal of the user's button and performing preliminary feature extraction; filtering and cutting the vibration signal of the preliminary extracted feature; normalizing the cut vibration signal; and normalizing Signal dimensionality reduction and dimensionality screening; input the vibration signal after dimensionality reduction processing into the neural network classification model to obtain the user button type.
  • the neural network classification model takes the sampling time sequence signal sequence of the vibration signal as input, and corresponds to the vibration signal
  • the key type is the output obtained through training.
  • the method of the embodiment of the present invention includes the following steps:
  • Step S110 Collect the vibration signal of the user's key press and perform the first feature extraction of the vibration signal.
  • a vibration signal When the user presses the key with a finger, a vibration signal will be generated. For different keys, the generated strength and direction are different.
  • the present invention is based on the correlation between the user's finger key press law and the generated vibration signal, by performing the vibration signal Feature analysis can identify the type of user buttons, and then obtain the text information input by the user (such as letters, numbers, symbols, etc.).
  • the index finger corresponds to 6 keyboard keys
  • the middle finger, ring finger, and little finger correspond to 3 keys on the keyboard
  • the thumb Corresponds to the space bar.
  • you need to hit the space bar your two thumbs will hit the keyboard together.
  • the thumb is only responsible for the corresponding arrow keys.
  • the left thumb is left, and the right big The thumb is to the right.
  • the keyboard needs to be switched to the numeric keyboard, the user needs to enter'left, right, right, left' to complete the switch from the alphabet keyboard to the numeric keyboard. After switching to the numeric keyboard, ten fingers correspond With 10 numbers.
  • the vibration signal of the user's button is collected and the first feature extraction is performed according to the following steps: the received vibration signal is collected through the vibration patch sensor technology; for the collected vibration signal, the Pap low-pass filter is used The amplifier removes high-frequency electrical noise; the signal that removes noise is amplified by a signal amplifier; the amplified signal is converted into a corresponding digital signal through an analog-to-digital converter.
  • the above-mentioned vibrating patch sensor, low-pass filter, signal amplifier, analog-to-digital converter, etc. can all be commercially available or customized devices, and the functions of the embodiments of the present invention can be realized by setting appropriate parameters.
  • the cutoff frequency FC of the Bap low-pass filter is set to a certain frequency less than 1000 Hz.
  • Step S120 filtering and cutting the signal after the first feature extraction to obtain the cut vibration signal.
  • the signal after the first feature extraction is filtered and cut.
  • filtering and cutting include the following steps: using a notch filter to remove high-frequency and low-frequency signal noise; judging whether the obtained vibration signal is a noise signal and cutting the vibration signal judged to be a non-noise signal, In order to obtain the target signal to be analyzed.
  • the threshold comparison method is used to determine whether it is a noise signal. For example, when the amplitude of the vibration signal exceeds a certain threshold and lasts for a period of time t (for example, the time threshold t is greater than 1ms), the vibration can be considered The signal is not a noise signal.
  • This method of combining the amplitude threshold and the duration t to determine the noise signal can identify the vibration signal generated by the user's unconscious sliding or sudden collision on the button as a noise signal, thereby filtering out invalid vibration signals as soon as possible.
  • the cutting method is to extract the signal between the cutting start point and the cutting end point. Specifically, first, determine the starting point of cutting, for example, select the starting point of cutting before the first point where the amplitude of the vibration signal exceeds a certain amplitude threshold, and take N1 sampling points (where N1 is an integer greater than or equal to 0); secondly, determine The cutting end point, for example, the cutting end point is selected after the first point where M consecutive sampling points of the vibration signal (where M is an integer greater than or equal to 0, which needs to be determined according to the sampling frequency and device sensitivity) is lower than a certain amplitude threshold, Take N2 sampling points (where N2 is an integer greater than or equal to M).
  • the vibration signal By cutting the vibration signal, extract the signal between the cutting start point and the cutting end point as the target signal to be analyzed, which can filter out the signal within a period of time after the user starts to press the key and before the end of the key.
  • the vibration during this period The signal has little meaning for identifying the type of button, and early filtering can improve the accuracy and speed of recognition.
  • the present invention does not limit the specific values of N1, M, N2, amplitude threshold, time threshold, etc., and those skilled in the art can set appropriate values according to the sampling frequency and the sensitivity of the wearable device, and different
  • the amplitude threshold can be set to the same or different values.
  • the amplitude threshold used to determine whether it is a noise signal and the amplitude threshold used in signal cutting can be the same or different, and the amplitude threshold used when determining the cutting start and end points Can be the same or different.
  • Step S130 Perform normalization processing on the cut vibration signal.
  • vibration signal after cutting may be normalized, for example, normalized adjustment based on the signal amplitude.
  • normalizing the cut vibration signal includes:
  • x and y correspond to the signals before and after normalization
  • x max and x min correspond to the maximum and minimum values before normalization
  • y max and y min correspond to the maximum and minimum values after normalization.
  • Step S140 Perform a second feature extraction on the normalized signal to obtain a reduced-dimensional vibration signal.
  • the second feature extraction is performed through dimensionality reduction and dimensionality screening to further remove signal jitter.
  • the use of Haar wavelet for dimensionality reduction and dimensionality screening specifically includes: using Haar wavelet to process the signal, and only retain the trend sub-signal, thereby removing the jitter of the signal, and reducing the feature dimension by half; calculating the difference between the dimensions of the signal If the variance is less than a certain variance threshold, the dimension is considered to be a characteristic dimension; otherwise, the dimension is regarded as not a characteristic dimension, and non-characteristic dimensions are filtered out.
  • the variance threshold in this embodiment can be set to an appropriate value through statistics according to the key usage scenario, for example, the variance threshold is preset to a constant.
  • Step S150 Input the reduced-dimensional vibration signal into the trained neural network classification model to obtain the key type corresponding to the vibration signal.
  • the neural network classification model is obtained through training.
  • the input vibration signal feature is a time series signal sequence of vibration sampling, and the output classification label is as shown in Figure 2.
  • the neural network classification model can use LSTM (Long Short Term Memory Network) or radial basis function neural network.
  • vibration signals of different types of buttons have certain differences in the time domain, the number of peaks, valleys, and shapes of the vibration signals corresponding to each type of button are not completely the same. Based on this local difference, in a preferred embodiment, Using a radial basis function neural network with local learning ability, this kind of neural network with local learning ability can avoid overfitting caused by the design of too many neurons.
  • vibration signals belong to nonlinear curves. They are linearly inseparable in low-dimensional space.
  • the basic idea of radial basis function neural networks is to use radial basis functions as the "base" of hidden units to form hidden layer space.
  • Containing layers to transform the input vector transform the low-dimensional pattern input data into the high-dimensional space, so that the linear inseparability problem in the low-dimensional space is linearly separable in the high-dimensional space, so that the vibration signal is well classified .
  • Figure 3 shows the neural network structure of the radial basis function.
  • the network structure includes three layers.
  • the first layer is the input layer and is composed of signal source nodes; the second layer is the hidden layer (hidden layer).
  • the transformation function of the neurons in the layer is the radial basis function, which is a non-negative linear function that is radially symmetric and attenuated to the center point.
  • the transformation function is a local response function; the third layer is the response to the output mode.
  • [x1,x2,...,xn] are the features of the input layer unit and the input signal and the corresponding label
  • [w1,w2,...,wN] is the connection weight of the hidden layer that needs to be trained to the output layer Value
  • B is the threshold, used to adjust the sensitivity of the neuron
  • Y is the output layer unit, that is, the output button classification.
  • the activation function of the radial basis function neural network uses a Gaussian function. Due to the characteristics of the Gaussian distribution function, each time the radial basis function neural network is input, only the neurons that are closer to the input sample vector will be active The corresponding weight will be updated, and the neurons that are far from the sample will not be activated, so that the number of neurons will not be designed too much, thereby avoiding the phenomenon of overfitting.
  • the activation function is expressed as Among them,
  • is the Euclidean norm
  • c i is the center of the Gaussian function of the i-th node in the hidden layer
  • ⁇ is Gaussian The variance of the function.
  • the radial basis function neural network used in the embodiment of the present invention needs to solve three parameters, which are the center of the activation function, the variance, and the weight from the hidden layer to the output layer.
  • the neural network In order to enable the neural network to change rapidly when the training set data enters, based on the characteristics of the vibration signal in multiple dimensions, each feature of the vibration signal is used to directly change the center of the activation function during the training process.
  • the unsupervised learning method of self-organizing selection center can be used to independently learn and calculate the center of the activation function.
  • the process of solving the basis function center c based on the unsupervised learning method of self-organizing selection center includes:
  • Step S152 group the input training sample set according to the nearest neighbor rule
  • Step S153 readjust the cluster center.
  • step S151 calculate the average value of the training samples in each cluster set ⁇ p , that is, the new cluster center c i . If the new cluster center does not change, the obtained c i is the radial basis function neural The final basis function center of the network, otherwise, return to step S151 to enter the next round of clustering center solving.
  • the least squares method is used to calculate the weights from the hidden layer to the output layer, which makes the neural network output layer linear with respect to the weights, thereby significantly speeding up learning and avoiding local minima capabilities.
  • the weight expression from the hidden layer to the output layer is as follows:
  • c max is the maximum distance between the selected centers.
  • step S160 the text input of the user is displayed according to the type of the button.
  • the key type After the key type is recognized, the key type can be displayed to obtain the input text combination, see the display effect shown in Figure 2(b).
  • the embodiment of the present invention may further include a correction process for the neural network classification model.
  • the correction process includes:
  • Step S161 It is judged whether there is a misjudgment on the type of the user's button.
  • the recognized letter combination is displayed.
  • the word library is matched to display the English words that the user may need to input. If the user selects the same as the recognized input letter
  • the combination of distinguishing English words means that the user may have misjudged the letters typed by the user or the user’s input may have been incorrect.
  • the word bank is all the existing Chinese and English letter combinations, and according to Alphabetical order.
  • the method of matching according to the word library is to select all words containing any number of the same letter from the word library according to the recognized letter the user has typed and the language that needs to be input, and compare these Words are scored, the scoring method is Among them, KEY same represents the number of letters that are the same as the recognized user input letter, and KEY total is the total number of words input by the user. After the scores are finished, the words are sorted according to the scores from largest to smallest for user selection.
  • Step S162 Determine whether to retrain the neural network classification model according to the misjudgment situation.
  • the misjudgment is judged, and the recognized letter combination input by the user is compared with each letter in the word that is different from the letter combination selected by the user. After finding a different letter, it is temporarily considered The letters are misjudged letters, and the vibration signal data of the misjudged letters is recorded. If the same letter is recorded as a misjudgment letter for the second time, compare the vibration signal data of the second misjudgment with the vibration signal data of the first misjudgment to determine whether the two vibration signal data are similar If there is similarity, record the misjudged vibration signal data; if there is no similarity, discard the previous misjudged data.
  • the vibration signal data recorded as misjudgments and the corresponding classification labels are put back into the network, and the neural network classification model is retrained to generate a new trained neural network classification model.
  • the way to judge the similarity of the two sets of vibration signals is to calculate the correlation coefficients of the two sets of signals, that is, to calculate the Pearson correlation coefficient ⁇ of the two sets of signals, when ⁇ is greater than a certain threshold ⁇ threshold (where, 0 ⁇ threshold ⁇ 1), it is judged that the two groups of signals are similar, otherwise, it is considered that the two groups of signals are not similar.
  • ⁇ threshold where, 0 ⁇ threshold ⁇ 1
  • Fig. 4 shows an intelligent input system based on bone conduction according to an embodiment of the present invention.
  • the system 400 includes a vibration signal sensing module 410, a processing module 420, and an output module 430.
  • the vibration signal perception module 410 includes a receiving end of the bone vibration signal, which is used to receive and collect the bone vibration signal, and further perform filtering and signal amplification with a low-pass filter, and finally convert it into a digital signal.
  • the processing module 420 is used to preprocess the collected vibration signals, complete the signal filtering and cutting, and then perform normalization processing and feature extraction, etc., to eliminate the interference caused by different percussive forces and reduce the feature dimension. Screen, and use the trained neural network classification model to classify and match the user's current keystrokes to detect user input.
  • the output module 430 is used for real-time output and display of the detected letter or symbol input by the current user.
  • system 400 further includes a correction module 440 for implementing the correction process of the embodiment of the present invention.
  • vibration signal sensing module 410 processing module 420, output module 430, and correction module 440 may be implemented by one functional unit or multiple functional units.
  • the present invention does not limit this, as long as it can realize the functions of the embodiments of the present invention.
  • the vibration signal perception module 410 includes: a signal acquisition unit for data acquisition of received bone vibration signals; a signal extraction unit for extracting vibration signal data; a filtering and amplifying unit for extracting vibration signal data Perform filtering and amplification processing; analog-to-digital conversion unit for converting analog signals into digital signals.
  • the processing module 420 includes: a filtering and cutting unit, used to denoise and cut the signal; a normalization unit, used to eliminate the impact of different levels of user tapping; a feature filtering unit, used to perform related Extraction and screening; the classification unit is used to classify the key types according to the vibration signal characteristics of the user keys.
  • the correction module 440 includes: a dictionary matching unit for matching corresponding words based on user input; a misjudgement recording unit for recording vibration signal data determined to be misjudged; a retraining unit for Retrain the original neural network classification model.
  • the output module 330 includes an output unit for displaying the classification result input by the user.
  • the intelligent input method and input system based on bone conduction in the embodiments of the present invention can realize the QWERTY virtual keyboard of a wearable smart device, and use a vibration sensor to receive vibration signals in the bones of a hand in a specific scene, for example, a paper on a solid desktop
  • the vibration signal generated by the bones when the key is hit on the quality keyboard is then normalized for feature extraction, and Haar wavelet processing is used for dimensionality reduction and dimensionality filtering, which significantly improves the accuracy of key type recognition.
  • the embodiment of the present invention performs classification by a machine learning classification method, and can detect the type of the corresponding button that the user hits, which is a high-response, high-sensitivity, and high-precision smart wearable input method.
  • the present invention takes advantage of the characteristics of human anatomy and proposes a more convenient and low-cost intelligent input method.
  • the keyboard is built on the communication medium of the stability and uniqueness of human bone characteristics, and the human-to-desktop With the help of human bones, the percussion of the wearable smart device can realize low energy consumption, fast and high-precision text input, and realize the realization method of the virtual keyboard of the wearable device.
  • the present invention further improves the stability and robustness of smart input by designing a correction process.
  • Fig. 5 is an electronic device 500 according to an embodiment of the present invention, including a memory 510, a processor 520, and a computer program 511 stored on the memory 510 and running on the processor 520.
  • the processor 520 implements the computer program 511 when the computer program 511 is executed.
  • the intelligent input method based on bone conduction of the embodiment of the invention includes, for example, the following steps: collecting the vibration signal of the user's key press and performing the first feature extraction on the vibration signal; filtering and cutting the signal after the first feature extraction to obtain the experience Cut the vibration signal; normalize the cut vibration signal; perform the second feature extraction on the normalized signal to obtain the reduced-dimensional vibration signal; input the reduced-dimensional vibration signal to the trained neural network Classification model to obtain the key type corresponding to the vibration signal; display the key type to obtain text input, etc.
  • the electronic device described in the present invention is a device that implements an intelligent input method based on bone conduction in the embodiment of the present invention, based on the method introduced in the embodiment of the present invention, those skilled in the art can understand the electronic device in this embodiment.
  • the specific implementation of the device and its various changes, so how the electronic device implements the method in the embodiment of the present invention will not be described in detail, as long as the person skilled in the art implements the method in the embodiment of the present invention, All belong to the scope of the present invention.
  • Electronic devices include but are limited to wearable devices, such as smart watches, Google glasses, smart bracelets, etc.
  • the present invention may be a system, a method and/or a computer program product.
  • the computer program product may include a computer-readable storage medium loaded with computer-readable program instructions for enabling a processor to implement various aspects of the present invention.
  • the computer-readable storage medium may be a tangible device that holds and stores instructions used by the instruction execution device.
  • the computer-readable storage medium may include, but is not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing, for example.
  • Computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM) Or flash memory), static random access memory (SRAM), portable compact disk read-only memory (CD-ROM), digital versatile disk (DVD), memory stick, floppy disk, mechanical encoding device, such as a printer with instructions stored thereon
  • RAM random access memory
  • ROM read-only memory
  • EPROM erasable programmable read-only memory
  • flash memory flash memory
  • SRAM static random access memory
  • CD-ROM compact disk read-only memory
  • DVD digital versatile disk
  • memory stick floppy disk
  • mechanical encoding device such as a printer with instructions stored thereon

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • General Health & Medical Sciences (AREA)
  • Computing Systems (AREA)
  • Computational Linguistics (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • Biomedical Technology (AREA)
  • Molecular Biology (AREA)
  • Biophysics (AREA)
  • Artificial Intelligence (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Health & Medical Sciences (AREA)
  • Human Computer Interaction (AREA)
  • User Interface Of Digital Computer (AREA)

Abstract

本发明提供一种基于骨传导的智能输入方法和系统。该方法包括：接收来自于按键的骨传导的振动信号并进行特征提取，获得经特征提取后的振动信号；将所述经特征提取后的振动信号的时序信号序列输入至经训练的神经网络分类模型，识别出所述骨传导振动信号对应的按键类型；基于识别出的按键类型确定用户输入的文字信息。本发明提供的基于骨传导的智能输入方法和系统能够快速且精确地识别用户输入的文字信息。

Description

一种基于骨传导的智能输入方法和系统 技术领域
本发明涉及可穿戴设备技术领域，尤其涉及一种基于骨传导的智能输入方法和系统。
背景技术
随着智能设备的快速发展，智能设备开始扮演着加快信息的传递、增进人与人之间的交流等重要角色。为了让智能设备更方便携带，智能设备被设计得越来越小，例如智能手表、谷歌眼镜等。为了解决这些便携式智能设备的视觉体验差的缺点，智能手表的投屏技术、增强现实技术和腕带的全息投影技术逐渐地成为研究热点。智能设备作为信息交流与传递、促进人与人之间的交流的工具，解决交互的技术难题同样是非常重要的。
目前，为了克服屏幕太小而无法完成文本输入的局限性，便携式设备一般使用语音输入。然而，语音输入对用户的口音、语速，以及智能设备所处的网络环境的要求都非常高，只要这三个条件其中一个没满足，都需要用户做许多次重新输入。并且，语音输入在对外界噪声的抵抗能力上和在对隐私保护的能力上都非常不理想。
除了使用语音输入，在现有技术中，还有通过追踪手指的方法实现与智能设备的交互与输入。与此相似的一些方法则是通过一些传感器，例如压电传感器、距离传感器等，对用户的手势进行识别从而完成交互。但是，使用书写的方式进行输入，不仅麻烦而且还很慢。此外，一些研究者在中指带上一个装有加速度计的戒指，通过手指控制来选择输入的字母，而这种方法难以实现快速的打字。也有的研究者使用对声音定位的方法来识别敲击键盘的位置，但是，这些方法太过依赖环境的稳定性。也有研究者尝试设计一种新型的键盘，这种键盘是一种触摸电路，通过安装这种触摸电路来实现输入。在最新的研究中，有的研究者则是使用照相机，通过对视频的分析来识别按键，这种方法虽然便捷，但是耗能太大，不适合便携式的智能设备。有的研究者利用振动传感器获得的信号来识别事先在手臂上 做的标记，但是，由于手臂的皮肤与肌肉会导致敲击位置产生偏差，从而导致敲击的信号不稳定。
因此，需要对现有技术进行改进，以提供普适性强、响应速度快和精确度高的智能输入方法。
发明内容
本发明的目的在于克服上述现有技术的缺陷，提供一种基于骨传导的智能输入方法和输入系统。
根据本发明的第一方面，提供了一种基于骨传导的智能输入方法。该方法包括以下步骤：
步骤S1，接收来自于按键的骨传导的振动信号并进行特征提取，获得经特征提取后的振动信号；
步骤S2，将所述经特征提取后的振动信号的时序信号序列输入至经训练的神经网络分类模型，识别出所述骨传导的振动信号对应的按键类型；
步骤S3，基于识别出的按键类型确定用户输入的文字信息。
在一个实施例中，步骤S1包括第一次特征提取，该第一次体征提取包括：通过振动贴片传感器对接收的振动信号进行采集；对于采集到的振动信号，使用巴氏低通滤波器去除高频的电噪声；对于去除高频噪声的振动信号，使用信号放大器进行放大；将放大后的振动信号通过模数转换器转换为对应的数字信号。
在一个实施例中，在所述第一次特征提取之后还包括：利用陷波滤波器去除高频和低频的信号噪声；对于去除高频和低频的信号噪声的振动信号，判断其是否为噪声信号；对于判断为非噪声信号的振动信号，确定切割起点和切割终点，提取所述切割起点和所述切割终点之间的信号作为待分析的目标信号。
在一个实施例中，将振动信号的幅度超过幅度阈值并且持续时间超过时间阈值的信号判断为非噪声信号。
在一个实施例中，对于判断为非噪声信号的振动信号，根据以下步骤确定所述切割起点和切割终点：
将所述切割起点选择在振动信号幅度超过第一幅度阈值的第一个点之前，取N1个采样点；
将所述切割终点选择在振动信号连续M个采样点低于第二幅度阈值 的第一个点之后，取N2个采样点，其中，N1、M、N2为正整数。
在一个实施例中，步骤S1还包括对于所述待分析的目标信号执行以下步骤：
将所述待分析的目标信号的峰值与训练所述神经网络分类模型的训练集中的所有振动信号进行峰值对齐；
使用归一化的方法，对所述待分析的目标信号进行幅度归一化处理。
在一个实施例中，还包括对于归一化处理的信号进行第二次特征提取，该第二次特征提取包括：将信号使用Haar小波进行处理之后，保留趋势子信号；计算信号各个维度之间的方差，将方差大于方差阈值的特征维度滤除。
在一个实施例中，步骤S3还包括：对比识别出的用户输入的字母组合和用户选择的字母组合，将不相同的字母标记为误判的字母；在同一字母多次被标记为误判的字母的情况下，根据其被标记时对应的振动信号的相似性，确定是否对所述神经网络分类模型重新进行训练。
在一个实施例中，对于两组振动信号，根据以下步骤确定是否具有相似性：计算该两组振动信号的皮尔逊相关系数ρ，当ρ大于阈值ρ threshold时，确定该两组振动信号有相似性，其中，0＜ρ threshold≤1。
在一个实施例中，所述神经网络分类模型是基于径向基函数的神经网络分类模型。
根据本发明的第二方面，提供了一种基于骨传导的智能输入系统。该系统包括：
振动信号处理单元：用于接收来自于按键的骨传导的振动信号并进行特征提取，获得经特征提取后的振动信号；
分类识别单元：用于将所述经特征提取后的振动信号的时序信号序列输入至经训练的神经网络分类模型，识别出所述骨传导的振动信号对应的按键类型；
输出单元：用于基于识别出的按键类型确定用户输入的文字信息。
与现有技术相比，本发明的有益效果在于：利用人体解剖学的特性，提出一种更便捷、成本更低的智能输入方法，该方法利用廉价的传感器，借助人体的骨骼，完成低耗能、快速和高精度的文本输入。此外，本发明还提出用归一化和特征筛选等进行数据处理的方法来对振动信号进行处理，进一步放大了信号相应的特征；并且，本发明利用径向基神经网络对 振动信号进行分类，能够精确地识别输入按键类型；进一步地，本发明还包括对误判数据进行检测和校正，提高了识别按键的稳定性和鲁棒性。
附图说明
以下附图仅对本发明作示意性的说明和解释，并不用于限定本发明的范围，其中：
图1是根据本发明一个实施例的基于骨传导的智能输入方法的流程图；
图2(a)是根据本发明一个实施例的键盘布局图；
图2(b)是根据本发明一个实施例的实验布置图；
图3是径向基函数神经网络结构示意图；
图4是根据本发明一个实施例的基于骨传导的智能输入系统的结构示意图；
图5是根据本发明一个实施例的电子设备的结构图。
具体实施方式
为了使本发明的目的、技术方案、设计方法及优点更加清楚明了，以下结合附图通过具体实施例对本发明进一步详细说明。应当理解，此处所描述的具体实施例仅用于解释本发明，并不用于限定本发明。
在本文示出和讨论的所有例子中，任何具体值应被解释为仅仅是示例性的，而不是作为限制。因此，示例性实施例的其它例子可以具有不同的值。
对于相关领域普通技术人员已知的技术、方法和设备可能不作详细讨论，但在适当情况下，所述技术、方法和设备应当被视为说明书的一部分。
根据本发明的一个实施例，提供了一种基于骨传导的智能输入方法，可应用于可穿戴设备的虚拟键盘的实现。简言之，该方法包括接收用户按键的骨骼传导的振动信号并进行初步特征提取；对初步提取特征的振动信号进行滤波和切割；对切割后的振动信号进行归一化处理；对归一化信号进行降维和维度筛选；将降维处理后的振动信号输入至神经网络分类模型，以获得用户按键类型，其中，神经网络分类模型是以振动信号的采样时序信号序列为输入，以振动信号对应的按键类型为输出通过训练获得。
具体地，参见图1所示，本发明实施例的方法包括以下步骤：
步骤S110，采集用户按键的振动信号并对振动信号进行第一次特征提 取。
用户在用手指按键时，会产生振动信号，对于不同按键，所产生的力度、方向等存在差异，本发明基于用户手指按键规律和所产生的振动信号之间的关联性，通过对振动信号进行特征分析，能够识别用户按键类型，进而获得用户输入的文字信息(如字母、数字、符号等)。
例如，对于图2(a)示意的键盘，当采用图2(b)所示的手指放置位置时，食指对应6个键盘按键，中指、无名指和小指分别对应着键盘上3个按键，大拇指对应着空格键，当需要敲击空格键时，则两个大拇指一起敲击键盘，当需要进行左右移动时，大拇指则只负责对应的方向键，左边大拇指为向左，右边的大拇指为向右，当需要将键盘切换到数字键盘时，用户则需要输入‘左，右，右，左’来完成从字母键盘到数字键盘的切换，切换到数字键盘后，十个手指则对应着10个数字。
在一个实施例中，根据以下步骤采集用户按键的振动信号并进行第一次特征提取：通过振动贴片传感器技术对接收的振动信号进行采集；对于采集到的振动信号，使用巴氏低通滤波器去除高频的电噪声；对于去除噪声的信号，使用信号放大器进行放大；将放大后的信号通过模数转换器，转换成对应的数字信号。
上述的振动贴片传感器、低通滤波器、信号放大器和模数转换器等均可采用市售的或定制的器件，通过设置适当参数来实现本发明实施例的功能。例如，将巴氏低通滤波器的截止频率FC设置为小于1000Hz的某一频率。
步骤S120，对经过第一次特征提取的信号进行滤波和切割，获得经切割的振动信号。
在此步骤S120中，为了进一步消除噪声或干扰，对经过第一次特征提取的信号进行滤波和切割。
在一个实施例中，滤波和切割包括以下步骤：利用陷波滤波器，去除高频和低频的信号噪声；判断获得的振动信号是否为噪声信号并对判断为非噪声信号的振动信号进行切割，从而获得待分析的目标信号。
在一个实施例中，通过阈值比较法来判断是否为噪声信号，例如，当振动信号的幅度超过某一阈值并且持续了一段时间t时(例如，时间阈值t大于1ms)，则可认为该振动信号不是噪声信号。这种结合幅度阈值和持续时间t来判断噪声信号的方式，能够将用户在按键上无意识的滑动或突 然的碰撞而产生的振动信号识别为噪声信号，从而尽早过滤掉无效的振动信号。
进一步地，对于判断为非噪声的振动信号进行切割，切割方法是将切割起点和切割终点之间的信号提取出来。具体地，首先，确定切割起点，例如，将切割起点选择在振动信号幅度超过某一幅度阈值的第一个点之前，取N1个采样点(其中N1为大于等于0的整数)；其次，确定切割终点，例如，将切割终点选择在振动信号连续M个采样点(其中M是大于等于0的整数，具体需要根据采样频率和设备灵敏度确定)低于某一幅度阈值的第一个点之后，取N2个采样点(其中N2为大于等于M的整数)。
通过对振动信号进行切割，提取切割起点和切割终点之间的信号作为待分析的目标信号，能够滤除用户在开始按键后一段时间和按键结束前一段时间内的信号，这段时间内的振动信号对识别按键类型的意义不大，提早滤除能够提高识别的精确度和识别速度。
需说明的是，本发明对N1、M、N2、幅度阈值、时间阈值等的具体取值不作限制，本领域的技术人员可根据采样频率和可穿戴设备的灵敏度设定合适的值，并且不同情况下的幅度阈值等可以设置为相同或不同的值，例如，判断是否为噪声信号使用的幅度阈值和信号切割中使用的幅度阈值可以相同或不同，确定切割起点和切割终点时使用的幅度阈值可以相同或不同。
步骤S130，对经切割的振动信号进行归一化处理。
进一步地，可对切割后的振动信号进行归一化处理，例如基于信号振幅进行归一化调整。
在一个实施例中，对切割的振动信号进行归一化处理包括：
寻找信号的峰值，将接收到的信号与训练集的所有信号进行峰值对齐，以保证信号之间的时序同步；
使用归一化方法，对切割后的信号进行归一化处理，例如，归一化处理的计算公式表示为：
Figure PCTCN2019078870-appb-000001
其中，x和y分别对应归一化前后的信号，x max和x min分别对应归一化前的最大值和最小值，y max和y min分别对应归一化后的最大值和最小值。
通过归一化处理，能够避免用户敲击按键时由于敲击力度不同而导致 的信号特征差异过大，从而适用于不同用户、不同场景。
步骤S140，对经归一化处理的信号进行第二次特征提取，获得降维的振动信号。
在此步骤S140中，通过降维和维度筛选进行第二次特征提取，以进一步去除信号的抖动。
在一个实施例中，使用Haar小波进行降维和维度筛选，具体包括：对信号使用Haar小波进行处理，只保留趋势子信号，从而去除信号的抖动，并且将特征维度缩小一半；计算信号各个维度之间的方差，如果方差小于某一方差阈值，则认为该维度为特征维度，否则，认为该维度不是特征维度，并将非特征维度滤除。
该实施例中的方差阈值可根据按键使用场景，通过统计设定合适值，例如，将方差阈值预置为常数。
步骤S150，将降维的振动信号输入到经训练的神经网络分类模型，获得振动信号对应的按键类型。
在本发明实施例中，神经网络分类模型通过训练获得，其中，基于训练集训练神经网络分类模型时，输入的振动信号特征是对振动采样的时序信号序列，输出的分类标签是如图2所示的按键所对应的字母或者符号，神经网络分类模型可采用LSTM(长短时记忆网络)或径向基函数神经网络等。
由于不同类型按键的振动信号在时域上存在一定的差异性，各类型按键对应的振动信号的波峰数量、波谷数量以及形状不完全一致，基于这种局部差异性，在一个优选实施例中，使用具有局部学习能力的径向基函数神经网络，这种拥有局部学习能力的神经网络可以避免由于神经元个数设计过多而引发的过度拟合现象。另一方面，振动信号属于非线性曲线，它们在低维空间内线性不可分，径向基函数神经网络的基本思想是，用径向基函数作为隐单元的“基”构成隐含层空间，隐含层对输入矢量进行变换，将低维的模式输入数据变换到高维空间内，使得在低维空间内的线性不可分问题在高维空间内线性可分，进而使得振动信号被很好地分类。
图3示出了径向基函数的神经网络结构，该网络结构包括三层，其中，第一层为输入层，由信号源节点组成；第二层为隐含层(隐藏层)，隐含层中神经元的变换函数即径向基函数，其是对中心点径向对称且衰减的非负线性函数，该变换函数是局部响应函数；第三层是对输出模式做出的响 应。在图3中，[x1,x2,…,xn]为输入层单元及输入信号的特征以及相应的标签，[w1,w2,…,wN]是需要训练的隐含层至输出层的连接权值，B为阈值，用于调整神经元的灵敏度，Y是输出层单元，即输出的按键分类。
在一个实施例中，径向基函数神经网络的激活函数采用高斯函数，由于高斯分布函数的特点，径向基函数神经网络每次输入时，只有和输入样本向量较为接近的神经元才会活跃起来，对应的权值才会更新，而与样本距离较远的神经元不会被激活，这样可以使得神经元个数设计不会过多，进而避免了过拟合现象。
例如，激活函数表示为
Figure PCTCN2019078870-appb-000002
其中，||x p-c i||为欧式范数，c i是隐藏层第i个节点的高斯函数中心，x p是输入向量(p＝1,2,…,n)，σ为高斯函数的方差。利用该激活函数对于按键的声音信号能够加快训练时的收敛速度，使得训练要求的计算资源更小，从而更容易适用于广泛的可穿戴智能设备。
本发明实施例采用的径向基函数神经网络需要求解三个参数，分别是激活函数的中心、方差以及隐含层到输出层的权值。为了使得神经网络在训练集数据进入时能够快速发生变化，基于振动信号在多个维度上特征不同的特点，利用振动信号的每个特征在训练过程中直接使激活函数的中心发生变化。
在一个实施例中，选用自组织选取中心的无监督学习法，可以自主学习并计算激活函数的中心，基于自组织选取中心的无监督学习法求解基函数中心c的过程包括：
步骤S151，随机选取m个训练样本作为聚类中心c i(i＝1,2,…,m)；
步骤S152，将输入的训练样本集按最近邻规则分组；
具体地，按照x p与中心c i之间的欧式距离将x p分配到输入样本的各个聚类集合Θ p(p＝1,2,…,n)中。
步骤S153，重新调整聚类中心。
具体地，计算各个聚类集合Θ p中训练样本的平均值，即新的聚类中心c i，如果新的聚类中心不再发生变化，则所得到的c i即为径向基函数神经网络最终的基函数中心，否则返回步骤S151，进入下一轮的聚类中心求解。
在一个实施例中，运用最小二乘法计算隐含层到输出层的权值，这使 得神经网络输出层对权值而言是线性的，从而显著加快学习速度并避免局部极小能力。隐含层到输出层的权值表达式如下：
Figure PCTCN2019078870-appb-000003
其中，c max为所选取中心之间的最大距离。
步骤S160，根据按键类型显示用户的文字输入。
在识别出按键类型之后，即可对按键类型进行显示，获得输入的文字组合，参见图2(b)所示的显示效果。
为了增加本发明的普适性，以适用可穿戴设备的不同场景，本发明实施例可进一步包括对神经网络分类模型的校正过程，具体地，校正过程包括：
步骤S161，判断是否对用户按键类型出现了误判。
首先，根据用户的输入显示所识别出的字母组合，同时，根据识别出的输入字母组合，通过单词库进行匹配，显示出用户可能需要输入的英文单词，如果用户选择了与所识别出输入字母有区别的英文单词的组合，则说明对用户敲击的字母可能发生了误判或者是用户的输入出现了错误，其中，单词库是现有的中英文的全部有意义的字母组合，并且按照字母顺序排列。
在一个实施例中，根据单词库进行匹配的方法是，根据识别出的用户敲击的字母和所需要输入的语言，从单词库中挑选出包含任意多个相同字母的所有单词，并对这些单词进行打分，打分方法是
Figure PCTCN2019078870-appb-000004
其中，KEY same表示与识别出的用户输入字母相同的字母个数，KEY total为用户总共输入的单词总数。在打完分数之后，对这些单词根据分数由大到小进行排序，以用于用户选择。
步骤S162，根据误判情况确定是否重新训练神经网络分类模型。
在此步骤中，对误判情况进行判断，对于识别出来的用户输入的字母组合与用户所选择的字母组合不相同的单词中的每个字母进行对比，寻找到不同的字母后，暂时认为该字母为误判的字母，并且对误判字母的振动信号数据进行记录。如果是同一个字母第2次被记录为误判字母时，将第2次误判的振动信号数据和第1次误判的振动信号数据进行对比，判断两次的振动信号数据是否有相似性，如果有相似性，则记录误判的振动信号 数据，如果没有相似性，则丢弃之前误判的数据。根据这种方式，如果对于同一个字母超过某一次数阈值(例如3次)的误判的振动信号数据都具有相似性，则认为是可穿戴设备的使用环境等因素发生了稳定性的改变，因此，将记录为误判的振动信号数据和对应的分类标签重新放入网络中，对神经网络分类模型进行重新训练，生成新的经训练的神经网络分类模型。
在一个实施例中，对于两组振动信号相似性的判断方式是，计算两组信号的相关性系数，即计算两组信号的皮尔逊相关系数ρ，当ρ大于某一阈值ρ threshold(其中，0＜ρ threshold≤1)，则判断为两组信号有相似性，否则，认为该两组信号没有相似性。
通过上述的对神经网络分类模型的校正过程，能够及时发现并修正可穿戴设备使用环境等因素发生改变而导致的用户输入误判情况，从而进一步提高本发明的普适性。
图4示出了根据本发明一个实施例的基于骨传导的智能输入系统。该系统400包括振动信号感知模块410、处理模块420、输出模块430。
振动信号感知模块410，包括骨骼振动信号的接收端，用于接收和采集骨骼振动的信号，并进一步进行低通滤波器的滤波和信号的放大，最后转化为数字信号。
处理模块420用于对采集到的振动信号进行预处理，完成信号的滤波和切割，再进行归一化处理和特征提取等，以消除敲击力度不同所产生的干扰和对特征维度进行缩小和筛选，并利用经训练的神经网络分类模型对用户当前敲击的按键进行分类和匹配，以检测用户的输入。
输出模块430，用于实时输出并显示检测到的当前用户输入的敲击字母或符号等。
可选地，该系统400还包括校正模块440，用于实现本发明实施例的校正过程。
需要说明的是，上述的振动信号感知模块410、处理模块420、输出模块430和校正模块440可采用一个功能单元或多个功能单元实现。本发明对此不进行限制，只有其能够实现本发明实施例的功能即可。
例如，振动信号感知模块410包括：信号采集单元，用于对接收的骨骼振动信号进行数据采集；信号提取单元，用于提取振动信号数据；过滤和放大单元，用于对提取得到的振动信号数据进行过滤和放大处理；模数转换单元，用于将模拟信号转化成数字信号。
例如，处理模块420包括：滤波与切割单元，用于对信号进行去噪和切割；归一化单元，用于消除用户敲击不同力度产生的影响；特征筛选单元，用于对相关的特征进行提取和筛选；分类单元，用于根据用户按键的振动信号特征对按键类型进行分类。
例如，校正模块440包括：词典匹配单元，用于基于用户的输入，根据单词库匹配相应的单词；误判记录单元，用于记录被确定为误判的振动信号数据；重新训练单元，用于重新训练原有的神经网络分类模型。
例如，输出模块330包括输出单元，用于显示用户输入的分类结果。
本发明实施例的基于骨传导的智能输入方法和输入系统，能够实现可穿戴智能设备的QWERTY虚拟全键盘，利用振动传感器接收特定场景的手的骨骼中的振动信号，例如，在固体桌面的纸质键盘上敲击时候骨骼所产生的振动信号，然后使用归一化进行特征提取，利用Haar小波处理进行降维和维度筛选等，显著提高了按键类型识别的精准度。此外，本发明实施例通过机器学习的分类方法进行分类，能够检测出用户敲击对应的按键类型，是一种高响应速度、高灵敏度和高精确度的智能可穿戴设备的输入方法。
综上所述，本发明利用了人体解剖学的特性，提出一种更便捷而且低成本的智能输入方法，将键盘建立在人的骨骼特征稳定性和唯一性的传播介质上，通过人对桌面的敲击借助人体的骨骼，能够实现可穿戴智能设备的低耗能、快速和高精度的文本输入，实现了可穿戴设备的虚拟键盘的实现方法。此外，本发明还通过设计校正过程，进一步提高了智能输入的稳定性和鲁棒性。
图5是根据本发明一个实施例的电子设备500，包括存储器510、处理器520及存储在存储器510上并可在处理器520上运行的计算机程序511，处理器520执行计算机程序511时实现本发明实施例的基于骨传导的智能输入方法，例如包括以下步骤：采集用户按键的振动信号并对振动信号进行第一次特征提取；对经过第一次特征提取的信号进行滤波和切割，获得经切割的振动信号；对经切割的振动信号进行归一化处理；对归一化后的信号进行第二次特征提取，获得降维的振动信号；将降维的振动信号输入到训练的神经网络分类模型，获得振动信号对应的按键类型；对按键类型进行显示，获得文字输入等。
由于本发明描述的电子设备是实施本发明实施例中一种基于骨传导 的智能输入方法的设备，故而基于本发明实施例中所介绍的方法，本领域所属技术人员能够了解本实施例的电子设备的具体实施方式以及其各种变化形式，所以对于该电子设备如何实现本发明实施例中的方法不再详细介绍，只要本领域所属技术人员实施本发明实施例中的方法所采用的设备，都属于本发明所欲保护的范围。电子设备包括但限于可穿戴设备，例如，智能手表、谷歌眼镜、智能手环等。
需要说明的是，虽然上文按照特定顺序描述了各个步骤，但是并不意味着必须按照上述特定顺序来执行各个步骤，实际上，这些步骤中的一些可以并发执行，甚至改变顺序，只要能够实现所需要的功能即可。并且某些步骤对于实现本发明的精神并不是必须的，例如，滤波、去噪过程等。
本发明可以是系统、方法和/或计算机程序产品。计算机程序产品可以包括计算机可读存储介质，其上载有用于使处理器实现本发明的各个方面的计算机可读程序指令。
计算机可读存储介质可以是保持和存储由指令执行设备使用的指令的有形设备。计算机可读存储介质例如可以包括但不限于电存储设备、磁存储设备、光存储设备、电磁存储设备、半导体存储设备或者上述的任意合适的组合。计算机可读存储介质的更具体的例子(非穷举的列表)包括：便携式计算机盘、硬盘、随机存取存储器(RAM)、只读存储器(ROM)、可擦式可编程只读存储器(EPROM或闪存)、静态随机存取存储器(SRAM)、便携式压缩盘只读存储器(CD-ROM)、数字多功能盘(DVD)、记忆棒、软盘、机械编码设备、例如其上存储有指令的打孔卡或凹槽内凸起结构、以及上述的任意合适的组合。
以上已经描述了本发明的各实施例，上述说明是示例性的，并非穷尽性的，并且也不限于所披露的各实施例。在不偏离所说明的各实施例的范围和精神的情况下，对于本技术领域的普通技术人员来说许多修改和变更都是显而易见的。本文中所用术语的选择，旨在最好地解释各实施例的原理、实际应用或对市场中的技术改进，或者使本技术领域的其它普通技术人员能理解本文披露的各实施例。

Claims (12)

  1. 一种基于骨传导的智能输入方法，包括以下步骤：
    步骤S1，接收来自于按键的骨传导的振动信号并进行特征提取，获得经特征提取后的振动信号；
    步骤S2，将所述经特征提取后的振动信号的时序信号序列输入至经训练的神经网络分类模型，识别出所述骨传导的振动信号对应的按键类型；
    步骤S3，基于识别出的按键类型确定用户输入的文字信息。
  2. 根据权利要求1所述的方法，其中，步骤S1包括第一次特征提取，该第一次体征提取包括：
    通过振动贴片传感器对接收的振动信号进行采集；
    对于采集到的振动信号，使用巴氏低通滤波器去除高频的电噪声；
    对于去除高频噪声的振动信号，使用信号放大器进行放大；
    将放大后的振动信号通过模数转换器转换为对应的数字信号。
  3. 根据权利要求2所述的方法，其中，在所述第一次特征提取之后还包括：
    利用陷波滤波器去除高频和低频的信号噪声；
    对于去除高频和低频的信号噪声的振动信号，判断其是否为噪声信号；
    对于判断为非噪声信号的振动信号，确定切割起点和切割终点，提取所述切割起点和所述切割终点之间的信号作为待分析的目标信号。
  4. 根据权利要求3所述方法，其中，将振动信号的幅度超过幅度阈值并且持续时间超过时间阈值的信号判断为非噪声信号。
  5. 根据权利要求3所述的方法，其中，对于判断为非噪声信号的振动信号，根据以下步骤确定所述切割起点和切割终点：
    将所述切割起点选择在振动信号幅度超过第一幅度阈值的第一个点之前，取N1个采样点；
    将所述切割终点选择在振动信号连续M个采样点低于第二幅度阈值的第一个点之后，取N2个采样点，其中，N1、M、N2为正整数。
  6. 根据权利要求3所述的方法，其中，步骤S1还包括对于所述待分析的目标信号执行以下步骤：
    将所述待分析的目标信号的峰值与训练所述神经网络分类模型的训练集中的所有振动信号进行峰值对齐；
    使用归一化的方法，对所述待分析的目标信号进行幅度归一化处理。
  7. 根据权利要求6所述的方法，其中，还包括对于归一化处理的信号进行第二次特征提取，该第二次特征提取包括：
    将信号使用Haar小波进行处理之后，保留趋势子信号；
    计算信号各个维度之间的方差，将方差大于方差阈值的特征维度滤除。
  8. 根据权利要求1所述的方法，其中，步骤S3还包括：
    对比识别出的用户输入的字母组合和用户选择的字母组合，将不相同的字母标记为误判的字母；
    在同一字母多次被标记为误判的字母的情况下，根据其被标记时对应的振动信号的相似性，确定是否对所述神经网络分类模型重新进行训练。
  9. 根据权利要求8所述的方法，其中，对于两组振动信号，根据以下步骤确定是否具有相似性：
    计算该两组振动信号的皮尔逊相关系数ρ，当ρ大于阈值ρ threshold时，确定该两组振动信号有相似性，其中，0＜ρ threshold≤1。
  10. 根据权利要求1所述的方法，其中，所述神经网络分类模型是基于径向基函数的神经网络分类模型。
  11. 一种基于骨传导的智能输入系统，包括：
    振动信号处理单元：用于接收来自于按键的骨传导的振动信号并进行特征提取，获得经特征提取后的振动信号；
    分类识别单元：用于将所述经特征提取后的振动信号的时序信号序列输入至经训练的神经网络分类模型，识别出所述骨传导的振动信号对应的按键类型；
    输出单元：用于基于识别出的按键类型确定用户输入的文字信息。
  12. 一种计算机可读存储介质，其上存储有计算机程序，其中，该程序被处理器执行时实现根据权利要求1至10中任一项所述方法的步骤。
PCT/CN2019/078870 2019-03-20 2019-03-20 一种基于骨传导的智能输入方法和系统 WO2020186477A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
PCT/CN2019/078870 WO2020186477A1 (zh) 2019-03-20 2019-03-20 一种基于骨传导的智能输入方法和系统

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2019/078870 WO2020186477A1 (zh) 2019-03-20 2019-03-20 一种基于骨传导的智能输入方法和系统

Publications (1)

Publication Number Publication Date
WO2020186477A1 true WO2020186477A1 (zh) 2020-09-24

Family

ID=72518905

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2019/078870 WO2020186477A1 (zh) 2019-03-20 2019-03-20 一种基于骨传导的智能输入方法和系统

Country Status (1)

Country Link
WO (1) WO2020186477A1 (zh)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113703568A (zh) * 2021-07-12 2021-11-26 中国科学院深圳先进技术研究院 手势识别方法、手势识别装置、手势识别系统及存储介质

Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108681709A (zh) * 2018-05-16 2018-10-19 深圳大学 基于骨传导振动与机器学习的智能输入方法及系统

Patent Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108681709A (zh) * 2018-05-16 2018-10-19 深圳大学 基于骨传导振动与机器学习的智能输入方法及系统

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113703568A (zh) * 2021-07-12 2021-11-26 中国科学院深圳先进技术研究院 手势识别方法、手势识别装置、手势识别系统及存储介质

Similar Documents

Publication Publication Date Title
Lee et al. Smart wearable hand device for sign language interpretation system with sensors fusion
Yu et al. Writinghacker: audio based eavesdropping of handwriting via mobile devices
Savur et al. Real-time american sign language recognition system using surface emg signal
EP2802975B1 (en) Intelligent touchscreen keyboard with finger differentiation
Aryanie et al. American sign language-based finger-spelling recognition using k-Nearest Neighbors classifier
Tinwala et al. Eyes-free text entry with error correction on touchscreen mobile devices
Schrapel et al. Pentelligence: Combining pen tip motion and writing sounds for handwritten digit recognition
CN109933202B (zh) 一种基于骨传导的智能输入方法和系统
Roth et al. Biometric authentication via keystroke sound
CN108182418B (zh) 一种基于多维声波特征的键击识别方法
CN110058689A (zh) 一种基于脸部振动的智能设备输入方法
Yin et al. Learning to recognize handwriting input with acoustic features
CN106934020A (zh) 一种基于多域实体索引的实体链接方法
Mohandes et al. Arabic sign language recognition an image-based approach
WO2020186477A1 (zh) 一种基于骨传导的智能输入方法和系统
Youme et al. Generalization of bangla sign language recognition using angular loss functions
Sahoo et al. Indian sign language recognition using skin color detection
Goyal Indian sign language recognition using mediapipe holistic
Zhao Learning user keystroke patterns for authentication
Yu et al. Mobile devices based eavesdropping of handwriting
Modanwal et al. Development of a new dactylology and writing support system especially for blinds
Robert et al. A review on computational methods based automated sign language recognition system for hearing and speech impaired community
Sadak et al. Dynamic and static feature fusion for increased accuracy in signature verification
CN111310546B (zh) 在线笔迹认证中一种书写节奏特征的提取及认证方法
Zikky et al. Development of educational game for recognizing indonesian sign language (SIBI) and breaking down the communication barrier with deaf people

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19920206

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 19920206

Country of ref document: EP

Kind code of ref document: A1