WO2022061500A1 - 一种基于电容触摸屏的人机交互系统和交互方法 - Google Patents

一种基于电容触摸屏的人机交互系统和交互方法 Download PDF

Info

Publication number
WO2022061500A1
WO2022061500A1 PCT/CN2020/116764 CN2020116764W WO2022061500A1 WO 2022061500 A1 WO2022061500 A1 WO 2022061500A1 CN 2020116764 W CN2020116764 W CN 2020116764W WO 2022061500 A1 WO2022061500 A1 WO 2022061500A1
Authority
WO
WIPO (PCT)
Prior art keywords
signal
touch screen
capacitance change
capacitive touch
effective part
Prior art date
Application number
PCT/CN2020/116764
Other languages
English (en)
French (fr)
Inventor
伍楷舜
关茂柠
Original Assignee
深圳大学
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 深圳大学 filed Critical 深圳大学
Priority to PCT/CN2020/116764 priority Critical patent/WO2022061500A1/zh
Publication of WO2022061500A1 publication Critical patent/WO2022061500A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
    • G06F3/041—Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
    • G06F3/044—Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means by capacitive means

Definitions

  • the present invention relates to the technical field of human-computer interaction, and more particularly, to a human-computer interaction system and interaction method based on a capacitive touch screen.
  • the traditional capacitive touch screen input uses a finger to touch the screen for input, but this input method can only detect whether the finger touches the screen, but cannot detect which finger of the user touches the screen.
  • the purpose of the present invention is to overcome the above-mentioned defects of the prior art, and provide a human-computer interaction system and interaction method based on a capacitive touch screen, which is a new technology scheme for screen interaction by using the capacitance change signals generated by different fingers touching the capacitive touch screen. .
  • a human-computer interaction method based on a capacitive touch screen includes the following steps:
  • the screen interaction function of the electronic device is controlled according to the set association relationship between the finger type and the touch screen function.
  • a human-computer interaction system based on a capacitive touch screen includes:
  • Signal acquisition unit used to collect capacitance change signals generated by touching the capacitive touch screen with a finger by using an electronic device with a capacitive touch screen;
  • Signal processing unit used to extract the effective part of the capacitance change signal, the effective part representing the capacitance signal change between the start point and the end point of the finger touching the capacitive screen;
  • Feature extraction unit for extracting Mel cepstral coefficients for the effective part of the capacitance change signal according to the recognition degree of touch sensing;
  • Classification and identification unit used to use the Mel cepstral coefficients as the input features of the trained Hidden Markov Model to identify the type of fingers that touch the capacitive touch screen;
  • Human-computer interaction unit used to control the screen interaction function of the electronic device according to the set association between the finger type and the touch screen function.
  • the present invention has the advantages of using the capacitance change signals generated by different fingers to touch the capacitive touch screen to perform screen interaction, which solves the problem of difficult interaction of electronic devices such as smart watches;
  • the Cepstral coefficient is used as the input feature, and the hidden Markov chain model is trained to solve the problem that the capacitance change signal changes due to the change of the duration of the same finger touching the capacitive touch screen.
  • FIG. 1 is a flowchart of a method for human-computer interaction based on a capacitive touch screen according to an embodiment of the present invention
  • Fig. 2 is a schematic process diagram of a human-computer interaction method based on a capacitive touch screen according to an embodiment of the present invention.
  • FIG. 3 is a schematic diagram of the working principle of a capacitive touch screen according to an embodiment of the present invention.
  • the human-computer interaction method based on a capacitive touch screen includes the following steps:
  • step S110 a capacitance change signal generated by the finger touching the capacitive touch screen is collected.
  • an electronic device with a capacitive touch screen uses an electronic device with a capacitive touch screen to collect a capacitance change signal generated by a finger touching the capacitive touch screen.
  • the electronic device may be a wearable device, including but not limited to a smart watch, a smart bracelet, a smart phone, and the like.
  • the electronic device may also be other electronic devices provided with capacitive touch screens, such as intelligent robots, tablet computers, and the like.
  • a smart watch will be used as an example for introduction.
  • the capacitance change signal of the touch point on the screen can be acquired by calling the underlying API of the smart watch Android system.
  • the capacitive screen uses human body induction to perform contact detection control, and does not require direct contact or only slight contact, and locates the touch coordinates by detecting the induced current. Therefore, corresponding to the induced current collected by the capacitance change signal, the capacitance change caused by the finger touching the screen of the smart watch can be reflected by the change in the magnitude of the induced current at the touch point, thereby obtaining the capacitance change signal.
  • the present invention reflects the change of the capacitance signal by collecting the induced current generated by the finger touching the capacitive screen, so the capacitance change signal described in this article refers to the change signal of the induced current at the touch point of the touch screen of the smart watch.
  • Step S120 extracting the effective part of the capacitance change signal.
  • the effective part is extracted by further processing.
  • the effective part is used to characterize the capacitance signal change between the start point and the end point of the finger touching the capacitive screen.
  • an energy-based dual-threshold endpoint detection method is used to detect the valid portion of the signal, including:
  • step S201 after the smart watch collects the capacitance change signal, it uses a Butterworth bandpass filter to filter it, and the cutoff frequencies are 10 Hz and 1000 Hz, respectively.
  • Step S202 calculating the short-term energy of the capacitance change signal.
  • E is the short-term energy of the frame signal
  • L is the length of the frame signal
  • S(i) is the amplitude of the capacitance change signal
  • t is the frame number index.
  • Step S204 set the maximum interval maxInter between the signal peaks of the same signal, and the minimum length minLen of the signal.
  • the maximum interval maxInter between signal peaks and the minimum length minLen of the signal can be determined empirically or by simulation.
  • Step S207 repeating S205 and S206 until all signal peaks in the entire signal segment are found.
  • Step S208 if the interval between the two signal peaks is less than maxInter, then combine the two signal peaks.
  • Step S209 repeat S208 until the interval between all signal peaks is greater than maxInter.
  • Step S210 if the length of the signal peak is less than minLen, the signal peak is directly discarded.
  • step S211 the number of signal peaks finally obtained should be 1, and the signal peak is an effective part of the signal.
  • step S212 if the number of signal peaks obtained in S211 is greater than 1, the signal should be regarded as an invalid signal and directly discarded.
  • step S120 through filtering, determining the starting point and the ending point, and combining signal peaks, ineffective parts such as noise and unconscious sliding of fingers can be effectively removed, while effective parts that can better reflect the touch characteristics of the user's fingers are retained, thereby improving the follow-up Accuracy and efficiency of finger classification recognition.
  • Step S130 for the effective part of the capacitance change signal, extract the Mel cepstral coefficient according to the recognition degree of the touch sensing.
  • extracting the Mel cepstral coefficients of the signal as features specifically includes:
  • step S301 pre-emphasis, framing and windowing are performed on the effective part of the extracted capacitance change signal.
  • the pre-emphasis coefficient is 0.96
  • the frame length is 20ms
  • the frame shift is 6ms
  • the window function is Hamming window.
  • Step S302 performing Fast Fourier Transform (FFT) on each frame of signal to obtain a corresponding frequency spectrum.
  • FFT Fast Fourier Transform
  • Step S303 passing the obtained spectrum through a Mel filter bank to obtain a Mel spectrum.
  • the mel filter frequency range is 10Hz to 1000Hz, and the number of filter channels is 28.
  • Step S304 take the logarithm of the obtained Mel spectrum, then perform discrete cosine transform (DCT), and finally take the first 14 coefficients as Mel cepstral coefficients (MFCCs, or Mel frequency cepstral coefficients).
  • DCT discrete cosine transform
  • MFCCs Mel cepstral coefficients
  • 14 Mel cepstral coefficients are selected according to the recognition degree of finger touch sensing. It should be understood that more or less Mel cepstral coefficients may also be selected.
  • Step S140 using the extracted Mel cepstral coefficients as input features to train a hidden Markov model.
  • the Baum-Welch algorithm uses the Baum-Welch algorithm to train a hidden Markov model, where the number of states of the hidden Markov model is 3, and each state has 2 mixture Gaussian probability density functions, including: Initialize the parameters; calculate the forward and backward probability matrices; calculate the transition probability matrix; calculate the mean and variance of each Gaussian probability density function; calculate the weight of each Gaussian probability density function; calculate the output probability of all observation sequences, and accumulate to get the sum output probability.
  • the Hidden Markov Model belongs to the prior art and will not be repeated here.
  • a corresponding hidden Markov model can be generated, so as to obtain multiple hidden Markov models, that is, for the five fingers of a hand, for each finger of the user
  • the training generates a corresponding hidden Markov model and obtains 5 hidden Markov models.
  • the number of iterations of the training process can be set according to the requirements for computing resources and training time. For example, considering the saving of computing resources, the training process is iterated only once.
  • the effectiveness of the trained hidden Markov model can be evaluated by using the test data, for example, classifying and identifying the test data, including: using the Viterbi algorithm to calculate the output probability of the test data for each hidden Markov model, and The best state path is given; the category corresponding to the hidden Markov model with the largest output probability is the classification result of the test data.
  • the hidden Markov model is selected, and the Mel cepstral coefficient feature is used as the observation sequence, and the change of the capacitive charging signal caused by the change of the touch screen duration of the same finger can also be accurately identified, and for different Finger types can be accurately distinguished. This is because the duration of the user touching the touch screen of the smart watch is different, so the length of the effective part of the detected capacitance change signal of the same finger is also different. But for the Hidden Markov Model, it allows the length of each sample of the same class to be inconsistent, so it can also accurately identify the change of the capacitive charging signal caused by the change of the same finger touching the screen duration.
  • Step S150 using the trained Hidden Markov Model to identify the finger type of the user to be detected touching the capacitive touch screen.
  • the training process of the hidden Markov model can also be performed offline on information processing equipment such as a server, a cloud, and a computer.
  • the trained Hidden Markov Model can be integrated into electronic equipment to realize human-computer interaction, that is, to obtain the capacitance change signal generated by the user's finger touching the capacitive touch screen in real time and extract the Mel cepstral coefficient features.
  • the Mel cepstral features are fed into a trained Hidden Markov Model to identify the user's finger type.
  • step S160 the screen interaction function of the electronic device is implemented according to the set association relationship between the finger type and the touch screen function.
  • the human-computer interaction is realized according to the recognition result and the preset association relationship between the finger type and the touch screen function.
  • the user touches the capacitive touch screen with different fingers corresponding to different functions, thereby expanding the interactive function of the screen.
  • touch with the index finger means save, middle finger means open, ring finger means delete, etc.
  • the screen interaction function of the electronic device can be expanded.
  • the present invention also provides a human-computer interaction system based on a capacitive touch screen, which is used to implement one or more aspects of the above method.
  • the system includes: a signal acquisition unit, which is used for using an electronic device with a capacitive touch screen to collect capacitance change signals generated by a finger touching the capacitive touch screen; a signal processing unit, which is used for extracting the effective part of the capacitance change signal, the effective part of the capacitance change signal Partially characterizes the capacitance signal change between the start point and the end point of the finger touching the capacitive screen; feature extraction unit: it is used to extract the Mel cepstral coefficient according to the recognition degree of touch sensing for the effective part of the capacitance change signal; classification and identification a unit, which is used to use the Mel cepstral coefficients as the input features of the trained Hidden Markov model to identify the type of fingers touching the capacitive touch screen; the human-computer interaction unit, which is used to identify the type of fingers that touch the
  • the technical solution provided by the present invention utilizes the capacitance change signals generated by different fingers touching the capacitive touch screen to perform screen interaction, which well solves the problem that the screen of a smart watch is too small and the interaction is difficult.
  • the present invention uses the extracted Mel cepstral coefficient of the capacitance change signal as the input feature, and can accurately identify the input of different fingers on the capacitive touch screen by training the hidden Markov chain model, thereby expanding the screen interaction function of the electronic device.
  • the present invention may be a system, method and/or computer program product.
  • the computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for causing a processor to implement various aspects of the present invention.
  • a computer-readable storage medium may be a tangible device that can hold and store instructions for use by the instruction execution device.
  • the computer-readable storage medium may be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing.
  • Non-exhaustive list of computer readable storage media include: portable computer disks, hard disks, random access memory (RAM), read only memory (ROM), erasable programmable read only memory (EPROM) or flash memory), static random access memory (SRAM), portable compact disk read only memory (CD-ROM), digital versatile disk (DVD), memory sticks, floppy disks, mechanically coded devices, such as printers with instructions stored thereon Hole cards or raised structures in grooves, and any suitable combination of the above.
  • RAM random access memory
  • ROM read only memory
  • EPROM erasable programmable read only memory
  • flash memory static random access memory
  • SRAM static random access memory
  • CD-ROM compact disk read only memory
  • DVD digital versatile disk
  • memory sticks floppy disks
  • mechanically coded devices such as printers with instructions stored thereon Hole cards or raised structures in grooves, and any suitable combination of the above.
  • Computer-readable storage media are not to be construed as transient signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (eg, light pulses through fiber optic cables), or through electrical wires transmitted electrical signals.
  • the computer readable program instructions described herein may be downloaded to various computing/processing devices from a computer readable storage medium, or to an external computer or external storage device over a network such as the Internet, a local area network, a wide area network, and/or a wireless network.
  • the network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and/or edge servers.
  • a network adapter card or network interface in each computing/processing device receives computer-readable program instructions from a network and forwards the computer-readable program instructions for storage in a computer-readable storage medium in each computing/processing device .
  • the computer program instructions for carrying out the operations of the present invention may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine dependent instructions, microcode, firmware instructions, state setting data, or instructions in one or more programming languages.
  • Source or object code written in any combination, including object-oriented programming languages, such as Smalltalk, C++, etc., and conventional procedural programming languages, such as the "C" language or similar programming languages.
  • the computer readable program instructions may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer, or entirely on the remote computer or server implement.
  • the remote computer may be connected to the user's computer through any kind of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (eg, using an Internet service provider through the Internet connect).
  • LAN local area network
  • WAN wide area network
  • custom electronic circuits such as programmable logic circuits, field programmable gate arrays (FPGAs), or programmable logic arrays (PLAs)
  • FPGAs field programmable gate arrays
  • PDAs programmable logic arrays
  • Computer readable program instructions are executed to implement various aspects of the present invention.
  • These computer readable program instructions may be provided to a processor of a general purpose computer, special purpose computer or other programmable data processing apparatus to produce a machine that causes the instructions when executed by the processor of the computer or other programmable data processing apparatus , resulting in means for implementing the functions/acts specified in one or more blocks of the flowchart and/or block diagrams.
  • These computer readable program instructions can also be stored in a computer readable storage medium, these instructions cause a computer, programmable data processing apparatus and/or other equipment to operate in a specific manner, so that the computer readable medium on which the instructions are stored includes An article of manufacture comprising instructions for implementing various aspects of the functions/acts specified in one or more blocks of the flowchart and/or block diagrams.
  • Computer readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other equipment to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other equipment to produce a computer-implemented process , thereby causing instructions executing on a computer, other programmable data processing apparatus, or other device to implement the functions/acts specified in one or more blocks of the flowcharts and/or block diagrams.
  • each block in the flowchart or block diagrams may represent a module, segment, or portion of instructions, which comprises one or more functions for implementing the specified logical function(s) executable instructions.
  • the functions noted in the blocks may occur out of the order noted in the figures. For example, two blocks in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved.
  • each block of the block diagrams and/or flowchart illustrations, and combinations of blocks in the block diagrams and/or flowchart illustrations can be implemented in dedicated hardware-based systems that perform the specified functions or actions , or can be implemented in a combination of dedicated hardware and computer instructions. It is well known to those skilled in the art that implementation in hardware, implementation in software, and implementation in a combination of software and hardware are all equivalent.

Abstract

本发明公开了一种基于电容触摸屏的人机交互方法和交互系统。该方法包括：使用具有电容触摸屏的电子设备采集手指触摸电容触摸屏所产生的电容变化信号；提取所述电容变化信号的有效部分，该有效部分表征手指触摸电容屏的起点和终点之间的电容信号变化；对于所述电容变化信号的有效部分，根据对触摸感应的辨识程度提取梅尔倒谱系数；以所述梅尔倒谱系数作为经训练的隐马尔可夫模型的输入特征，以识别出触摸电容触摸屏的手指类型；根据设定的手指类型与触屏功能的关联关系，控制所述电子设备的屏幕交互功能。利用本发明能够准确识别电容触摸屏不同手指的输入，进而扩展电子设备的人机交互功能。

Description

一种基于电容触摸屏的人机交互系统和交互方法 技术领域
本发明涉及人机交互技术领域，更具体地，涉及一种基于电容触摸屏的人机交互系统和交互方法。
背景技术
传统的电容触摸屏输入是利用手指触摸屏幕来进行输入，但是这种输入方式只能检测手指是否触摸屏幕，而不能检测是用户的哪一根手指触摸了屏幕。
目前，有些研究人员通过在用户的手指上佩戴电磁铁来检测是用户的哪一根手指触摸了按键，但是这种实现方式需要用户在手指上佩戴电磁铁，导致使用成本增加，并降低了用户的使用体验。
发明内容
本发明的目的是克服上述现有技术的缺陷，提供一种基于电容触摸屏的人机交互系统和交互方法，其是利用不同手指触摸电容触摸屏所产生的电容变化信号来进行屏幕交互的新技术方案。
根据本发明的第一方面，提供一种基于电容触摸屏的人机交互方法。该方法包括以下步骤：
使用具有电容触摸屏的电子设备采集手指触摸电容触摸屏所产生的电容变化信号；
提取所述电容变化信号的有效部分，该有效部分表征手指触摸电容屏的起点和终点之间的电容信号变化；
对于所述电容变化信号的有效部分，根据对触摸感应的辨识程度提取梅尔倒谱系数；
以所述梅尔倒谱系数作为经训练的隐马尔可夫模型的输入特征，以识 别出触摸电容触摸屏的手指类型；
根据设定的手指类型与触屏功能之间的关联关系，控制所述电子设备的屏幕交互功能。
根据本发明的第二方面，提供一种基于电容触摸屏的人机交互系统。该系统包括：
信号获取单元：用于使用具有电容触摸屏的电子设备采集手指触摸电容触摸屏产生的电容变化信号；
信号处理单元：用于提取所述电容变化信号的有效部分，该有效部分表征手指触摸电容屏的起点和终点之间的电容信号变化；
特征提取单元：用于对于所述电容变化信号的有效部分，根据对触摸感应的辨识程度提取梅尔倒谱系数；
分类识别单元：用于以所述梅尔倒谱系数作为经训练的隐马尔可夫模型的输入特征，以识别出触摸电容触摸屏的手指类型；
人机交互单元：用于根据设定的手指类型与触屏功能之间的关联关系，控制所述电子设备的屏幕交互功能。
与现有技术相比，本发明的优点在于，利用不同手指触摸电容触摸屏产生的电容变化信号来进行屏幕交互，解决了智能手表等电子设备的交互困难的问题；以提取的电容变化信号的梅尔倒谱系数作为输入特征，通过训练隐马尔可夫链模型，解决同一个手指触摸电容触摸屏持续时间的改变而导致电容变化信号发生改变的问题。
通过以下参照附图对本发明的示例性实施例的详细描述，本发明的其它特征及其优点将会变得清楚。
附图说明
被结合在说明书中并构成说明书的一部分的附图示出了本发明的实施例，并且连同其说明一起用于解释本发明的原理。
图1根据本发明一个实施例的基于电容触摸屏的人机交互方法的流程图；
图2是根据本发明一个实施例的基于电容触摸屏的人机交互方法的过 程示意图。
图3是根据本发明一个实施例的电容式触摸屏的工作原理示意。
具体实施方式
现在将参照附图来详细描述本发明的各种示例性实施例。应注意到：除非另外具体说明，否则在这些实施例中阐述的部件和步骤的相对布置、数字表达式和数值不限制本发明的范围。
以下对至少一个示例性实施例的描述实际上仅仅是说明性的，决不作为对本发明及其应用或使用的任何限制。
对于相关领域普通技术人员已知的技术、方法和设备可能不作详细讨论，但在适当情况下，所述技术、方法和设备应当被视为说明书的一部分。
在这里示出和讨论的所有例子中，任何具体值应被解释为仅仅是示例性的，而不是作为限制。因此，示例性实施例的其它例子可以具有不同的值。
应注意到：相似的标号和字母在下面的附图中表示类似项，因此，一旦某一项在一个附图中被定义，则在随后的附图中不需要对其进行进一步讨论。
结合图1和图2所示，本发明实施例提供的基于电容触摸屏的人机交互方法包括以下步骤：
步骤S110，采集手指触摸电容触摸屏所产生的电容变化信号。
例如，使用具有电容触摸屏的电子设备采集手指触摸电容触摸屏所产生的电容变化信号。
电子设备可以是可穿戴设备，包括但不限于智能手表、智能手环、智能手机等。电子设备也可以是其他设有电容触摸屏的电子设备，如智能机器人、平板电脑等。为清楚起见，在下文的描述中，将以智能手表为例进行介绍。
在一个实施例中，对于电容变化信号采集，可通过调用智能手表安卓系统的底层API来获取屏幕触摸点的电容变化信号。
具体地，参见图3所示，电容屏利用人体感应进行触点检测控制，不 需要直接接触或只需要轻微接触，通过检测感应电流来定位触摸坐标。所以对应于电容变化信号采集的感应电流，可以通过触摸点感应电流大小的变化来反映出由于手指触摸智能手表的屏幕而导致的电容变化，从而得到电容变化信号。
由于在智能手表中，本发明是通过采集手指触摸电容式屏幕产生的感应电流来反映电容信号的变化，因此本文所述的电容变化信号指的是智能手表触摸屏的触摸点感应电流的变化信号。
步骤S120，提取电容变化信号的有效部分。
在采集到电容变化信号之后，通过进一步处理提取其中的有效部分。所述有效部分用于表征手指触摸电容屏的起点和终点之间的电容信号变化。
在一个实施例中，用基于能量的双门限端点检测法来检测信号的有效部分，具体包括：
步骤S201，智能手表采集到电容变化信号之后，使用巴特沃斯带通滤波器对其进行滤波，截止频率分别为10Hz和1000Hz。
步骤S202，计算出电容变化信号的短时能量。
例如，短时能量的计算表示为：
Figure PCTCN2020116764-appb-000001
其中，E是帧信号的短时能量，L是帧信号的长度，S(i)是电容变化信号的幅度，t是帧号索引。
步骤S203，计算噪声的平均能量，记为u，计算信号短时能量的标准差，记为σ；设置切断时的低门限为TL＝u+σ，高门限为TH＝u+3σ。
步骤S204，设置同一个信号的信号峰之间的最大间隔maxInter，信号的最小长度minLen。
例如，可根据经验或仿真确定信号峰之间的最大间隔maxInter，信号的最小长度minLen。
步骤S205，找出信号中能量最大的一帧信号且该帧信号的能量需要高于所设置的高门限TH＝u+3σ。
步骤S206，从该帧信号分别向左和向右延伸，直到下一帧信号的能量低于所设置的低门限TL＝u+σ，记录此时的帧位置，可以得到左边的帧位置为该信号峰的起点，右边的帧位置为该信号峰的终点，同时把信号中该信号峰所在位置的帧能量置为零。
步骤S207，重复S205和S206，直到找出整段信号中的所有信号峰。
步骤S208，若两个信号峰的间隔小于maxInter，则合并两个信号峰。
步骤S209，重复S208，直到所有信号峰之间的间隔都大于maxInter。
步骤S210，若信号峰的长度小于minLen，则直接舍弃该信号峰。
步骤S211，最后得到的信号峰的数量应该为1，且该信号峰即为信号的有效部分。
步骤S212，若S211得到的信号峰的数量大于1，则该信号应视为无效信号，直接舍弃。
在该步骤S120中，通过滤波、确定起点和终点、以及合并信号峰等处理，能够有效去除噪音、手指无意识滑动等无效部分，而保留更能体现用户手指触摸特点的有效部分，从而提高了后续手指分类识别的准确度和效率。
步骤S130，对于电容变化信号的有效部分，根据对触摸感应的辨识程度提取梅尔倒谱系数。
在一个实施例中，提取信号的梅尔倒谱系数作为特征，具体包括：
步骤S301，对提取到的电容变化信号的有效部分进行预加重、分帧和加窗。
例如，预加重的系数为0.96，帧长为20ms，帧移为6ms，窗函数为Hamming窗。
步骤S302，对每一帧信号进行快速傅里叶变换(FFT)得到对应的频谱。
步骤S303，将得到的频谱通过梅尔滤波器组得到梅尔频谱。
例如，梅尔滤波频率范围为10Hz到1000Hz，滤波器通道数为28。
步骤S304，对得到的梅尔频谱取对数，然后进行离散余弦变换(DCT)，最后取前14个系数作为梅尔倒谱系数(MFCCs，或称梅尔频率倒谱系数)。
在该实施例中，根据对手指触摸感应的辨识程度，选取了14个梅尔倒谱系数。应理解的是，也可以选取更多或更少的梅尔倒谱系数。
步骤S140，以提取的梅尔倒谱系数作为输入特征，训练隐马尔可夫模型。
例如，使用鲍姆-韦尔奇算法训练隐马尔可夫模型，其中隐马尔可夫模型的状态数为3，每个状态有2个混合高斯概率密度函数，包括：对隐马尔可夫模型的参数进行初始化；计算前、后向概率矩阵；计算转移概率矩阵；计算各个高斯概率密度函数的均值和方差；计算各个高斯概率密度函数的权重；计算所有观察序列的输出概率，并进行累加得到总和输出概率。隐马尔克夫模型属于现有技术，在此不再进行赘述。
在训练过程中，对于每种类型的手指，可生成一个对应的隐马尔可夫模型，从而获得多个隐马尔可夫模型，即对于一只手的五个手指，为用户的每一个手指都训练生成一个对应的隐马尔可夫模型，获得5个隐马尔可夫模型。此外，训练过程的迭代次数可根据对计算资源和训练时间的要求进行设置。例如，考虑到节约计算资源，训练过程只迭代了1次。
进一步的，可利用测试数据评估经训练的隐马尔可夫模型的有效性，例如，对测试数据进行分类识别，包括：利用维特比算法计算测试数据对于各个隐马尔可夫模型的输出概率，并给出最佳的状态路径；输出概率最大的隐马尔可夫模型所对应的类别即为该测试数据的分类结果。
在本发明实施例中，选择隐马尔可夫模型，并利用梅尔倒谱系数特征作为观察序列，对于同一个手指触摸屏幕持续时间改变而导致电容充电信号发生改变同样能够准确识别，并且对于不同手指类型能够准确区分。这是由于用户触摸智能手表的触摸屏的持续时间不一样，所以检测到的同一个手指的电容变化信号有效部分的长度也不一样。但是对于隐马尔可夫模型，它允许对于同一类的各个样本的长度大小可以不一致，因此对于同一个手指触摸屏幕持续时间改变而导致电容充电信号发生改变同样能够准确识别。
步骤S150，利用经训练的隐马尔可夫模型识别出待检测用户触摸电容触摸屏的手指类型。
应理解的是，隐马尔可夫模型的训练过程也可在服务器、云端、电脑等信息处理设备离线进行。在实际应用中，可将经训练的隐马尔可夫模型集成到电子设备中，用于实现人机交互，即实时获取用户手指触摸电容触摸屏产生的电容变化信号并提取梅尔倒谱系数特征，将梅尔倒谱系数特征输入经训练的隐马尔可夫模型，从而识别出用户的手指类型。
步骤S160，根据设定的手指类型与触屏功能的关联关系，实现电子设备的屏幕交互功能。
进一步地，根据识别结果，以及预先设定的手指类型与触屏功能的关联关系实现人机交互。
例如，预先设定用户用不同的手指触摸电容触摸屏对应不同的功能，从而拓展屏幕的交互功能。如用食指触摸表示保存，中指表示打开，无名指表示删除等。通过这种方式，可以扩展电子设备的屏幕交互功能。
相应地，本发明还提供一种基于电容触摸屏的人机交互系统，用于实现上述方法的一个方面或多个方面。例如，该系统包括：信号获取单元，其用于使用具有电容触摸屏的电子设备采集手指触摸电容触摸屏产生的电容变化信号；信号处理单元，其用于提取所述电容变化信号的有效部分，该有效部分表征手指触摸电容屏的起点和终点之间的电容信号变化；特征提取单元：其用于对于所述电容变化信号的有效部分，根据对触摸感应的辨识程度提取梅尔倒谱系数；分类识别单元，其用于以所述梅尔倒谱系数作为经训练的隐马尔可夫模型的输入特征，以识别出触摸电容触摸屏的手指类型；人机交互单元，其用于根据设定的手指类型与触屏功能的关联关系，控制所述电子设备的屏幕交互功能。该系统中的各单元可采用软件、专用逻辑器件或处理器实现。
综上所述，对于电容式触控屏，由于人体成为线路的一部分，因而漂移现象比较严重；并且对于屏幕太小的电子设备，人机交互存在困难。本发明提供的技术方案，利用不同手指触摸电容触摸屏产生的电容变化信号来进行屏幕交互，很好地解决了智能手表等屏幕太小而交互困难的问题。此外，本发明以提取的电容变化信号的梅尔倒谱系数作为输入特征，通过训练隐马尔可夫链模型，能够准确地识别电容触摸屏不同手指的输入，进 而扩展了电子设备的屏幕交互功能。
本发明可以是系统、方法和/或计算机程序产品。计算机程序产品可以包括计算机可读存储介质，其上载有用于使处理器实现本发明的各个方面的计算机可读程序指令。
计算机可读存储介质可以是可以保持和存储由指令执行设备使用的指令的有形设备。计算机可读存储介质例如可以是――但不限于――电存储设备、磁存储设备、光存储设备、电磁存储设备、半导体存储设备或者上述的任意合适的组合。计算机可读存储介质的更具体的例子(非穷举的列表)包括：便携式计算机盘、硬盘、随机存取存储器(RAM)、只读存储器(ROM)、可擦式可编程只读存储器(EPROM或闪存)、静态随机存取存储器(SRAM)、便携式压缩盘只读存储器(CD-ROM)、数字多功能盘(DVD)、记忆棒、软盘、机械编码设备、例如其上存储有指令的打孔卡或凹槽内凸起结构、以及上述的任意合适的组合。这里所使用的计算机可读存储介质不被解释为瞬时信号本身，诸如无线电波或者其他自由传播的电磁波、通过波导或其他传输媒介传播的电磁波(例如，通过光纤电缆的光脉冲)、或者通过电线传输的电信号。
这里所描述的计算机可读程序指令可以从计算机可读存储介质下载到各个计算/处理设备，或者通过网络、例如因特网、局域网、广域网和/或无线网下载到外部计算机或外部存储设备。网络可以包括铜传输电缆、光纤传输、无线传输、路由器、防火墙、交换机、网关计算机和/或边缘服务器。每个计算/处理设备中的网络适配卡或者网络接口从网络接收计算机可读程序指令，并转发该计算机可读程序指令，以供存储在各个计算/处理设备中的计算机可读存储介质中。
用于执行本发明操作的计算机程序指令可以是汇编指令、指令集架构(ISA)指令、机器指令、机器相关指令、微代码、固件指令、状态设置数据、或者以一种或多种编程语言的任意组合编写的源代码或目标代码，所述编程语言包括面向对象的编程语言—诸如Smalltalk、C++等，以及常规的过程式编程语言—诸如“C”语言或类似的编程语言。计算机可读程序指令可以完全地在用户计算机上执行、部分地在用户计算机上执行、作为一个 独立的软件包执行、部分在用户计算机上部分在远程计算机上执行、或者完全在远程计算机或服务器上执行。在涉及远程计算机的情形中，远程计算机可以通过任意种类的网络—包括局域网(LAN)或广域网(WAN)—连接到用户计算机，或者，可以连接到外部计算机(例如利用因特网服务提供商来通过因特网连接)。在一些实施例中，通过利用计算机可读程序指令的状态信息来个性化定制电子电路，例如可编程逻辑电路、现场可编程门阵列(FPGA)或可编程逻辑阵列(PLA)，该电子电路可以执行计算机可读程序指令，从而实现本发明的各个方面。
这里参照根据本发明实施例的方法、装置(系统)和计算机程序产品的流程图和/或框图描述了本发明的各个方面。应当理解，流程图和/或框图的每个方框以及流程图和/或框图中各方框的组合，都可以由计算机可读程序指令实现。
这些计算机可读程序指令可以提供给通用计算机、专用计算机或其它可编程数据处理装置的处理器，从而生产出一种机器，使得这些指令在通过计算机或其它可编程数据处理装置的处理器执行时，产生了实现流程图和/或框图中的一个或多个方框中规定的功能/动作的装置。也可以把这些计算机可读程序指令存储在计算机可读存储介质中，这些指令使得计算机、可编程数据处理装置和/或其他设备以特定方式工作，从而，存储有指令的计算机可读介质则包括一个制造品，其包括实现流程图和/或框图中的一个或多个方框中规定的功能/动作的各个方面的指令。
也可以把计算机可读程序指令加载到计算机、其它可编程数据处理装置、或其它设备上，使得在计算机、其它可编程数据处理装置或其它设备上执行一系列操作步骤，以产生计算机实现的过程，从而使得在计算机、其它可编程数据处理装置、或其它设备上执行的指令实现流程图和/或框图中的一个或多个方框中规定的功能/动作。
附图中的流程图和框图显示了根据本发明的多个实施例的系统、方法和计算机程序产品的可能实现的体系架构、功能和操作。在这点上，流程图或框图中的每个方框可以代表一个模块、程序段或指令的一部分，所述模块、程序段或指令的一部分包含一个或多个用于实现规定的逻辑功能的 可执行指令。在有些作为替换的实现中，方框中所标注的功能也可以以不同于附图中所标注的顺序发生。例如，两个连续的方框实际上可以基本并行地执行，它们有时也可以按相反的顺序执行，这依所涉及的功能而定。也要注意的是，框图和/或流程图中的每个方框、以及框图和/或流程图中的方框的组合，可以用执行规定的功能或动作的专用的基于硬件的系统来实现，或者可以用专用硬件与计算机指令的组合来实现。对于本领域技术人员来说公知的是，通过硬件方式实现、通过软件方式实现以及通过软件和硬件结合的方式实现都是等价的。
以上已经描述了本发明的各实施例，上述说明是示例性的，并非穷尽性的，并且也不限于所披露的各实施例。在不偏离所说明的各实施例的范围和精神的情况下，对于本技术领域的普通技术人员来说许多修改和变更都是显而易见的。本文中所用术语的选择，旨在最好地解释各实施例的原理、实际应用或对市场中的技术改进，或者使本技术领域的其它普通技术人员能理解本文披露的各实施例。本发明的范围由所附权利要求来限定。

Claims (10)

  1. 一种基于电容触摸屏的人机交互方法，包括以下步骤：
    使用具有电容触摸屏的电子设备采集手指触摸电容触摸屏所产生的电容变化信号；
    提取所述电容变化信号的有效部分，该有效部分表征手指触摸电容屏的起点和终点之间的电容信号变化；
    对于所述电容变化信号的有效部分，根据对触摸感应的辨识程度提取梅尔倒谱系数；
    以所述梅尔倒谱系数作为经训练的隐马尔可夫模型的输入特征，以识别出触摸电容触摸屏的手指类型；
    根据设定的手指类型与触屏功能之间的关联关系，控制所述电子设备的屏幕交互功能。
  2. 根据权利要求1所述的基于电容触摸屏的人机交互方法，其中，所述电子设备包括智能手表、智能手环、智能手机、机器人，通过调用所述电子设备的操作系统的底层API来获取屏幕触摸点的电容变化信号。
  3. 根据权利要求1所述的基于电容触摸屏的人机交互方法，其中，提取所述电容变化信号的有效部分包括：
    对采集的电容变化信号使用巴特沃斯带通滤波器进行滤波，截止频率分别设置为10Hz和1000Hz；
    对于经滤波的电容变化信号，计算其短时能量；
    基于短时能量的双门限端点检测法来检测出信号有效部分的起点和终点；
    根据获取的起点和终点对截取出电容变化信号的有效部分。
  4. 根据权利要求3所述的方法，其中，根据以下步骤截取出电容变化信号的有效部分：
    基于经滤波的电容变化信号的短时能量标准差σ设置第一门限和第二门限，其中，第一门限是TL＝u+σ，第二门限是TH＝u+3σ，u是背景噪声的平均能量；
    找出信号中短时能量最大的一帧信号且该帧信号的能量高于所述第 二门限；
    从该帧信号的前序帧和后序帧，分别找出能量低于所述第一门限并且在时序上与该帧信号最近的帧，将获得的前序帧位置作为起点，将获得的后续帧位置作为终点，截取起点和终点之间的部分作为信号的有效部分。
  5. 根据权利要求4所述的方法，其中，截取出电容变化信号的有效部分还包括：
    对于电容变化信号，设置信号峰之间的最大间隔门限maxInter和最小长度门限minLen；
    若电容变化信号的两个信号峰之间的间隔小于所述最大间隔门限maxInter，则将该两个信号峰作为该电容变化信号的一个信号峰；
    若电容变化信号的一个信号峰的长度小于所述最小长度门限minLen，则舍弃该信号峰。
  6. 根据权利要求1所述的基于电容触摸屏的人机交互方法，其中，对于所述电容变化信号的有效部分，根据对触摸感应的辨识程度提取梅尔倒谱系数包括：
    对获取的信号有效部分进行预加重、分帧和加窗；
    对每一个短时分析窗，通过短时傅里叶变换得到对应的频谱；
    将获得的频谱通过梅尔滤波器组得到梅尔频谱；
    对得到的梅尔频谱取对数，并进行离散余弦变换，进而取前14个系数作为提取出的梅尔倒谱系数。
  7. 根据权利要求1所述的基于电容触摸屏的人机交互方法，其中，以所述梅尔倒谱系数作为观察序列，使用鲍姆-韦尔奇算法训练隐马尔可夫模型，并且对应于每种手指类型，隐马尔可夫模型的状态数为3，每个状态有2个混合高斯概率密度函数，训练过程包括：对隐马尔可夫模型的参数进行初始化；计算前、后向概率矩阵；计算转移概率矩阵；计算各个高斯概率密度函数的均值和方差；计算各个高斯概率密度函数的权重；计算所有观察序列的输出概率，并进行累加得到总和输出概率。
  8. 根据权利要求7所述的基于电容触摸屏的人机交互方法，其中，还包括根据以下步骤评估经训练的隐马尔可夫模型：
    利用维特比算法计算测试数据对于各个隐马尔可夫模型的输出概率，并给出最佳的状态路径；
    将输出概率最大的隐马尔可夫模型所对应的手指类型作为该测试数据的分类结果。
  9. 一种基于电容触摸屏的人机交互系统，包括：
    信号获取单元：用于使用具有电容触摸屏的电子设备采集手指触摸电容触摸屏产生的电容变化信号；
    信号处理单元：用于提取所述电容变化信号的有效部分，该有效部分表征手指触摸电容屏的起点和终点之间的电容信号变化；
    特征提取单元：用于对于所述电容变化信号的有效部分，根据对触摸感应的辨识程度提取梅尔倒谱系数；
    分类识别单元：用于以所述梅尔倒谱系数作为经训练的隐马尔可夫模型的输入特征，以识别出触摸电容触摸屏的手指类型；
    人机交互单元：用于根据设定的手指类型与触屏功能之间的关联关系，控制所述电子设备的屏幕交互功能。
  10. 一种计算机可读存储介质，其上存储有计算机程序，其中，该程序被处理器执行时实现根据权利要求1至8中任一项所述方法的步骤。
PCT/CN2020/116764 2020-09-22 2020-09-22 一种基于电容触摸屏的人机交互系统和交互方法 WO2022061500A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
PCT/CN2020/116764 WO2022061500A1 (zh) 2020-09-22 2020-09-22 一种基于电容触摸屏的人机交互系统和交互方法

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2020/116764 WO2022061500A1 (zh) 2020-09-22 2020-09-22 一种基于电容触摸屏的人机交互系统和交互方法

Publications (1)

Publication Number Publication Date
WO2022061500A1 true WO2022061500A1 (zh) 2022-03-31

Family

ID=80846027

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/116764 WO2022061500A1 (zh) 2020-09-22 2020-09-22 一种基于电容触摸屏的人机交互系统和交互方法

Country Status (1)

Country Link
WO (1) WO2022061500A1 (zh)

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103282869A (zh) * 2010-08-12 2013-09-04 谷歌公司 触摸屏上的手指识别
US20190102003A1 (en) * 2017-10-03 2019-04-04 Microsoft Technology Licensing, Llc Touch sensor locating mode
CN109739385A (zh) * 2019-01-08 2019-05-10 合肥京东方光电科技有限公司 基于电容信号进行触控手指识别的方法和装置及触摸屏
CN110058689A (zh) * 2019-04-08 2019-07-26 深圳大学 一种基于脸部振动的智能设备输入方法
JP2019168885A (ja) * 2018-03-23 2019-10-03 カシオ計算機株式会社 接触検知装置、接触検知方法及びプログラム
KR20200077827A (ko) * 2018-12-21 2020-07-01 전남대학교산학협력단 정전용량 방식 섬유형 터치패드 및 그 제조방법

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103282869A (zh) * 2010-08-12 2013-09-04 谷歌公司 触摸屏上的手指识别
US20190102003A1 (en) * 2017-10-03 2019-04-04 Microsoft Technology Licensing, Llc Touch sensor locating mode
JP2019168885A (ja) * 2018-03-23 2019-10-03 カシオ計算機株式会社 接触検知装置、接触検知方法及びプログラム
KR20200077827A (ko) * 2018-12-21 2020-07-01 전남대학교산학협력단 정전용량 방식 섬유형 터치패드 및 그 제조방법
CN109739385A (zh) * 2019-01-08 2019-05-10 合肥京东方光电科技有限公司 基于电容信号进行触控手指识别的方法和装置及触摸屏
CN110058689A (zh) * 2019-04-08 2019-07-26 深圳大学 一种基于脸部振动的智能设备输入方法

Similar Documents

Publication Publication Date Title
US8619063B2 (en) Method for determining a touch event and touch sensitive device
US10599251B2 (en) Method and apparatus for differentiating touch screen users based on touch event analysis
KR102460737B1 (ko) 공중 필기 인식을 위한 방법, 장치, 기기 및 컴퓨터 판독 가능한 저장 매체
WO2017140221A1 (zh) 文本信息处理方法和装置
JP2013519132A (ja) 表面音響信号を用いたユーザインタフェース提供装置及び方法、ユーザインタフェースを備えるデバイス
CN112382285B (zh) 语音控制方法、装置、电子设备和存储介质
CN111475030B (zh) 一种利用近红外传感器的微手势识别方法
Yin et al. Ubiquitous writer: Robust text input for small mobile devices via acoustic sensing
JP2022160590A (ja) 事前訓練済みモデルの決定方法及びその決定装置、電子機器並びに記憶媒体
CN113827240B (zh) 情绪分类方法和情绪分类模型的训练方法、装置及设备
WO2022061500A1 (zh) 一种基于电容触摸屏的人机交互系统和交互方法
Li et al. Recognizing text through sound alone
US20190188831A1 (en) System and method for drawing beautification
Chen et al. WritePad: Consecutive number writing on your hand with smart acoustic sensing
CN112949433A (zh) 视频分类模型的生成方法、装置、设备和存储介质
Kawahata et al. Design of a low-false-positive gesture for a wearable device
WO2022056915A1 (zh) 一种基于电容按键的人机交互方法和交互系统
WO2022001791A1 (zh) 一种基于ppg信息的智能设备交互方法
CN112130710A (zh) 一种基于电容触摸屏的人机交互系统和交互方法
CN113590774B (zh) 事件查询方法、装置以及存储介质
JP5398811B2 (ja) 文書分類装置及び方法及びプログラム
WO2022061499A1 (zh) 一种基于振动信号的身份验证方法和系统
CN112905743B (zh) 文本对象检测的方法、装置、电子设备和存储介质
CN112131541A (zh) 一种基于振动信号的身份验证方法和系统
CN114495292A (zh) 身份识别方法、装置、设备及可读存储介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20954354

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 28.06.2023)

122 Ep: pct application non-entry in european phase

Ref document number: 20954354

Country of ref document: EP

Kind code of ref document: A1