CN112099624A

CN112099624A - Multimode diamond-shaped frame type capacitive sensing gesture recognition system

Info

Publication number: CN112099624A
Application number: CN202010860659.XA
Authority: CN
Inventors: 李志斌
Original assignee: Individual
Current assignee: Individual
Priority date: 2020-08-25
Filing date: 2020-08-25
Publication date: 2020-12-18
Anticipated expiration: 2040-08-25
Also published as: CN112099624B

Abstract

A multi-mode diamond-frame type capacitance sensing gesture recognition system adopts gesture space capacitance analysis under a deep learning frame. The capacitive acquisition module FDC2214 is an anti-noise and EMI, high-resolution, high-speed and multi-channel capacitive digital converter facing the capacitive sensing solution, and the capacitive acquisition module adopts an innovative EMI resistant architecture, so that the performance can be kept unchanged even in a high-noise environment. Unlike the conventional capacitance conversion IC which filters sinusoidal waves by external capacitance and reads a dc component to estimate a capacitance value, FDC2214 obtains a resonance frequency by driving an LC resonance tank. And in combination with the spatial capacitance data, a neural network fusion algorithm of the spatial capacitance value and the image data is designed, and the multi-element data is subjected to fusion processing, so that accurate classification of spatial gestures in multiple modes is realized. The space capacitance data acquisition of the diamond frame type space capacitance sensor is effectively applied to machine learning and deep learning, and the problem of classification of postures in a space can be solved.

Description

Multimode diamond-shaped frame type capacitive sensing gesture recognition system

Technical Field

The invention belongs to the technical field of calculation gesture recognition, and relates to a multi-mode diamond-shaped frame type capacitive sensing gesture recognition system

Background

In recent years, machine language and man-machine interaction are more and more popular. Gesture recognition is a way of human-computer interaction, and is gradually applied and popular in people's life, and the development of computers provides a powerful computing platform for gesture recognition, so that interaction activities between people and computers are more and more frequent. The method aims to analyze the specific meaning of each gesture by using a computer through a certain means so as to obtain the whole expression of a gesture initiator, thereby achieving the purpose of realizing human-computer interaction. Capacitive sensing is a non-contact sensing technology with low power consumption, low cost and high resolution, and can be used for detecting the state change of objects such as rain, fog, ice, snow and the like.

The sensors in the capacitive sensing system may be made of any metal or conductor, thus allowing for a highly flexible low cost system design. FDC2214 is a noise and EMI resistant, high resolution, high speed, multi-channel capacitive-to-digital converter for capacitive sensing solutions introduced by Texas Instruments (TI) that employs an innovative EMI resistant architecture to maintain performance even in high noise environments. The traditional capacitance converter filters sine waves through an external capacitance pole plate, a direct current component is read to estimate the capacitance value between the pole plates, and FDC2214 obtains the resonance frequency by driving an LC resonance tank so as to obtain the corresponding capacitance value.

At present, the recognition of gestures is mainly performed by contact recognition, and a series of inaccuracies exist in non-contact recognition. The hand position will produce different measurements when not in contact with the identification. In the past, gesture recognition is mostly performed on the basis of visual image information, and is mostly a static gesture recognition technology, and basically, color images obtained by using a traditional camera are researched. Because the input data and information have differences, the identification mode also has differences. The patent considers the physical design of the suspended hands of people, matches with a special diamond frame type capacitor data acquisition, fitting comparison processing, filtering algorithm processing, multi-element data fusion and the like, and mainly solves the problem of accurately judging the gesture after machine learning. Aiming at solving the problems of variability and difference of rotation, size and translation of static gestures

Disclosure of Invention

The structure of the invention is beneficial to collecting the slight capacitance value change in the three-dimensional space of the frame and realizing the real-time gesture recognition. The MCU is used for cleaning the acquired data, further performing filtering algorithm processing such as Kalman filtering algorithm and the like on the processed numerical value to obtain stable capacitance value data, uploading the capacitance value to an upper computer, integrating gesture visual image data with the capacitance data to realize multi-modal gesture recognition, designing a neural network integration algorithm of a space capacitance value and the image data, and realizing training and testing of a gesture recognition model.

The invention adopts the following technical scheme for solving the technical problems:

A multi-mode diamond-frame type capacitive sensing gesture recognition system comprises two parts:

a first part: the invention relates to an FDC2214 chip externally connected with a diamond frame type space capacitance sensor based on TI (Texas instruments), which uses a copper foil outer frame structure as a closed capacitance polar plate, collects capacitance sampling values in the frame and transmits the capacitance sampling values to an external machine control panel MCU (Arduino mega2560) through an I2C interface.

The four channels CH0, CH1, CH2, and CH3 of the FDC2214 capacitance-to-digital converter are respectively connected to the copper foil plates of the diamond frame to form a diamond frame sensor with 4 acquisition signals, as shown in fig. 2. Each surface has a signal acquisition, and the two signals are used as a pair of opposite capacitor plates.

The embedded processor Arduino mega2560 is provided with two paths of I2C interfaces, completes serial port communication on sensor data of 4 paths of capacitance signals, and performs LED real-time visualization on the data. The collected signals are connected to an Arduino Mega2560 board through an I2C interface for filtering processing. The Mega2560 transmits the preprocessed data to an upper computer through UART communication, and fusion processing with image data is achieved.

A second part: capacitance-sensing gesture recognition

The data collection and transmission flow chart is shown in fig. 3. In the classification training process, 10 different types of gestures are defined by users, which are respectively defined as "stone", "scissors", "cloth", "1", "2", "3", "4", "5", "6" and "7", and the types of gestures are shown in fig. 4. In the data acquisition mode of the diamond-frame type space capacitive sensor, as shown in fig. 6, the gesture is placed in the diamond-frame type space.

Spatial capacitance and image data fusion algorithm 1 for biological heuristic learning: kalman filtering:

the existing algorithm is used, Kalman filtering is selected on the MCU to remove noise and restore real data, the Kalman filtering can estimate the state of a dynamic system from a series of data with measurement noise under the condition that the measurement variance is known, and the random quantity is quantitatively deduced according to observation data. And performing Kalman filtering treatment on a copper plate at the bottom of the system, and performing filtering treatment on a 4-channel capacitance value in the space structure.

A fusion algorithm of space capacitance and image data of biological heuristic learning 2: neural network fusion algorithm of space capacitance and image data (upper computer training process)

A gesture capacitance data set is established based on a diamond frame capacitive sensor, which contains 3000 gesture samples and corresponding spatial capacitance value samples. Each sample consists of a gesture image shot in a complex background and a set of 4-channel capacitance data, wherein the data come from images corresponding to capacitance values and vision of a hand in a certain gesture in a diamond-shaped frame space, and the corresponding relation is shown in figure 5.

After the AlexNet convolutional neural network is adopted for image training, 4-dimensional capacitance data are fused to establish a three-layer sparse neural network, and the network structure is shown in FIG. 6. And training gesture image data by using AlexNet, extracting image characteristics, and connecting the space capacitance value of the gesture. As can be seen from fig. 5, the 4-dimensional vectors represent diamond-frame-type spatial capacitance values of 4 channels, the 48-dimensional vectors represent feature data of the convolutional neural network extracted image, and the two are connected and fused into 52-dimensional vectors as three-layer sparse neural network input for final recognition model training. In the 10 classification tasks, the module design shows a high recognition rate, the recognition rate of the test set is over 95%, and the model training test result is shown in fig. 6. The FDC 2214-based diamond-frame type capacitive sensing gesture recognition system has the characteristics of simple equipment, high sensitivity, convenience in use and the like, and is convenient for collecting spatial data characteristics. The gesture judgment is realized by collecting, analyzing and learning the spatial capacitance values of the 10 gestures. Through the algorithm 2 (neural network fusion algorithm of space capacitance and image data), a deep learning module can be effectively built, and efficient gesture recognition is achieved.

The diamond frame type space capacitance sensor adopts gesture space capacitance analysis under a deep learning frame. The capacitive acquisition module FDC2214 is an anti-noise and EMI, high-resolution, high-speed and multi-channel capacitive digital converter facing the capacitive sensing solution, and the capacitive acquisition module adopts an innovative EMI resistant architecture, so that the performance can be kept unchanged even in a high-noise environment. Unlike the conventional capacitance conversion IC that simply filters the sine wave through an external capacitor and reads a dc component to estimate the capacitance, FDC2214 obtains the resonant frequency by driving the LC resonant tank, thereby obtaining the corresponding capacitance change. And in combination with the spatial capacitance data, a neural network fusion algorithm of the spatial capacitance value and the image data is designed, and the multi-element data is subjected to fusion processing, so that accurate classification of spatial gestures in multiple modes is realized. The space capacitance data acquisition of the diamond frame type space capacitance sensor is effectively applied to machine learning and deep learning, and the problem of classification of postures in a space can be solved. The method aims to solve the problem that static space gestures have the variability of rotation, size and translation.

Drawings

FIG. 1 is a schematic diagram of a diamond frame type capacitive acquisition sensor plate wiring.

Fig. 2 is a simplified circuit diagram of FDC 2214-based capacitance acquisition.

Fig. 3 is a flow chart of data collection and transmission.

FIG. 4 is a schematic diagram of gesture image data acquisition.

FIG. 5 is a graph of spatial capacitance data versus visual image correspondence.

FIG. 6 is a diagram of a neural network structure of spatial capacitance and image data.

FIG. 7 is a graph of model training test results.

Detailed Description

The technical solution of the present invention will be described in detail with reference to the accompanying drawings.

The outer frame structure of the diamond frame type space capacitance sensor is used as a closed capacitance polar plate to acquire accurate capacitance values in the frame. The processor of the invention selects Arduino Mega2560, uses an FDC2214 capacitance digital converter to respectively connect out four channels CH0, CH1, CH2 and CH3 of a plate, and connects the four channels to a copper foil plate to form two pairs of closed capacitance plates, so as to form a diamond frame type space capacitance sensor with 4 paths of collected signals, as shown in figure 1. The simplified circuit diagram of the capacitor acquisition is shown in fig. 2, in which an FDC2214 capacitor-to-digital converter is used to acquire four-channel space capacitor values (C0, C1, C2, C3), and the values are transmitted to an outdoor unit control board through an I2C interface, and an Arduino Mega2560 is used as a lower computer microprocessor. The Arduino Mega2560 firstly carries out rolling filtering processing on the collected capacitance value with large fluctuation amplitude, and carries out Kalman filtering processing on the four-channel numerical value to obtain a space capacitance value with small variation amplitude difference, and the space capacitance value is uploaded to an upper computer as classification data to realize later-stage multivariate data fusion processing. The flow chart of multivariate data acquisition and transmission is shown in fig. 3.

The Arduino Mega2560 is used as a microprocessor, and the data collected are different according to different gestures placed in the diamond frame type space. As shown in fig. 4, 10 different types of gestures are defined as "stone", "scissors", "cloth", "1", "2", "3", "4", "5", "6" and "7", respectively. The diamond frame type space capacitance sensor collects space capacitance data of corresponding gestures. The data acquisition mode of the diamond-frame type space capacitance sensor is shown in fig. 5, and the gesture is placed in the diamond-frame type space.

Arduino mega2560 serves as a processor. The channels CH0, CH1, CH2 and CH3 are connected to a copper foil plate through a shielding wire not longer than 10CM to form two pairs of capacitance sampling plates, and for the space complexity of the complex gesture, the channels CH0 and CH1 are designed into top diamond-shaped frame type polar plates, so that the Microsoft signal acquisition of the space complex gesture is realized. Experimental tests show that when the lead wires connecting the CH0, CH1, CH2 and CH3 with the copper foil plate are general lead wires, the interference is obviously larger than that of a shielding wire, and the interference is larger when the lead wires are longer, so that the shielding wire is selected to be not more than 10 CM.

The larger the distance between the two polar plates is, the smaller the capacitance is, the larger the opposite area is, and the larger the capacitance value is.

The digitized sensor measurement (DATAX) for each channel represents the ratio of the sensor frequency to the reference frequency.

The data output (DATAX) of FDC2214 is represented as:

the sensor capacitance CSENSE of the differential sensor configuration may be determined by:

parallel sensor capacitance

FDC2112 and FDC2114 sensor frequency fSCORSOX are determined by:

and (5) data acquisition and transmission process.

The first step is as follows: acquiring capacitance values of four-channel polar plates of a diamond frame type space capacitance sensor by using Arduino Mega2560, and performing Kalman filtering treatment;

the second step is that: respectively displaying the processed analog quantities of different gestures through an LCD display screen, uploading the analog quantities to an upper computer through a UART, and recording and storing the analog quantities;

the third step: image data of the stored gesture is collected using PS3, the gesture image data corresponding to the spatial capacitance data of the corresponding gesture. As shown in fig. 5, the spatial capacitance data corresponds to the visual image;

the fourth step: using the above algorithm 2 (neural network fusion algorithm of spatial capacitance and image data), the structure diagram of the neural network is shown in fig. 6. Training the neural network classification model, and storing the model. The training results are shown in fig. 7.

The fifth step: and training the three-person capacitance data and the image data, and then checking the generalization capability of the model and the accuracy of gesture recognition, wherein the accuracy of the test set is more than 95%.

And (3) analyzing a test result:

table (1): test set accuracy

Gesture	Test person	1	Tester 2	Tester 3
					1	98.1％	100％	98.0％
2	93.4％	91.6％	90.0％
				3	96.2％	97.1％	97.6％
4	98.8％	94.2％	96.5％
				5	90.1％	86.3％	88.4％
6	99.1％	98.2％	97.1％
				7	95.7％	97.6％	97.8％
Stone (W.E.)	99.7％	100％	100％
				Scissors	93.5％	87.2％	89.4％
Cloth	91.1％	87.2％	81.4％

As can be seen from the above table, in limited tests, the accuracy of the test results is greater than 90%, and most of the accuracy reaches 95%, which proves that the scheme is feasible, and the visible gesture 5 and cloth recognition rate needs to be improved.

The use of Arduino Mega2560 with an FDC2214 capacitive-to-digital converter. And (3) filtering the 4-channel capacitance value data by the Arduino mega2560, and establishing the basis of upper computer data fusion and multi-mode identification. As shown in fig. 3, a data acquisition, data transmission and data fusion network is constructed.

And building a neural network model of the space capacitance value and the image data. As shown in fig. 6, the structure diagram of the neural network is that after the AlexNet convolutional neural network is used for image feature extraction, 4-dimensional capacitance data are fused to establish a three-layer sparse neural network.

Claims

1. A multi-mode diamond frame type capacitive sensing gesture recognition system is characterized in that,

a first part: establishing TI-based FDC2214 chip external diamond frame type space capacitance sensor

Four channels CH0, CH1, CH2 and CH3 of the FDC2214 capacitance digital converter are respectively connected out and connected to a copper foil plate of the diamond-shaped frame to form a diamond-shaped frame sensor with 4 paths of collected signals; each surface is provided with a signal acquisition, and the two signals are used as a pair of opposite capacitor plates;

the embedded processor Arduino mega2560 is provided with two paths of I2C interfaces, completes serial port communication on sensor data of 4 paths of capacitance signals and performs LED real-time visualization on the data; the collected signals are connected to an Arduino Mega2560 board through an I2C interface for filtering processing; the Mega2560 transmits the preprocessed data to an upper computer through UART communication to realize fusion processing with image data;

in the classification training process, 10 types of different gestures are defined in a self-defining mode, namely, a stone mode, a scissors mode, a cloth mode, a 1 mode, a2 mode, a 3 mode, a 4 mode, a 5 mode, a 6 mode and a 7 mode, and the gestures are placed in a diamond frame type space;

a second part: capacitance-sensing gesture recognition

the third step: collecting and storing image data of the gesture by using PS3, wherein the image data of the gesture corresponds to the space capacitance data of the corresponding gesture;

the fourth step: training the neural network classification model by using a neural network fusion algorithm of a space capacitance value and image data, and storing the model; the neural network fusion algorithm of the space capacitance value and the image data comprises the following steps: establishing a gesture capacitance data set based on the diamond-shaped frame capacitive sensor, wherein the gesture capacitance data set comprises 3000 gesture samples and corresponding space capacitance value samples; each sample consists of a gesture image shot in a complex background and a group of 4-channel capacitance data, wherein the data are from images corresponding to capacitance values and vision of a hand in a certain gesture in a diamond-shaped frame space;

the fifth step: and training the three-person capacitance data and the image data, and then checking the generalization capability of the model and the accuracy of gesture recognition.

2. The system of claim 1, wherein the wires connecting the CH0, CH1, CH2, CH3 and the copper foil are not larger than 10CM of shielding.