CN112099624B

CN112099624B - Multimode diamond-shaped frame type capacitive sensing gesture recognition system

Info

Publication number: CN112099624B
Application number: CN202010860659.XA
Authority: CN
Inventors: 李志斌
Original assignee: Individual
Current assignee: Individual
Priority date: 2020-08-25
Filing date: 2020-08-25
Publication date: 2022-11-22
Anticipated expiration: 2040-08-25
Also published as: CN112099624A

Abstract

A multi-mode diamond-frame type capacitance sensing gesture recognition system adopts gesture space capacitance analysis under a deep learning frame. The capacitive acquisition module FDC2214 is an anti-noise and EMI, high-resolution, high-speed and multi-channel capacitive digital converter facing the capacitive sensing solution, and the capacitive acquisition module adopts an innovative EMI resistant architecture, so that the performance can be kept unchanged even in a high-noise environment. Unlike the conventional capacitance conversion IC which filters sinusoidal waves by external capacitance and reads a dc component to estimate a capacitance value, FDC2214 obtains a resonance frequency by driving an LC resonance tank. And in combination with the spatial capacitance data, a neural network fusion algorithm of the spatial capacitance value and the image data is designed, and the multi-element data is subjected to fusion processing, so that accurate classification of spatial gestures in multiple modes is realized. The space capacitance data acquisition of the diamond frame type space capacitance sensor is effectively applied to machine learning and deep learning, and the problem of classification of postures in a space can be solved.

Description

Multimode diamond-shaped frame type capacitive sensing gesture recognition system

Technical Field

The invention belongs to the technical field of calculation gesture recognition, and relates to a multi-mode diamond-shaped frame type capacitive sensing gesture recognition system

Background

In recent years, machine language and man-machine interaction are more and more popular. Gesture recognition is a way of human-computer interaction, and is gradually applied and popular in people's life, and the development of computers provides a powerful computing platform for gesture recognition, so that interaction activities between people and computers are more and more frequent. The method aims to analyze the specific meaning of each gesture by a certain means through a computer so as to obtain the whole expression of a gesture initiator, thereby achieving the aim of realizing human-computer interaction. Capacitive sensing is a non-contact sensing technology with low power consumption, low cost and high resolution, and can be used for detecting the state change of objects such as rain, fog, ice, snow and the like.

The sensors in the capacitive sensing system may be made of any metal or conductor, thus allowing for a highly flexible low cost system design. FDC2214 is an anti-noise and EMI, high resolution, high speed, multi-channel capacitive-to-digital converter for capacitive sensing oriented solutions introduced by Texas Instruments (TI) that employs an innovative anti-EMI architecture to maintain performance even in high noise environments. The traditional capacitance converter filters sine waves through an external capacitance pole plate, reads a direct current component to estimate capacitance between the pole plates, and FDC2214 obtains resonance frequency by driving an LC resonance tank so as to obtain a corresponding capacitance value.

At present, the recognition of gestures is mainly performed by contact recognition, and a series of inaccuracies exist in non-contact recognition. The hand position will produce different measurements when not in contact with the identification. In the past, gesture recognition is mostly performed on the basis of visual image information, and is mostly a static gesture recognition technology, and basically, color images obtained by using a traditional camera are researched. Because the input data and information have differences, the identification mode also has differences. The patent considers the physical design of the suspended hands of people, matches with a special diamond frame type capacitor data acquisition, fitting comparison processing, filtering algorithm processing, multi-element data fusion and the like, and mainly solves the problem of accurately judging the gesture after machine learning. Aiming at solving the problem that static gestures have variability and difference of rotation, size and translation

Disclosure of Invention

The structure of the invention is beneficial to acquiring slight capacitance value change in the three-dimensional space of the frame and realizing real-time gesture recognition. The MCU cleans the acquired data, further performs filtering algorithm processing such as Kalman filtering algorithm and the like on the processed numerical value to obtain capacitance value data tending to stability, so that the capacitance value is uploaded to an upper computer, gesture visual image data and capacitance data are fused to realize multi-mode gesture recognition, a neural network fusion algorithm of the spatial capacitance value and the image data is designed, and a gesture recognition model training test is realized.

The invention adopts the following technical scheme for solving the technical problems:

A multi-mode diamond-frame type capacitance induction gesture recognition system comprises two parts:

a first part: the invention relates to an FDC2214 chip externally connected with a diamond frame type space capacitance sensor based on TI (Texas instruments), which uses a copper foil outer frame structure as a closed capacitance polar plate, collects capacitance sampling values in the frame and transmits the capacitance sampling values to an external machine control panel MCU (Arduino mega 2560) through an I2C interface.

The four channels CH0, CH1, CH2, CH3 of the FDC2214 capacitance-to-digital converter are connected out respectively and connected to the copper foil plate of the diamond frame to make a diamond frame sensor with 4 channels of collected signals, as shown in fig. 2. Each surface has a signal acquisition, and the two signals are used as a pair of opposite capacitor plates.

The embedded processor Arduino mega2560 is provided with two paths of I2C interfaces, completes serial port communication on sensor data of 4 paths of capacitance signals, and performs LED real-time visualization on the data. The collected signals are connected to an Arduino Mega2560 board through an I2C interface for filtering processing. The Mega2560 transmits the preprocessed data to an upper computer through UART communication, and fusion processing with image data is achieved.

A second part: capacitance-sensing gesture recognition

The data collection and transmission flow chart is shown in fig. 3. In the classification training process, 10 types of different gestures are customized, which are respectively defined as "stone", "scissors", "cloth", "1", "2", "3", "4", "5", "6" and "7", and the gesture types are shown in fig. 4. In the data acquisition mode of the diamond-frame type space capacitive sensor, as shown in fig. 6, the gesture is placed in the diamond-frame type space.

Spatial capacitance and image data fusion algorithm 1 for biological heuristic learning: kalman filtering:

the existing algorithm is used, kalman filtering is selected on the MCU to remove noise and restore real data, the Kalman filtering can estimate the state of a dynamic system from a series of data with measurement noise under the condition that the measurement variance is known, and the random quantity is quantitatively deduced according to observation data. And performing Kalman filtering treatment on a copper plate at the bottom of the system, and performing filtering treatment on a 4-channel capacitance value in the space structure.

A fusion algorithm of space capacitance and image data of biological heuristic learning 2: neural network fusion algorithm of space capacitance and image data (upper computer training process)

A gesture capacitance dataset was established based on a diamond frame capacitive sensor, containing 3000 gesture samples and corresponding spatial capacitance value samples. Each sample consists of a gesture image shot in a complex background and a set of 4-channel capacitance data, wherein the data come from images corresponding to capacitance values and vision of a hand in a certain gesture in a diamond-shaped frame space, and the corresponding relation is shown in figure 5.

After the AlexNet convolutional neural network is adopted for image training, 4-dimensional capacitance data are fused to establish a three-layer sparse neural network, and the network structure is shown in FIG. 6. And training gesture image data by using AlexNet, extracting image characteristics, and connecting the space capacitance value of the gesture. As can be seen from fig. 5, the 4-dimensional vector represents the diamond-frame-type spatial capacitance values of 4 channels, the 48-dimensional vector represents the feature data of the convolutional neural network extracted image, and the two are connected and fused into a 52-dimensional vector to be used as the three-layer sparse neural network input for the final recognition model training. In the 10 classification tasks, the module design shows a high recognition rate, the recognition rate of the test set is over 95%, and the model training test result is shown in fig. 6. The FDC 2214-based diamond-frame type capacitive sensing gesture recognition system has the characteristics of simple equipment, high sensitivity, convenience in use and the like, and is convenient for collecting spatial data characteristics. The gesture judgment is realized by collecting, analyzing and learning the spatial capacitance values of the 10 gestures. Through the algorithm 2 (neural network fusion algorithm of space capacitance and image data), a deep learning module can be effectively built, and efficient gesture recognition is achieved.

The diamond frame type space capacitance sensor adopts gesture space capacitance analysis under a deep learning frame. The capacitive acquisition module FDC2214 is an anti-noise and EMI, high-resolution, high-speed and multi-channel capacitive digital converter facing the capacitive sensing solution, and the capacitive acquisition module adopts an innovative EMI resistant architecture, so that the performance can be kept unchanged even in a high-noise environment. Unlike the conventional capacitance conversion IC that simply filters the sine wave through an external capacitor and reads a dc component to estimate the capacitance, FDC2214 obtains the resonant frequency by driving the LC resonant tank, thereby obtaining the corresponding capacitance change. And in combination with the spatial capacitance data, a neural network fusion algorithm of the spatial capacitance value and the image data is designed, and the multi-element data is subjected to fusion processing, so that accurate classification of spatial gestures under multiple modes is realized. The space capacitance data acquisition of the diamond frame type space capacitance sensor is effectively applied to machine learning and deep learning, and the problem of classification of postures in a space can be solved. The method aims to solve the problem that static space gestures have the variability of rotation, size and translation.

Drawings

FIG. 1 is a schematic diagram of a diamond frame type capacitive acquisition sensor plate wiring.

Fig. 2 is a simplified circuit diagram of FDC 2214-based capacitance acquisition.

Fig. 3 is a flow chart of data collection and transmission.

FIG. 4 is a schematic diagram of gesture image data acquisition.

FIG. 5 is a graph of spatial capacitance data versus visual image correspondence.

FIG. 6 is a diagram of a neural network structure of spatial capacitance and image data.

Fig. 7 is a graph of the results of model training tests.

Detailed Description

The technical solution of the present invention will be described in detail with reference to the accompanying drawings.

The outer frame structure of the diamond frame type space capacitance sensor is used as a closed capacitance polar plate to acquire accurate capacitance values in the frame. The processor selects Arduino Mega2560, uses an FDC2214 capacitance-digital converter to respectively connect out four channels CH0, CH1, CH2 and CH3 of a plate and connect the channels to a copper foil plate to form two pairs of closed capacitance plates, and a diamond-frame type space capacitance sensor with 4 paths of collected signals is manufactured, as shown in figure 1. An FDC2214 capacitance digital converter is used for collecting four-channel space capacitance values (C0, C1, C2 and C3), the values are transmitted to an outdoor unit control panel through an I2C interface, an Arduino Mega2560 is used as a lower computer microprocessor, and a simplified circuit diagram for capacitance collection is shown in figure 2. The Arduino Mega2560 firstly carries out rolling filtering processing on the collected capacitance value with large fluctuation amplitude, and carries out Kalman filtering processing on the four-channel numerical value to obtain a space capacitance value with small variation amplitude difference, and the space capacitance value is uploaded to an upper computer as classification data to realize later-stage multivariate data fusion processing. The flow chart of multivariate data acquisition and transmission is shown in fig. 3.

The Arduino Mega2560 is used as a microprocessor, and the data collected are different according to different gestures placed in the diamond frame type space. As shown in fig. 4, 10 different types of gestures are defined as "stone", "scissors", "cloth", "1", "2", "3", "4", "5", "6" and "7", respectively. The diamond frame type space capacitance sensor collects space capacitance data of corresponding gestures. The data acquisition mode of the diamond-frame type space capacitance sensor is shown in fig. 5, and the gesture is placed in the diamond-frame type space.

Arduino mega2560 serves as a processor. The channels CH0, CH1, CH2 and CH3 are connected to a copper foil plate through a shielding wire not longer than 10CM to form two pairs of capacitance sampling plates, and the channels CH0 and CH1 are designed into a top diamond-shaped frame type polar plate according to the space complexity of the complex gesture, so that Microsoft signal acquisition of the space complex gesture is realized. Experimental tests show that when the leads connecting the CH0, the CH1, the CH2 and the CH3 with the copper foil plates are general leads, the interference is obviously much larger than that of using shielding wires, and meanwhile, the interference is larger when the leads are longer, so that the shielding wires with the length not larger than 10CM are selected.

The larger the distance between the two polar plates is, the smaller the capacitance is, the larger the opposite area is, and the larger the capacitance value is.

The digitized sensor measurement (DATAX) for each channel represents the ratio of the sensor frequency to the reference frequency.

The data output (DATAX) of FDC2214 is represented as:

the sensor capacitance CSENSE of the differential sensor configuration may be determined by:

c = parallel sensor capacitance

FDC2112 and FDC2114 sensor frequency fSCORSOX are determined by:

and (5) data acquisition and transmission process.

The first step is as follows: acquiring capacitance values of four-channel polar plates of a diamond frame type space capacitance sensor by using Arduino Mega2560, and performing Kalman filtering treatment;

the second step is that: respectively displaying the processed analog quantities of different gestures through an LCD display screen, uploading the analog quantities to an upper computer through a UART, and recording and storing the analog quantities;

the third step: and (3) collecting and storing image data of the gesture by using the PS3, wherein the gesture image data corresponds to the space capacitance data of the corresponding gesture. As shown in fig. 5, the spatial capacitance data corresponds to the visual image;

the fourth step: using the above algorithm 2 (neural network fusion algorithm of spatial capacitance and image data), the structure diagram of the neural network is shown in fig. 6. Training the neural network classification model, and storing the model. The training results are shown in fig. 7.

The fifth step: and training the three-person capacitance data and the image data, and then checking the generalization capability of the model and the accuracy of gesture recognition, wherein the accuracy of the test set is more than 95%.

And (3) analyzing a test result:

table (1): test set accuracy

Gesture	Test person	1	Tester 2	Tester 3
					1	98.1％	100％	98.0％
2	93.4％	91.6％	90.0％
				3	96.2％	97.1％	97.6％
4	98.8％	94.2％	96.5％
				5	90.1％	86.3％	88.4％
6	99.1％	98.2％	97.1％
				7	95.7％	97.6％	97.8％
(Stone)	99.7％	100％	100％
				Scissors	93.5％	87.2％	89.4％
Cloth	91.1％	87.2％	81.4％

As can be seen from the above table, in limited tests, the accuracy of the test results is greater than 90%, and most of the accuracy reaches 95%, which proves that the scheme is feasible, and the recognition rate of the visible gesture 5 and cloth needs to be improved.

The use of Arduino Mega2560 with an FDC2214 capacitive-to-digital converter. And the Arduino mega2560 filters the 4-channel capacitance value data, and establishes the basis of upper computer data fusion and multi-mode identification. As shown in fig. 3, a data acquisition, data transmission and data fusion network is constructed.

And building a neural network model of the space capacitance value and the image data. As shown in fig. 6, the structure diagram of the neural network is that after the AlexNet convolutional neural network is used for image feature extraction, 4-dimensional capacitance data are fused to establish a three-layer sparse neural network.

Claims

1. A multi-mode diamond frame type capacitive sensing gesture recognition system is characterized in that,

a first part: establishing an FDC2214 chip externally connected with a diamond frame type space capacitance sensor based on TI;

four channels CH0, CH1, CH2 and CH3 of the FDC2214 capacitance digital converter are respectively connected out and connected to a copper foil plate of the rhombic frame to form a rhombic frame sensor with 4 paths of collected signals; each surface is provided with a signal acquisition, and the two signals are used as a pair of opposite capacitor plates;

the embedded processor Arduino mega2560 is provided with two paths of I2C interfaces, completes serial port communication on sensor data of 4 paths of capacitance signals and performs LED real-time visualization on the data; the collected signals are connected to an Arduino Mega2560 board through an I2C interface for filtering; the Mega2560 transmits the preprocessed data to an upper computer through UART communication to realize fusion processing with image data;

in the classification training process, 10 types of different gestures are defined in a self-defining mode, namely, a stone gesture, a scissors gesture, a cloth gesture, a 1 gesture, a2 gesture, a 3 gesture, a 4 gesture, a 5 gesture, a 6 gesture and a 7 gesture are respectively defined, and the data acquisition mode of the diamond-frame type space capacitance sensor is that the gestures are placed in a diamond-frame type space;

a second part: recognizing a capacitive sensing gesture;

the third step: acquiring and storing image data of the gesture by using PS3, wherein the gesture image data corresponds to the space capacitance data of the corresponding gesture;

the fourth step: training the neural network classification model by using a neural network fusion algorithm of a space capacitance value and image data, and storing the model; the neural network fusion algorithm of the spatial capacitance value and the image data is as follows: establishing a gesture capacitance data set based on the diamond-shaped frame capacitive sensor, wherein the gesture capacitance data set comprises 3000 gesture samples and corresponding space capacitance value samples; each sample consists of a gesture image shot in a complex background and a group of 4-channel capacitance data, wherein the data are from images corresponding to capacitance values and vision of a hand in a certain gesture in a diamond-shaped frame space;

the fifth step: the capacitance data and the image data are trained, and then the generalization capability of the model and the accuracy of the gesture recognition are checked.

2. The system of claim 1, wherein the wires connecting the CH0, CH1, CH2, CH3 and the copper foil plates are shielded wires of not more than 10 CM.