CN113673292A - Capacitive imaging sensor and gesture form sensing method - Google Patents
Capacitive imaging sensor and gesture form sensing method Download PDFInfo
- Publication number
- CN113673292A CN113673292A CN202110045386.8A CN202110045386A CN113673292A CN 113673292 A CN113673292 A CN 113673292A CN 202110045386 A CN202110045386 A CN 202110045386A CN 113673292 A CN113673292 A CN 113673292A
- Authority
- CN
- China
- Prior art keywords
- capacitance
- gesture
- capacitor
- imaging sensor
- space
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000003384 imaging method Methods 0.000 title claims abstract description 74
- 238000000034 method Methods 0.000 title claims abstract description 18
- 239000003990 capacitor Substances 0.000 claims abstract description 63
- 230000008447 perception Effects 0.000 claims abstract description 19
- 238000004422 calculation algorithm Methods 0.000 claims abstract description 13
- 230000008859 change Effects 0.000 claims description 33
- 230000033001 locomotion Effects 0.000 claims description 10
- 238000001514 detection method Methods 0.000 claims description 6
- 230000002452 interceptive effect Effects 0.000 claims description 5
- 230000009471 action Effects 0.000 claims description 4
- 238000013459 approach Methods 0.000 claims description 3
- 238000013135 deep learning Methods 0.000 claims description 3
- 238000010801 machine learning Methods 0.000 claims description 3
- 230000000877 morphologic effect Effects 0.000 claims description 3
- 230000003247 decreasing effect Effects 0.000 claims description 2
- 229910003460 diamond Inorganic materials 0.000 claims description 2
- 239000010432 diamond Substances 0.000 claims description 2
- 230000008569 process Effects 0.000 claims description 2
- 239000003989 dielectric material Substances 0.000 claims 1
- 238000003909 pattern recognition Methods 0.000 claims 1
- 238000003062 neural network model Methods 0.000 abstract description 3
- 238000001914 filtration Methods 0.000 description 15
- 238000012360 testing method Methods 0.000 description 13
- 238000010586 diagram Methods 0.000 description 12
- 238000013528 artificial neural network Methods 0.000 description 10
- 238000013461 design Methods 0.000 description 9
- 239000004575 stone Substances 0.000 description 9
- 238000012545 processing Methods 0.000 description 8
- 238000012549 training Methods 0.000 description 8
- 230000004913 activation Effects 0.000 description 7
- 230000006870 function Effects 0.000 description 7
- 238000005070 sampling Methods 0.000 description 7
- 230000003993 interaction Effects 0.000 description 6
- 238000005516 engineering process Methods 0.000 description 5
- 238000002474 experimental method Methods 0.000 description 5
- 239000004744 fabric Substances 0.000 description 5
- 230000005684 electric field Effects 0.000 description 4
- 230000035945 sensitivity Effects 0.000 description 4
- 238000005452 bending Methods 0.000 description 3
- 230000008901 benefit Effects 0.000 description 3
- 238000013145 classification model Methods 0.000 description 3
- 238000012706 support-vector machine Methods 0.000 description 3
- 238000004458 analytical method Methods 0.000 description 2
- 238000003491 array Methods 0.000 description 2
- 230000005540 biological transmission Effects 0.000 description 2
- 238000013480 data collection Methods 0.000 description 2
- 230000026058 directional locomotion Effects 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 230000005057 finger movement Effects 0.000 description 2
- 238000011160 research Methods 0.000 description 2
- 230000003068 static effect Effects 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000007635 classification algorithm Methods 0.000 description 1
- 239000004020 conductor Substances 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000013079 data visualisation Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 239000000284 extract Substances 0.000 description 1
- 238000005286 illumination Methods 0.000 description 1
- 239000003999 initiator Substances 0.000 description 1
- 229910001338 liquidmetal Inorganic materials 0.000 description 1
- 239000000696 magnetic material Substances 0.000 description 1
- 238000012634 optical imaging Methods 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 230000004044 response Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/017—Gesture based interaction, e.g. based on a set of recognized hand gestures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
- G06F18/2411—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on the proximity to a decision surface, e.g. support vector machines
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/70—Denoising; Smoothing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20024—Filtering details
- G06T2207/20032—Median filtering
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Computation (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Biology (AREA)
- Bioinformatics & Computational Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Computational Linguistics (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Human Computer Interaction (AREA)
- Measurement Of Length, Angles, Or The Like Using Electric Or Magnetic Means (AREA)
Abstract
The invention discloses a capacitive imaging sensor and a gesture form sensing method, and discloses a capacitive imaging sensor with a multi-channel capacitor combined plate structure. The high-frequency capacitance digital converter collects capacitance values of the capacitor, multi-path capacitance values of the high-frequency capacitance digital converter can represent the three-dimensional form of a target object, and the formed three-dimensional form is the imaging result of the capacitive imaging sensor on the target object. The multi-channel capacitor combined plate structure has higher resolution ratio on the object form, and can establish a neural network model for multi-channel capacitance data to quickly realize accurate identification on the object form. The form perception system avoids external electromagnetic interference and environment visible light interference, reduces the complexity of a classification recognition algorithm, and ensures the accuracy of object form recognition. The invention can be applied to the field of gesture form perception, and can effectively expand the gesture form recognition sensor types.
Description
Technical Field
The invention belongs to the technical field of capacitive imaging sensing, and relates to a non-contact gesture form sensing method.
Background
The perception of gesture form is one of the important research directions in human-computer interaction technology, and the method is mainly used in the fields of robot control, virtual reality, sign language recognition, game control and the like, and has recently gained wide attention and research. The method aims to analyze the specific meaning of each space gesture by using a computer through a certain means so as to acquire the whole expression information of a gesture initiator, thereby achieving the purpose of realizing human-computer interaction. The current gesture shape perception is mainly divided into two categories of non-contact recognition and contact recognition.
The non-contact gesture form perception scheme is mainly based on gesture recognition perception of computer vision, mainly depends on optical imaging, collects spatial image information to analyze hand characteristics, and distinguishes images in a deep learning mode. The gesture form perception technology based on computer vision can identify complex gestures, is high in accuracy, is extremely easy to be interfered by external environment light, is low in operation efficiency and poor in real-time performance, and depends on high-performance peripherals and processors. In the CN201711224479.7 patent, gesture recognition is mainly performed by means of images, and the recognition operation relies on a depth information recognition technology, and is basically developed by color images collected by a camera. The image is greatly influenced by ambient light, and because the data and information input in different illumination environments have large difference, the identification efficiency in the environment with poor light is low. Another non-contact scheme is to use a capacitive sensing sensor to identify gestures, and the cn201710473710.x patent uses a capacitive detection module to detect dynamic gestures, and is applied to a human-computer interaction system of household appliance control and automobiles. The invention uses the capacitance sensing unit to detect the gesture approach signal, if the sensing signal is effective, the gesture control signal is generated according to the sensing sequence of the sensing signal, so as to judge the dynamic sensing action of the gesture. The recognized gesture is single, and only the movement direction of the gesture can be judged, for example, the gesture can be recognized from left to right and from top to bottom, but fixed gestures cannot be recognized. It can be seen that there are many disadvantages, such as single gesture motion, large requirement for gesture motion space, and failure to complete gesture recognition.
There are currently a few non-mainstream non-contact gesture shape sensing technologies based on radar, infrared, laser, electromagnetic sensors, etc., but due to the long response time of sensors (e.g., radar), the influence of the environment (e.g., infrared and laser) and the insensitivity of targets made of non-magnetic materials (e.g., magnetic sensors). Compared with other technologies, the capacitive imaging sensor of the invention has good characteristics: simple design, low power consumption, high sensitivity, strong anti-interference capability and the like.
At present, the mainstream gesture form perception solution is mainly contact identification, and the problem that the gesture form perception solution is not easy to wear exists, such as wearable strain resistors and inertial sensors. The CN201721290807.9 patent designs a conductive material layer and a recognition control circuit for human skin contact, and CN107907146A proposes a gesture recognition method based on liquid metal electrode gloves, such contact gesture recognition is cumbersome to wear and needs to contact with human skin.
The design intensively considers the defects existing in the current gesture recognition scheme, and provides the non-contact capacitive imaging sensor based on the capacitance. The array type capacitive imaging sensor with high sensitivity is designed, a bottom layer single chip microcomputer and a digital capacitive converter complete capacitance value collection and filtering algorithm processing of a multi-path capacitor combination, and an upper computer receives stable capacitance imaging data to realize quick and accurate judgment of non-contact gesture forms.
Disclosure of Invention
The capacitance imaging sensor is adopted to carry out three-dimensional imaging on a target object, a capacitor is formed between the target object and the capacitance sensing polar plate, the capacitance value is increased along with the decrease of the distance between the outline edge of the target object and the capacitance polar plate, and otherwise, the capacitance value is decreased; the capacitance value is increased along with the increase of the relative area of the outline edge of the target object and the capacitance plate, and is reduced otherwise; the space polar plate combination of the capacitive imaging sensor is constructed by relying on the principle and forms a plurality of capacitor combinations, and the combination of the capacitors and the space distribution of the polar plates can sense the contour edge of the target object through the change of capacitance. The object contour edge can change the capacitance value of each capacitor, the capacitance value of each capacitor changes along with the space change of the contour edge, the three-dimensional shape of the target object can be drawn, and the formed three-dimensional shape is the imaging result of the capacitive imaging sensor on the target object. The imaging result can be used for morphological perception and tracking of complex gestures.
The present invention constructs a non-contact array capacitive imaging sensor system to measure the capacitive signal generated by the target object and the plate without direct contact with the target object profile edge. The invention aims to provide an object form perception scheme resisting external electromagnetism and ambient visible light interference. The invention designs an array type space capacitor polar plate, the polar plate is made of strip electrodes, and the space array capacitor polar plate collects capacitance values generated by the contour edge of an object in a frame and uses the capacitance values for imaging of gesture forms. Variable capacitance is formed between the hand and the array polar plate, and the capacitance is changed due to the change of the relative area of the distance between the hand with different forms and the polar plate in the space. The high-frequency digital capacitance converter is used for collecting and amplifying weak capacitance generated by the array polar plate and the target gesture in the space, and the gesture capacitance value can be captured with high precision. Each single-array strip electrode has one path of signal acquisition, and the multi-path array capacitance signals can acquire capacitance values of corresponding multi-channel polar plates for different specific gestures under the state that hands are randomly placed in a suspended state, so that multi-dimensional characteristic data acquisition of human body gestures is realized. The invention relates to a non-contact sensing technical scheme with low power consumption, low cost and high resolution, which takes multi-channel capacitance data as gesture form sensing characteristics, wherein the capacitance has higher resistance to environment light intensity interference and electromagnetic interference, and can fully realize accurate sensing of gesture forms in high-noise environments and environments with weak light intensity. The array capacitance data of the multi-channel capacitor combination of the array capacitance imaging sensor can be effectively applied to machine learning and deep learning, and efficient non-contact gesture form sensing is achieved.
The capacitive imaging method provided by the invention is an imaging method free from ambient light interference and electromagnetic interference, and has the advantages of simple construction, low cost, good stability and the like. Compared with the existing imaging system, the imaging system formed by the invention has strong anti-interference performance and can be conveniently applied to the fields of gesture recognition, human-computer interaction and the like. The array structure of the non-contact capacitive imaging sensor is as follows:
structure 1: as shown in fig. 1, a diamond-frame capacitive imaging sensor structure is designed, and four channels CH0, CH1, CH2 and CH3 of a capacitive transducer are respectively connected to strip electrodes of a diamond frame, so as to form a diamond-frame capacitive imaging sensor with 4 channels of collected signals. Each polar plate finishes one-path signal acquisition, the two-path polar plates form a pair of opposite capacitors, and a pair of variable capacitors is formed between the polar plates and the target gesture. As shown in fig. 6, a variable capacitor is formed between the capacitor plate and the gesture with a specific shape, and the variable capacitor is used as spatial imaging data sensed by the gesture shape. Channels CH0, CH1, CH2 and CH3 are connected to a capacitor pole plate through a shielding wire not longer than 10CM to form two pairs of capacitor sampling plates, aiming at the space complexity of complex gestures, the channels CH0 and CH1 are designed into a top diamond-shaped frame type pole plate, and channels CH2 and CH3 are connected with a bottom pole plate, so that weak signal acquisition of the space complex gestures is realized. The capacitance change can be caused by the weak change of the number of the fingers, the approaching distance between the fingers and each polar plate and the relative area change of the fingers and the polar plates can generate a sampling value with obvious capacitance difference, such as two gestures of 'stone' and 'scissors', the obvious capacitance increase can be caused by the fingers approaching the top polar plate in the 'scissors' gesture, and the gesture change and the gesture type can be determined by the multi-polar plate capacitance data characteristics.
Structure 2: as shown in fig. 2, the spatial structure of the spherical array capacitive imaging sensor uses 3 capacitive converters, and 9 channels of the capacitive converters are respectively connected to a spatial array plate to be used as an array spatial capacitive acquisition sensor. The spatial structure of the array plates is shown in fig. 2, each array is used as a capacitor plate and is responsible for completing signal acquisition, a hand is used as a common plate of a relative capacitor, and a pair of variable capacitors is formed between the plates and a target gesture. The single-channel connection circuit for capacitance acquisition is shown in fig. 6, a variable capacitance is formed between a strip-shaped capacitor plate and a target gesture, and the multi-dimensional array capacitance is used as feature data for gesture classification. The spatial structure of the spherical array type capacitive imaging sensor effectively improves the gesture resolution in the vertical direction, is sensitive to the movement of fingers in the vertical direction, and effectively improves the resolution of complex gestures in the vertical direction. Because the strip-shaped arrays are sequentially arranged along the cambered surface of the sphere in the space, when the finger moves in the vertical direction, the capacitance value of one polar plate is weakened, and the capacitance value of the opposite polar plate is increased. The structure is sensitive to capacitance change caused by finger movement in the vertical direction, and the change of the number of fingers can cause obvious change of the capacitance of the polar plate opposite to the fingers.
Structure 3: as shown in fig. 3, the spatial structure of the horizontal semicircular array capacitive imaging sensor applies 3 capacitive converters, and 12 channels of the capacitive converters are respectively connected to the spatial array plate, and the bottom plate is connected to GND. Each array in the transverse semicircle is used as a capacitor polar plate to finish signal acquisition, and the hand is placed on the surface of the GND polar plate at the bottom and is used as a common polar plate of the relative capacitor. As shown in FIG. 6, a variable capacitor is formed between the capacitor plate and the gesture of the target, and the variable capacitor is used as the characteristic data for gesture classification. The space structure of the transverse semicircular array type capacitive sensor can effectively improve the gesture resolution in the depth direction, is sensitive to the movement change of the gesture in the depth direction, and effectively improves the resolution of complex gestures in the depth direction. Because the strip-shaped arrays are sequentially arranged in the vertical direction in the space, when a finger moves in the depth direction, the capacitance value of some plates can be changed. When the finger stretches out, a plurality of polar plates can sense the capacitance; when the finger is bent and changed, the number of the opposite capacitor plates is changed, and the capacitance value of the array is obviously changed. The structure is sensitive to capacitance change caused by finger movement in the depth direction, and the change of the bending degree of the finger can cause the change of the polar plate opposite to the finger, for example, the finger bending movement can cause the finger to be far away from the former polar plate and to be close to the latter polar plate. The method realizes the resolution of complex gestures changing in the depth direction, and the change of capacitance is caused by the weak change of the extending hand index.
Structure 4: as shown in fig. 4, the longitudinal semicircular array type capacitive imaging sensor space structure. The structure uses 3 capacitive converters, 12 channels of the capacitive converters are respectively connected to a space array polar plate, and a bottom polar plate is connected with GND (ground potential) to be used as an array type capacitive imaging sensor. Each array is used as a capacitor plate and is responsible for completing signal acquisition of one path, a hand is used as a common plate of a relative capacitor, and the hand is placed on the surface of a GND (ground) plate at the bottom. As shown in FIG. 6, a variable capacitor is formed between the capacitor plate and the gesture of the target, and the variable capacitor is used as the characteristic data for gesture classification. The longitudinal semicircular array type capacitive sensor space structure is sensitive to the depth of the gesture and the number of the extending fingers, and the gesture resolution in the depth direction and the vertical direction can be effectively improved. For example, the degree of bending of the finger will cause the capacitance of the plate opposite to the finger to change, and when the finger is bent away from the corresponding plate, the capacitance value will be weakened; when the gesture vertically moves up and down, the opposite target polar plate is changed; when the number of the extending fingers is different, the number of the plates causing the capacitance change is also different. The structure has sensitive sensing capability to the number of the extending fingers and the depth of the inserted fingers.
The design of the gesture form perception system based on the capacitive imaging sensor comprises the following hardware and software parts:
(1) designing a hardware system: the four array space capacitance imaging sensors are designed, and a high-frequency digital capacitance converter is used for collecting complex gesture forms in a frame and capacitance values generated by capacitor plates.
The array capacitive imaging sensor space hardware structure is shown in fig. 1, fig. 2, fig. 3 and fig. 4, and the array capacitive plate can accurately acquire complex gesture forms in a frame and capacitance values generated by the capacitive plate. The design considers the space free form perception design when hands with different forms are suspended, the array type space capacitance imaging sensor with high sensitivity is designed, the bottom layer single chip microcomputer completes multi-channel filtering algorithm processing, and the upper computer receives and obtains stable array capacitance values, so that the non-contact gesture form can be rapidly and accurately judged.
The bottom layer controller selects STM32F103, and a plurality of FDC2214 capacitance digital converters are used for respectively connecting the multi-channel capacitance acquisition interfaces with the unit capacitance electrodes. In the closed space, variable capacitance is formed between each group of capacitor plates and a hand, and the capacitance is changed due to the change of the relative area between a gesture with a specific shape in the space and the plates. For example, a gesture is placed on the array capacitive imaging sensor, and some specific fingers of the gesture will approach the opposite capacitive array plates, and the capacitance values of the plates will increase significantly. The single-array electrode plate surface is provided with one path of signal acquisition, and the multi-path array capacitance signals can acquire corresponding multi-channel array electrode plate capacitance values for different specific gestures under the condition that hands are randomly placed in a suspended state, so that the gestures of a human body can be accurately recognized.
As shown in fig. 9, 10 different types of gestures are defined as "stone", "scissors", "cloth", "1", "2", "3", "4", "5", "6", and "7", respectively, and the gestures are placed in the space of the array capacitive imaging sensor to complete the acquisition of the array capacitive data corresponding to the multiple gestures, and the array capacitive imaging sensor has higher sensitivity to the gesture change through testing. The simplified graph of the change of the electric field lines of the capacitor plates caused by the gestures is shown in fig. 10, different gestures are put into the plate array space, different capacitance values are generated between the different gestures and the different plates, and the change of the capacitance difference caused by the different gestures is shown in fig. 11.
(2) Software algorithm program: the gesture recognition and classification method based on the neural network mainly comprises two algorithms of median average filtering of a lower computer, a neural network classification and recognition algorithm of an upper computer and the like.
Algorithm 1: median average filtering:
and selecting median average filtering on the MCU, which is also called an anti-pulse interference average filtering method. Filtering each channel, continuously sampling 40 data, performing bubble sorting on the sequence, removing maximum 10 data and minimum 10 data, extracting middle 20 effective data, and calculating the arithmetic mean of the 20 data. Median average filtering can filter spikes in the time series with the aim of extracting stable filtered data. The filtered data keeps the variation trend of the original data, simultaneously removes the influence of spike pulse on analysis, can effectively eliminate accidental impulsive interference, and can eliminate sampling value deviation caused by the impulsive interference. And after the MCU completes median value average filtering on the multichannel capacitance data, sending the stable multichannel capacitance data to an upper computer through a UART (universal asynchronous receiver/transmitter) to perform classification model testing.
And 2, algorithm: neural network classification recognition algorithm
The invention adopts a simple full-connection neural network to carry out classification experiments on multi-channel capacitance characteristic data, and the network has the characteristics of simple structure, small operand, high classification and identification speed and the like and comprises an input layer and 2 hidden layers. The activation function selects a non-linear rectifying linear unit (Relu) and a Softmax activation function.
Although Relu is not differentiable at z-0, its second derivative is almost everywhere zero and in the activated state is that the first derivative is constantly 1, so it has an inherent advantage in the gradient direction compared to the Sigmoid activation function.
The method comprises the steps of designing and completing network node configuration and super-parameter initial value setting according to data characteristics, setting exponential decay of learning rate for learning rate (learning rate) in super-parameters to help network convergence, adopting early stop detection for batch processing size (batch size) and training round (epoch num) in the super-parameters to avoid overfitting of a network, and enabling the classification accuracy of gesture recognition 3 of a test set to reach more than 99% and the classification accuracy of gesture recognition 10 to reach more than 95%. Or the SVM classification algorithm is selected, so that a better static gesture classification recognition effect can be achieved, and the 10 classification accuracy rate is up to more than 95%.
The gesture form classification recognition result is applied to interactive control of the intelligent robot, multifunctional action gestures such as forward and backward movement of the robot are defined, and accurate control is carried out through space gestures.
The gestures such as the numbers '1', '2', '3' and the like are applied to the gear adjustment of the air conditioner and the on-demand of television programs, and the accurate identification of the digital gestures realizes the accurate interaction of household appliances.
Drawings
FIG. 1 is a diagram of a common frame-type capacitive imaging sensor;
FIG. 2 is a diagram of an external appearance of a ball-array frame-type capacitive imaging sensor;
FIG. 3 is a schematic diagram of an external appearance of a horizontal array frame-type capacitive imaging sensor;
FIG. 4 is a schematic diagram of an external appearance of a vertical array frame-type capacitive imaging sensor;
FIG. 5 is a simplified circuit diagram of an FDC 2214-based capacitive imaging sensor;
FIG. 6 is a circuit diagram of a unipolar plate capacitor acquisition circuit;
FIG. 7 is a circuit diagram of a capacitive imaging sensor plate capacitance acquisition circuit;
FIG. 8 is a flow chart of data collection and transmission;
FIG. 9 is a schematic diagram illustrating a test gesture shape definition;
FIG. 10 is a schematic diagram of the gesture pattern causing a change in the electric field of the array capacitor plate;
FIG. 11 is a schematic diagram illustrating differences in electric field variations in different gesture configurations;
FIG. 12 is a schematic view of channel 1 test data visualization;
FIG. 13 is a comparison graph of imaging data of a 1 channel "stone" and "scissors" gesture;
FIG. 14 is a schematic diagram of a fully-connected neural network loss iteration;
Detailed Description
The specific experimental steps are as follows:
the first step is as follows: the MCU lower computer collects the multichannel capacitance value of the array type capacitance imaging sensor, the digital capacitance converter adopts a unique narrow-band framework to eliminate harmful noise and common environmental interference, and the MCU lower computer performs median mean filtering processing on the capacitance value;
the second step is that: the lower computer visualizes analog quantity of the filtered multi-channel array capacitance value, and uploads stable multi-channel capacitance data to the upper computer for training and testing a classification model;
the third step: the structure 4 of the longitudinal array type capacitive imaging sensor is selected for data acquisition, and the structure is shown as figure 4. Acquiring capacitance data of 10 types of gestures such as three testers, namely gestures such as 'stone', 'scissors', 'cloth', '1', '2' and the like, wherein the data acquired by each gesture is 540 groups;
the fourth step: a multi-layer full-connection neural network is adopted for carrying out classification experiments on the multi-channel capacitance characteristic data, and the classification experiments comprise an input layer and 2 hidden layers. The activation function selects nonlinear rectifying linear unit Relu and Softmax activation functions.
The fifth step: establishing a fully-connected neural network model, carrying out classification training by using multi-channel capacitance data, carrying out supervised model training test by using multi-channel capacitance values corresponding to three gestures (stone, scissors and cloth) of the three testers, wherein the model is converged at about 20 periods, the 3-classification average accuracy rate is 99%, and the 10-classification accuracy rate is more than 95%.
And a sixth step: the gesture form sensing device of the capacitive imaging sensor is applied to an intelligent robot, air conditioner gear adjustment, video-on-demand of television programs and accurate recognition of digital gestures, and accurate interactive control of household appliances is achieved.
The technical solution of the present invention will be described in detail with reference to the accompanying drawings.
As shown in fig. 1, fig. 2, fig. 3, and fig. 4, the present invention designs 4 array capacitive imaging sensor spatial structures, which are used as array plate structures of the spatial capacitive imaging sensor, and can acquire capacitance data caused by a target gesture between the array plates. The embedded processor selects STM32F103 with a multi-block FDC2214 capacitive-to-digital converter. Each array in the capacitive imaging sensor structure is used as a capacitive pole plate and is responsible for completing signal acquisition of one path, a hand is used as a common electrode of a relative capacitor, and the hand is placed on the surface of a GND pole plate at the bottom. As shown in FIG. 6, a variable capacitor is formed between the capacitor plate and the gesture with a specific shape, and the variable capacitor is used as gesture shape sensing classification data. A simplified graph of the change of the electric field lines of the capacitor plates caused by a single gesture is shown in fig. 10, different form gestures are put into the plate array space, different capacitance values are generated between the different form gestures and the different plates, and the simplified graph of the change of the capacitance difference caused by the different form gestures is shown in fig. 11.
An FDC2214 capacitor-to-digital converter is used for acquiring multi-channel space capacitor values (C0, C1, C2 and C3 …), the multi-channel space capacitor values are transmitted to an outdoor unit control board through an I2C interface, an STM32F103 is used as a lower computer microprocessor, and a circuit diagram is acquired based on an FDC2214 capacitor as shown in fig. 7. The STM32F103 performs median average filtering processing on the acquired multi-channel capacitance data, performs filtering processing on each channel, continuously samples 40 data, performs bubble sorting on the sequence, removes the maximum 10 data and the minimum 10 data, extracts the middle 20 effective data, and then calculates the arithmetic average of the 20 data. The filtered data keeps the variation trend of the original data, simultaneously removes the influence of spike pulse on analysis, can effectively eliminate accidental impulsive interference, and can eliminate sampling value deviation caused by the impulsive interference. And outputting the space capacitance values with smaller variation amplitude difference, and uploading the space capacitance values serving as classification data to an upper computer to realize neural network model classification. The flow chart of multivariate data collection and transmission is shown in fig. 8.
The STM32F103 is used as a microprocessor, and the longitudinal array type frame-type capacitive acquisition sensor is selected to acquire data of different gestures in space. Gesture definitions as shown in fig. 9, 10 different types of gestures are defined as "stone", "scissors", "cloth", "1", "2", "3", "4", "5", "6" and "7", respectively. The capacitive imaging sensor will collect spatial capacitance data corresponding to the gesture.
STM32103 is selected as a processor, and STM32F103 uploads acquired multi-channel capacitance data to an upper computer after median average filtering processing. The signal collecting line of the polar plate is connected to the capacitance sampling plate through a shielding line which is not longer than 10CM, and when the conducting wire which is obtained through experimental tests and is used for connecting the channel of the sampling polar plate and the strip electrode is a common conducting wire, the interference is obviously much larger than that of the shielding line, and meanwhile, the interference is larger when the conducting wire is longer, so that the shielding line which is not larger than 10CM is selected.
The collected data are visualized, the data of the channel 1 are collected continuously, the gestures 'stone', 1, 2, 3, 4, 5 and 6 are changed continuously, and the visible data are shown in a graph 12, so that obvious distinguishing characteristics exist among the gestures in different forms. As shown in fig. 13, the gestures "stone" and "scissors" on channel 1 showed significant difference on a single channel, and the test data was stable after the filtering process.
In the model training test, a longitudinal array frame type capacitive imaging sensor is selected for data acquisition, and the structure is shown in figure 4. The method comprises the steps of carrying out capacitance data acquisition on 10 gestures of three testers, namely 'stone', 'scissors' and 'cloth', wherein the data acquisition data of each gesture is 540 groups, carrying out model training by using the data acquisition data, and then checking the generalization capability of a model and the accuracy of gesture recognition.
The invention has the beneficial effects that: the array type capacitive imaging sensor is innovatively designed, array type space capacitance values of a plurality of channels are efficiently collected, stable multi-channel gesture capacitance values are quickly extracted through median value average filtering processing, the maximum difference of gestures in a specific form can be effectively collected, and the gestures can be classified through a simple fully-connected neural network to obtain a gesture result with high accuracy. The design realizes efficient classification application of the gesture forms and achieves accurate recognition of slightly-changed gesture forms.
In the gesture form perception test, a classification experiment is carried out on multi-channel array capacitance characteristic data by adopting a multi-layer fully-connected neural network, and the classification experiment comprises an input layer and 2 hidden layers. The activation function selects nonlinear rectifying linear unit Relu and Softmax activation functions. For the multi-channel capacitance data structure, the final determination uses the network node configuration and the hyper-parameter initial value settings as shown in tables 1 and 2. For learning rate (learning rate) in the hyper-parameters, we set an exponential decay of the learning rate to help network convergence, and for batch size (batch size) and training round (epoch num) in the hyper-parameters, we use early-stop detection to avoid overfitting of the network.
TABLE 1 fully-connected network architecture
TABLE 2 hyper-parameter configuration
And (3) analyzing a test result: as shown in fig. 14, when training is performed using the data of the 3 testers, the model converges at about 20 cycles, the average accuracy of 3 classification is 99% or more, and the accuracy of 10 classification is 95% or more.
TABLE 3 full connectivity network test results
The array type capacitive imaging sensor can acquire capacitance data with obvious gesture space morphological characteristics, and the data is combined with a full-connection network to obtain better classification identification accuracy. Array capacitive imaging sensor has realized the spatial feature capacitance data acquisition of gesture form, and this data can combine more complicated neural network algorithm to carry out classification prediction in the later stage, will accomplish more complicated space gesture recognition. The establishment of the classification model can also carry out the rapid recognition of the gesture posture through machine learning algorithms such as a Support Vector Machine (SVM) and the like, and the model can also obtain better classification recognition effect on static three-dimensional gestures.
The capacitive imaging sensor and the gesture form sensing method are applied to the following examples:
example 1: the gesture form classification recognition result is applied to interactive control of the intelligent robot, multifunctional action gestures such as forward and backward movement of the robot are defined, and accurate control is carried out through space gestures. The non-contact capacitive imaging sensor is used as a data acquisition sensor for man-machine interaction, an automatic recognition system is established by the gesture form recognition model, and the gesture recognition result of the fully-connected neural network is used for interacting with the robot. The 10 gestures are allocated to specific motion instructions of the directional motion of the intelligent robot, and the intelligent robot is guided to complete the directional movement indoors by using different gestures.
Example 2: the method has the advantages that the target gestures are rapidly collected and recognized by using the capacitive imaging sensor and the gesture form sensing method, gesture form information (1, 2, 3 and the like) of a target user is rapidly and accurately recognized, digital information of the gestures is used as control decision signals of a television and an air conditioner, the control decision signals are applied to gear adjustment of the air conditioner and video-on-demand of television programs, and rapid and accurate control over household appliances is achieved.
Claims (6)
1. A capacitive imaging sensor, characterized by:
the capacitance imaging sensor is formed by combining 3 groups or more than 3 groups of capacitors, and one group of capacitors is formed by two electrodes and a dielectric material between the two electrodes;
the combination and spatial arrangement of the capacitors realize the accurate detection of capacitance change caused by the three-dimensional shape of an object in the space, and the capacitance imaging sensor can sense the fine capacitance value in the three-dimensional space;
when the form and the position of the target object change, the capacitance of each capacitor changes, the sensing polar plate space distribution of the capacitance imaging sensor is constructed, a plurality of capacitor combinations are formed, and the contour edge of the target object can be sensed through the capacitance change of the capacitor combinations and the space distribution;
the object contour edge is close to or far away from the capacitor plate, the capacitance is increased or decreased, the target object form detection is realized, the capacitance can draw the three-dimensional form of the target object along with the spatial change of the contour edge, the formed three-dimensional form is the imaging result of the capacitive imaging sensor on the target object, and the imaging result can be used for the identification and tracking of complex gestures.
2. The capacitor assembly and spatial distribution of claim 1, wherein:
the capacitor combination and spatial distribution include, but are not limited to, spatial right combination distribution, spatial diamond frame combination distribution, spatial spherical shape arrangement, spatial spherical shape array combination distribution and the like, and the capacitance sensor with the three-dimensional structure senses capacitance change of an internal space caused by different object shapes.
3. The target object morphology detection as claimed in claim 1, characterized in that:
and the target object comprises a conductive object and a non-conductive object, the capacitance value of each capacitor changes in the process of combining the approaching capacitors, and if a certain capacitance of the imaging sensor is increased, the outline edge of the target object is judged to approach the position of the capacitor plate or the relative area of the outline edge of the target object is judged to be increased, so that the dynamic information of the object outline in a certain direction is presented.
4. A gesture form sensing method of a capacitive imaging sensor is characterized by comprising the following steps:
the three-dimensional space form of the gesture can be outlined by the change of the capacitance values of the capacitor combination, the formed three-dimensional form is the imaging result of the capacitance imaging sensor on the gesture form, and the imaging result directly judges the gesture form result through a form perception model;
different gesture forms form different distances and relative areas with the polar plate of the space capacitor, the change of the distances and the relative areas influences the size of the capacitance value, and different gesture form characteristics influence the capacitance value of the corresponding polar plate of the capacitor, so that the gesture forms can be directly recognized;
a gesture form perception model can be constructed by using the combined capacitance of a plurality of groups of capacitors to accurately recognize and judge the human gesture form in real time.
5. The morphological perceptual model of claim 4, wherein:
the capacitance combination data can be fused and applied to a pattern recognition algorithm, a machine learning algorithm or a deep learning algorithm, a form perception model is established by using the capacitance combination data, the model can calculate and obtain contour edge position information of object forms and accurate classification of the object forms, and the form perception model can be used for establishing a human body gesture recognition system.
6. The human gesture recognition system of claim 5, wherein:
the three-dimensional gesture form is used as an imaging perception target, the recognition result of the three-dimensional gesture form is applied to interactive control of the intelligent robot, action gestures such as forward and backward movement of the robot are defined, and accurate control is performed by using a space gesture;
the gestures such as the numbers '1', '2', '3' and the like are applied to the gear adjustment of the air conditioner and the on-demand of television programs, the digital gestures are accurately recognized, and the accurate interactive control of household appliances is realized.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110045386.8A CN113673292A (en) | 2021-01-14 | 2021-01-14 | Capacitive imaging sensor and gesture form sensing method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110045386.8A CN113673292A (en) | 2021-01-14 | 2021-01-14 | Capacitive imaging sensor and gesture form sensing method |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113673292A true CN113673292A (en) | 2021-11-19 |
Family
ID=78538009
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110045386.8A Pending CN113673292A (en) | 2021-01-14 | 2021-01-14 | Capacitive imaging sensor and gesture form sensing method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113673292A (en) |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107079141A (en) * | 2014-09-22 | 2017-08-18 | 三星电子株式会社 | Image mosaic for 3 D video |
| CN108369345A (en) * | 2015-10-20 | 2018-08-03 | 奇跃公司 | Virtual objects are selected in three dimensions |
| CN109283230A (en) * | 2018-11-30 | 2019-01-29 | 燕山大学 | A kind of planar array capacitance imaging method and system |
| CN109343694A (en) * | 2018-08-13 | 2019-02-15 | 浙江大学 | A kind of gesture recognition system and method for finger-guessing game finger-guessing game game |
| CN109491507A (en) * | 2018-11-14 | 2019-03-19 | 南京邮电大学 | Gesture identifying device based on FDC2214 |
| CN109738494A (en) * | 2019-01-29 | 2019-05-10 | 燕山大学 | A kind of Sensitivity distribution method of coplanar array capacitor sensor |
| CN110275614A (en) * | 2019-05-30 | 2019-09-24 | 福建工程学院 | A kind of non-contact gesture identification device and its method |
| CN110766057A (en) * | 2019-10-09 | 2020-02-07 | 南京理工大学 | Gesture recognition device and method |
| CN112099624A (en) * | 2020-08-25 | 2020-12-18 | 李志斌 | Multimode diamond-shaped frame type capacitive sensing gesture recognition system |
-
2021
- 2021-01-14 CN CN202110045386.8A patent/CN113673292A/en active Pending
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107079141A (en) * | 2014-09-22 | 2017-08-18 | 三星电子株式会社 | Image mosaic for 3 D video |
| CN108369345A (en) * | 2015-10-20 | 2018-08-03 | 奇跃公司 | Virtual objects are selected in three dimensions |
| CN109343694A (en) * | 2018-08-13 | 2019-02-15 | 浙江大学 | A kind of gesture recognition system and method for finger-guessing game finger-guessing game game |
| CN109491507A (en) * | 2018-11-14 | 2019-03-19 | 南京邮电大学 | Gesture identifying device based on FDC2214 |
| CN109283230A (en) * | 2018-11-30 | 2019-01-29 | 燕山大学 | A kind of planar array capacitance imaging method and system |
| CN109738494A (en) * | 2019-01-29 | 2019-05-10 | 燕山大学 | A kind of Sensitivity distribution method of coplanar array capacitor sensor |
| CN110275614A (en) * | 2019-05-30 | 2019-09-24 | 福建工程学院 | A kind of non-contact gesture identification device and its method |
| CN110766057A (en) * | 2019-10-09 | 2020-02-07 | 南京理工大学 | Gesture recognition device and method |
| CN112099624A (en) * | 2020-08-25 | 2020-12-18 | 李志斌 | Multimode diamond-shaped frame type capacitive sensing gesture recognition system |
Non-Patent Citations (1)
| Title |
|---|
| 张立峰;朱炎峰;: "电容层析成像系统阵列电极三维优化设计", 计量学报, no. 08, pages 1 - 3 * |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110222643B (en) | Steady-state visual evoked potential signal classification method based on convolutional neural network | |
| Li et al. | A multi-scale fusion convolutional neural network based on attention mechanism for the visualization analysis of EEG signals decoding | |
| CN111553307B (en) | Gesture recognition system fusing bioelectrical impedance information and myoelectric information | |
| CN105654037B (en) | A kind of electromyography signal gesture identification method based on deep learning and characteristic image | |
| KR20110132349A (en) | Device and method for monitoring an object's behavior | |
| US20170220923A1 (en) | Gesture classification apparatus and method using emg signal | |
| Boyali et al. | Spectral Collaborative Representation based Classification for hand gestures recognition on electromyography signals | |
| CN112099624B (en) | Multimode diamond-shaped frame type capacitive sensing gesture recognition system | |
| CN105739702A (en) | Multi-posture fingertip tracking method for natural man-machine interaction | |
| CN103353935A (en) | 3D dynamic gesture identification method for intelligent home system | |
| CN102799270B (en) | Human-computer interaction method based on electrostatic detection and myoelectric detection | |
| CN102830797A (en) | Man-machine interaction method and system based on sight judgment | |
| CN107212883B (en) | A kind of mechanical arm writing device and control method based on brain electric control | |
| CN104598090B (en) | The multi-point touch locating method and touch panel device of a kind of touch-screen | |
| CN102789326B (en) | Non-contact man-machine interaction method based on electrostatic detection | |
| KR20120122617A (en) | Electroencephalography Classification Method for Movement Imagination and Apparatus Thereof | |
| CN107480716A (en) | A kind of combination EOG and video pan signal recognition method and system | |
| CN108985157A (en) | A kind of gesture identification method and device | |
| CN107193374A (en) | A kind of detection means and detection method of the intentional gesture motion of active | |
| CN108932060A (en) | Gesture three-dimensional interaction shadow casting technique | |
| Baldwin et al. | Gait analysis for fall prediction using hierarchical textile-based capacitive sensor arrays | |
| WO2018103495A1 (en) | Eog-based method and apparatus for asynchronous character input | |
| CN109583294A (en) | A kind of multi-mode Human bodys' response method based on sport biomechanics | |
| CN106598234B (en) | Gesture identification method based on inertia sensing | |
| CN109343694B (en) | Gesture recognition system and method for finger-guessing and punching game |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20211119 |
|
| WD01 | Invention patent application deemed withdrawn after publication |