Disclosure of Invention
Aiming at the defects in the prior art, the invention provides a 4D gesture recognition method based on an MIMO frequency modulation continuous wave millimeter wave radar, which can track the micro motion of a hand while obtaining the accurate three-dimensional space positioning of the hand, thereby completing the detection of the fourth dimension.
The invention is realized by the following technical scheme:
the invention relates to a 4D gesture recognition method of an MIMO frequency modulation continuous wave millimeter wave radar, which comprises the steps of collecting signals of an MIMO radar antenna array, generating a virtual array, obtaining specific positioning information of a hand in a 3D space through digital beam forming, further classifying gesture movement according to the 3D space positioning information, and carrying out corresponding movement detection according to different types of gestures to obtain a tracking result.
The invention relates to a system for realizing the method, which comprises the following steps: MIMO signal processing unit, digital beam forming unit, gesture motion judge unit and little gesture tracking unit of phase method, wherein: the MIMO signal processing unit is connected with the radar ADC sampling data to process the most original radar signal, the digital beam forming unit processes the signal obtained by the MIMO signal processing unit and obtains the 3D space positioning information of a target, the gesture motion judging unit judges the whole gesture motion according to the space position of a hand in the digital beam forming unit, and the phase method micro gesture tracking unit is responsible for tracking the extremely tiny hand motion track.
Technical effects
The invention integrally solves the defect that the prior art cannot track the space position of gesture movement and the slight movement of the hand at the same time; compared with the prior art, the method can acquire the 1D fine motion change track of the hand with high precision while acquiring the 3D space position information of the gesture, and forms gesture motion recognition with 4 dimensions, namely 4D gesture recognition. Based on an FMCW radar with a higher working frequency band, the gesture space distance resolution can reach 5 cm; by utilizing an improved digital beam forming technology, the spatial angle resolution of the hand can reach 2 degrees, and meanwhile, the visual angle reaches a horizontal angle +/-40 degrees and a pitch angle +/-30 degrees; the detection of sub-millimeter-scale micro hand movements can be achieved using a phase-based micro motion tracking algorithm.
Detailed Description
As shown in fig. 2, the MIMO FMCW radar system for implementing the 4D gesture recognition method in this embodiment includes: signal source 1, transmission power divider 2, reception power divider 3, power amplifier 4, transmission antenna array TX, reception antenna array RX, low noise amplifier 5, low pass filter 6, analog-to-digital converter 7 and signal processor 8, wherein: a signal source 1 generates a transmitting signal, and the transmitting signal is output to ports of transmitting antennas TX through a transmitting power divider 2; meanwhile, the transmitted signals are mixed with signals of all receiving antennas RX through the receiving power divider 3, and the signals subjected to down-conversion are subjected to gesture recognition through the signal processor 8 after being converted and sampled by the analog-to-digital converter 7 after passing through the low-noise amplifier 5 and the low-pass filter 6.
As shown in fig. 3, the structure of the transmit antenna array Tx and the receive antenna array RX is specifically a 3 × 4 MIMO radar antenna array, that is, three transmit antennas Tx2-Tx4 and four receive antennas RX1-RX4 are adopted.
As shown in fig. 4, the MIMO radar antenna array senses a target in an environment through a plurality of transmitters and receivers, wherein: a MIMO radar with M transmit antennas and N receive antennas is equivalent to a SIMO radar with 1 transmit antenna, mxn receive antennas.
The transmitting antenna array Tx in the MIMO radar antenna array is a vertically arranged three-unit linear array with equal half-wavelength intervals; the receive antenna array RX is a horizontally arranged four-element linear array with equal half-wavelength spacing, so the equivalent virtual array is a 3 x4 planar array.
Each element in the virtual array of the MIMO radar antenna array is (m, n), wherein: m is the row of the element, n is the column of the element, when the signal received by the (m, n) th array element in the virtual array is x(m,n)(t), the time domain signal received by Rxn after transmission from Txm. A total of 12 channels of data are thus available which are acquired by the digital-to-analog converter 7.
As shown in FIG. 5, for the digital beamforming process for a linear array, when the linear array is equidistantThe spacing of the columns being d, the signal being from the array normal at θ0Angle of incidence, then:
the existing DBF treatment process comprises the following steps: the signal processor 8 for a given direction theta0Output signal Y ═ WT(θ0) X, wherein: received signal X ═ X1,X2,...,XNThe received signal of the ith array element in the Nth array element is Xi,W={w0,w1,...,wNIs the signal steering vector, wi=2πdi·sin(θ0N, i is 1, 2, and the phase difference Δ Φ between signals received by two adjacent antennas is kdsin θ0=2πdsinθ0K corresponds to the propagation constant of the electromagnetic wave, λ being the wavelength.
Processing procedure of 3D space positioning in this embodiment is: at a given horizontal angle (theta) and a given pitch angle
And (4) respectively carrying out the steps of (i) in the direction and multiplying the results: the
signal processor 8 for a given horizontal angle (theta) and a given pitch angle
Directional output signal
Wherein: x' is a
3X 4 matrix of received signals, W (theta) and
respectively a quaternary linear array pointing horizontal angle (theta) and a ternary linear array pointing pitch angle
A signal steering vector of time; for intermediate frequency signals after FMCW radar down-conversion
Wherein: s
Tx(t) and s
Rx(t) is the received and transmitted signal, A is the signal amplitude, gamma is the slope of the frequency modulation of the FM continuous wave, f
cFor the carrier frequency, R (τ) is the relative distance of the target and radar, τ is the so-called "slow time", and c is the speed of light.
Obtaining a frequency spectrum of the intermediate frequency signal by Fourier transform, and obtaining the frequency spectrum of the intermediate frequency signal by R ═ f
IFc/2 γ further converts the frequency spectrum into a distance spectrum, i.e.
Thus, any given direction in space is accomplished
Obtaining a distance spectrum signal of (1).
But the frequency f of the intermediate frequency signalIFNot sensitive to small displacements, e.g. when objects stay in the same range column, fIFAnd is not changed. But intermediate frequency signal sIF(f) Is very sensitive to small displacements. It is possible to perform the detection of the minute motion of 1D by acquiring the phase change of the intermediate frequency signal. By extracting the phase term Φ (t) of the intermediate frequency signal, the small movements can be calculated from this: x (t) ═ c Φ (t)/4 pi fc。
As shown in fig. 6, the overall 4D gesture recognition after obtaining the 3D spatial position information and the 1D motion trajectory tracking respectively is specifically:
1) acquiring signals of an MIMO radar antenna array, generating a virtual array, and obtaining 3D space positioning of a hand through digital beam forming;
2) according to the result of the spatial positioning in the step 1, the large gesture of the radial motion, the large gesture of the plane and the micro gesture, the specific steps are as follows: view changes in distance terms: if the distance column of the object is changed all the time in the whole hand motion, the gesture can be classified as a large gesture of radial motion. If the distance column where the target is located is kept unchanged in the process of hand movement, the hand movement can be divided into plane gestures;
3) for the plane gesture, extracting the 3D space positioning of the hand obtained in the step 1, selecting the hand on an xy plane vertical to the radar, and if the distance of the target motion exceeds a threshold Ad, identifying the plane large gesture according to the specific motion in which direction; otherwise, the target is deemed to be performing a small movement.
4) Corresponding motion detection is carried out according to different types of gestures, and the method specifically comprises the following steps: the large gesture of the radial motion adopts a mode of recording the radial motion information in the 3D space position after the digital beam forming; for the large planar gesture, a mode of recording positioning information in an xy plane in a 3D space position after digital beam forming is adopted; and for the micro gesture, the phase of the radar intermediate frequency signal is extracted and then demodulated and restored, so that 1D micro motion tracking is performed, and any one of three tracking results is obtained.
As shown in fig. 7, the present embodiment further verifies the settings through experiments: the hand is located 20cm directly in front of the radar. The radar was calibrated before the experiment by placing a corner reflector at a specified angle directly in front of the radar. Thereafter, large and small hand movements are detected.
As shown in fig. 8, the large gesture includes: the radial back and forth movement of the hand, the upward sliding, the downward sliding, the left sliding, the right sliding and the two diagonal sliding of the plane are a series of actions. In this example, Δ d is takenx=ΔdyThe determination of the planar gesture is made 5 cm. Each row of fig. 8 corresponds to a gesture. In each row, the first image is a schematic representation of the gesture movement, and the second and third images are the locations of the hand position in the xy two-dimensional plane at the beginning and end of the gesture movement, respectively. And performing space division of a two-dimensional plane according to the position of the target. The fourth graph is a motion track graph of the object, and the fifth graph is motion identification according to the track and a two-dimensional space area where the hand is located in the whole motion process. The last action is the identification of radial motion. The radial motion is successfully judged because the distance column of the target is kept unchanged in the whole motion. By extracting data from the distance spectrum, a plot of distance versus time for radial motion can be obtained. As can be seen in FIG. 8, the invention is applicable to large gesturesThe method is effective and can accurately recognize various large gesture actions.
As shown in fig. 9, the micro-gesture includes: sliding of the thumb. And (4) experimental result graphs. During the experiment, the thumb makes a reciprocating movement along the index finger with an amplitude of about 10 mm. In this case, the minute motion is tracked by using the phase of 1D. The upper right graph of fig. 9 is a graph of distance traveled versus time, and the trend of back and forth motion can be seen. By derivation of the motion trail, a relation graph of the motion speed and the motion time can be obtained. Wherein, when the moving speed is 0, the motion of the thumb to the top is represented. By calculating the positive and negative of the acceleration at the point, whether the point is the end point closest to the radar or the end point farthest from the radar can be judged, as can be seen from fig. 9, the method provided by the invention is very accurate in identification of micro-motion, and can achieve millimeter-level detection.
Through specific practical experiments, under the environment settings of a microwave darkroom and a normal office, the device is operated by using sawtooth wave modulation, 4GHz frequency sweep bandwidth and 60GHz central frequency, and a receiving and transmitting antenna is configured into 4 receiving antennas with 3 transmitting antennas, so that the experimental data which can be obtained is as follows: the motion trail diagrams of the large gesture and the small gesture are shown in fig. 8 and fig. 9.
Compared with the prior art, the method utilizes two-dimensional digital beam forming, and the spatial angle resolution of detection reaches 2 degrees; the distance detection precision is 5cm by means of high-frequency-band sawtooth wave frequency sweeping; the micro-motion demodulated by the FMCW intermediate frequency signal phase information reaches the sub-millimeter level precision.
In conclusion, the invention uses the multi-channel data of the MIMO array to obtain the three-dimensional space positioning information of the target, and simultaneously focuses on the phase change of the intermediate frequency signal caused by the micro motion, thereby combining the two, and completing the creative 4D gesture recognition.
The foregoing embodiments may be modified in many different ways by those skilled in the art without departing from the spirit and scope of the invention, which is defined by the appended claims and all changes that come within the meaning and range of equivalency of the claims are therefore intended to be embraced therein.