WO2020042121A1 - 一种手势识别方法及终端、存储介质 - Google Patents

一种手势识别方法及终端、存储介质 Download PDF

Info

Publication number
WO2020042121A1
WO2020042121A1 PCT/CN2018/103362 CN2018103362W WO2020042121A1 WO 2020042121 A1 WO2020042121 A1 WO 2020042121A1 CN 2018103362 W CN2018103362 W CN 2018103362W WO 2020042121 A1 WO2020042121 A1 WO 2020042121A1
Authority
WO
WIPO (PCT)
Prior art keywords
millimeter wave
signal
terminal
information
standard
Prior art date
Application number
PCT/CN2018/103362
Other languages
English (en)
French (fr)
Inventor
周安福
马华东
刘建华
杨宁
Original Assignee
Oppo广东移动通信有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Oppo广东移动通信有限公司 filed Critical Oppo广东移动通信有限公司
Priority to EP18923764.7A priority Critical patent/EP3640674B1/en
Priority to PCT/CN2018/103362 priority patent/WO2020042121A1/zh
Priority to CN201880038905.4A priority patent/CN110799927B/zh
Publication of WO2020042121A1 publication Critical patent/WO2020042121A1/zh
Priority to US16/826,073 priority patent/US11061115B2/en

Links

Images

Classifications

    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S7/00—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00
    • G01S7/02—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00 of systems according to group G01S13/00
    • G01S7/41—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00 of systems according to group G01S13/00 using analysis of echo signal for target characterisation; Target signature; Target cross-section
    • G01S7/415—Identification of targets based on measurements of movement associated with the target
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/017—Gesture based interaction, e.g. based on a set of recognized hand gestures
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S13/00—Systems using the reflection or reradiation of radio waves, e.g. radar systems; Analogous systems using reflection or reradiation of waves whose nature or wavelength is irrelevant or unspecified
    • G01S13/02—Systems using reflection of radio waves, e.g. primary radar systems; Analogous systems
    • G01S13/06—Systems determining position data of a target
    • G01S13/42—Simultaneous measurement of distance and other co-ordinates
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S13/00—Systems using the reflection or reradiation of radio waves, e.g. radar systems; Analogous systems using reflection or reradiation of waves whose nature or wavelength is irrelevant or unspecified
    • G01S13/02—Systems using reflection of radio waves, e.g. primary radar systems; Analogous systems
    • G01S13/50—Systems of measurement based on relative movement of target
    • G01S13/58—Velocity or trajectory determination systems; Sense-of-movement determination systems
    • G01S13/583—Velocity or trajectory determination systems; Sense-of-movement determination systems using transmission of continuous unmodulated waves, amplitude-, frequency-, or phase-modulated waves and based upon the Doppler effect resulting from movement of targets
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S13/00—Systems using the reflection or reradiation of radio waves, e.g. radar systems; Analogous systems using reflection or reradiation of waves whose nature or wavelength is irrelevant or unspecified
    • G01S13/02—Systems using reflection of radio waves, e.g. primary radar systems; Analogous systems
    • G01S13/50—Systems of measurement based on relative movement of target
    • G01S13/58—Velocity or trajectory determination systems; Sense-of-movement determination systems
    • G01S13/583—Velocity or trajectory determination systems; Sense-of-movement determination systems using transmission of continuous unmodulated waves, amplitude-, frequency-, or phase-modulated waves and based upon the Doppler effect resulting from movement of targets
    • G01S13/584—Velocity or trajectory determination systems; Sense-of-movement determination systems using transmission of continuous unmodulated waves, amplitude-, frequency-, or phase-modulated waves and based upon the Doppler effect resulting from movement of targets adapted for simultaneous range and velocity measurements
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S13/00—Systems using the reflection or reradiation of radio waves, e.g. radar systems; Analogous systems using reflection or reradiation of waves whose nature or wavelength is irrelevant or unspecified
    • G01S13/88—Radar or analogous systems specially adapted for specific applications
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S7/00—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00
    • G01S7/02—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00 of systems according to group G01S13/00
    • G01S7/35—Details of non-pulse systems
    • G01S7/352—Receivers
    • G01S7/354—Extracting wanted echo-signals
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S7/00—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00
    • G01S7/02—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00 of systems according to group G01S13/00
    • G01S7/41—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00 of systems according to group G01S13/00 using analysis of echo signal for target characterisation; Target signature; Target cross-section
    • G01S7/414—Discriminating targets with respect to background clutter
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2203/00—Indexing scheme relating to G06F3/00 - G06F3/048
    • G06F2203/01—Indexing scheme relating to G06F3/01
    • G06F2203/011—Emotion or mood input determined on the basis of sensed human body parameters such as pulse, heart rate or beat, temperature of skin, facial expressions, iris, voice pitch, brain activity patterns

Definitions

  • the present application relates to the field of electronic applications, and in particular, to a gesture recognition method, a terminal, and a storage medium.
  • gesture recognition solutions include sonic gesture recognition and gesture recognition based on visible light camera image analysis. The existing gesture recognition solutions have the problem of low accuracy of gesture recognition.
  • the terminal Take the sonic gesture recognition scheme as an example.
  • the terminal reconstructs the gesture action according to the ultrasonic signals generated by the user's gesture action.
  • the accuracy of the gesture recognition scheme of the sonic gesture recognition scheme in a noisy environment will be greatly reduced.
  • the terminal reconstructs gesture actions based on multi-angle gesture images collected through the camera.
  • the accuracy of gesture recognition in the scheme in low-light or no-light environments Will be lower.
  • the embodiments of the present application are expected to provide a gesture recognition method, a terminal, and a storage medium, which can improve the accuracy of gesture recognition.
  • An embodiment of the present application provides a gesture recognition method, which is applied to a terminal.
  • the terminal is provided with a millimeter wave device, and the method includes:
  • the first millimeter wave is processed to obtain at least one set of signal characteristic values corresponding to the first millimeter wave, and each of the at least one set of signal characteristic values
  • the signal characteristic value corresponds to a frame signal in the first millimeter wave
  • the first control instruction is used to control the first application to implement a corresponding function.
  • An embodiment of the present application provides a terminal.
  • the terminal includes a processor, a receiver, a memory, and a communication bus.
  • the terminal is provided with a millimeter wave device, and the receiver is configured to receive a gesture action and return via the millimeter wave device.
  • the first millimeter wave, the first millimeter wave is a second millimeter wave emitted by the millimeter wave device and modulated by a gesture action;
  • the processor is configured to execute an operation program stored in the memory to Implement the following steps:
  • the first millimeter wave is processed to obtain at least one set of signal characteristic values corresponding to the first millimeter wave, and each of the at least one set of signal characteristic values
  • the signal characteristic value corresponds to a frame signal in the first millimeter wave
  • the standard characteristic value and control instruction correspondence database is used to identify the at least one set of signal characteristic values to obtain a first control instruction corresponding to the gesture action Using the first control instruction to control the first application to implement a corresponding function.
  • An embodiment of the present application provides a storage medium on which a computer program is stored, which is applied to a terminal, and when the computer program is executed by a processor, implements any one of the gesture recognition methods described above.
  • An embodiment of the present application provides a gesture recognition method, a terminal, and a storage medium.
  • the method includes: receiving a first millimeter wave through a millimeter wave device, where the first millimeter wave is a second millimeter wave transmitted by the millimeter wave device and modulated by a gesture action. Reflected waves; based on two types of time array and Doppler estimation, the first millimeter wave is processed to obtain at least one set of signal characteristic values corresponding to the first millimeter wave, and each set of signal characteristics in at least one set of signal characteristic values The value corresponds to a frame signal in the first millimeter wave.
  • the standard feature value and control instruction correspondence database is used to identify at least one set of signal characteristic values to obtain a first control instruction corresponding to a gesture action.
  • the first control instruction is used to control The first application implements the corresponding function.
  • the terminal receives the first millimeter wave modulated by the gesture action through the millimeter wave device, and processes the first millimeter wave based on the two types of time arrays based on the characteristics of the small wavelength of the first millimeter wave, and uses Doppler estimation To obtain at least one set of signal characteristic values corresponding to the processed first millimeter wave, and finally use at least one set of signal characteristic values and standard characteristic value and control command correspondence database to obtain a first control command corresponding to a gesture action, thereby being able to Recognize subtle gestures, which improves the accuracy of gesture perception.
  • FIG. 1 is a first flowchart of a gesture recognition method according to an embodiment of the present application
  • FIG. 2 is a structural composition diagram of an exemplary terminal according to an embodiment of the present application.
  • FIG. 3 is a display diagram of an exemplary feature value corresponding to a frame signal according to an embodiment of the present application.
  • FIG. 4 is an architecture diagram of an exemplary gesture control according to an embodiment of the present application.
  • FIG. 5 is a second flowchart of a gesture recognition method according to an embodiment of the present application.
  • FIG. 6 is a schematic diagram of an exemplary FM continuous wave according to an embodiment of the present application.
  • FIG. 7 is a schematic diagram of an exemplary Doppler frequency shift provided by an embodiment of the present application.
  • FIG. 8 is a schematic diagram of an exemplary convolutional neural network model according to an embodiment of the present application.
  • FIG. 9 is an exemplary gesture recognition flowchart based on a Matlab program according to an embodiment of the present application.
  • FIG. 10 is a schematic diagram of an exemplary gesture action provided by an embodiment of the present application.
  • FIG. 11 is a schematic structural diagram of a terminal according to an embodiment of the present application.
  • Millimeter wave refers to the frequency band of 30-300 GHz. Because the available bandwidth of this frequency band is very sufficient, the transmission rate of data transmission in the millimeter wave band is very large; millimeter wave is the first because of its large bandwidth and high rate.
  • the communication frequency band used by the fifth generation wireless communication technology (5G, 5th-Generation) uses millimeter wave for data transmission, which can greatly increase the wireless network rate.
  • IEEE802.11ad running in the 60GHz band supports data transmission rates up to 6.7Gbps, while its evolutionary standard IEEE802.11ay will provide data transmission rates of 20Gbps. Therefore, millimeter-wave radio is expected to bring wireless network access into the multi-Gbps era.
  • the millimeter-wave radio module will be widely installed on mobile phones, wearables, smart hardware or a wider range of IoT devices, becoming a mainstream communication technology.
  • the short wavelength, large bandwidth, and directional beam of millimeter wave also make high-resolution, highly robust human gesture perception possible.
  • the millimeter wave technology perception technology can provide a smarter, more convenient and interesting human-computer interaction application experience.
  • the basic principle is to use millimeter-wave radio frequency modules to transmit millimeter waves, and the receiving module receives the reflected waves of gestures, and uses the reflected waves to estimate the distance, angle, speed, and energy in the gesture process to classify the actions.
  • the millimeter wave supports a variety of sensing functions such as distance measurement, gesture detection, proximity detection, number of people detection, distance measurement, and presence detection. It can be applied in the following scenarios:
  • the user can reduce the ringing volume by using certain gestures (such as approaching a mobile phone) until it is muted.
  • a series of gestures can be used to "tell" the mobile phone when to take pictures, adjust the lightness and darkness, and adjust the focal length, etc., thereby eliminating the inconvenient operation of touching the screen of the mobile phone.
  • buttons operations such as sliding pages, adjusting video volume and brightness, switching music, and switching camera filters.
  • the following is a scene of gesture recognition using millimeter waves in a photographing scene according to an embodiment of the present application.
  • a gesture recognition method provided in an embodiment of the present application is applied to a terminal, and the terminal is provided with a millimeter wave device. As shown in FIG. 1, the method may include:
  • S101 Receive a first millimeter wave through a millimeter wave device, where the first millimeter wave is a reflected wave modulated by a gesture action of a second millimeter wave transmitted by the millimeter wave device.
  • a gesture recognition method provided in the embodiment of the present application is applied to a scenario in which a user's gesture is sensed to implement non-contact photographing.
  • the terminal may be any device having communication and storage functions, such as a tablet computer, a mobile phone, an e-reader, a remote controller, a personal computer (PC), a notebook computer, a vehicle-mounted device, and a network television.
  • a tablet computer such as a tablet computer, a mobile phone, an e-reader, a remote controller, a personal computer (PC), a notebook computer, a vehicle-mounted device, and a network television.
  • PC personal computer
  • notebook computer a vehicle-mounted device
  • network television a network television.
  • wearable devices are specifically selected according to actual conditions, and are not specifically limited in the embodiments of the present application.
  • a millimeter wave device is provided inside the screen of the terminal, and the millimeter wave device includes a transmitting antenna and a receiving antenna.
  • the millimeter wave radio can penetrate non-metallic materials such as plastic, the millimeter wave device is concealed and deployed inside the terminal screen, which does not change the appearance of the terminal, which is of great significance to the appearance design of the terminal.
  • the terminal transmits a wireless signal (second millimeter wave) through a transmitting antenna of the millimeter wave device, and after the hand signal is modulated in the transmission range of the wireless signal, a reflected signal (first millimeter wave) is formed, and then the reflected signal Captured by the receiving antenna of the millimeter wave device.
  • a wireless signal second millimeter wave
  • first millimeter wave reflected signal
  • the terminal may transmit a wireless signal through a transmitting antenna of the millimeter wave device when a preset transmission time arrives, or may transmit through a millimeter wave device when starting a first application such as a photographing application or a video photographing application.
  • the antenna transmits a wireless signal, and the timing for a specific terminal to transmit the wireless signal through the transmitting antenna of the millimeter wave device is selected according to the actual situation, which is not specifically limited in this embodiment of the present application.
  • the form of the wireless signal transmitted by the millimeter wave device is periodically transmitting FMCW (Frequency Modulated Continuous Wave), so that the first millimeter wave frequency and the second millimeter wave have the same change law, which are both triangle wave rules.
  • FMCW Frequency Modulated Continuous Wave
  • the terminal can calculate the target distance by using this small time difference.
  • a digital signal processor (DSP, Digital Signal Processing) is provided on the terminal.
  • the DSP includes a distance processing module, a Capon beam former, and an object detection unit.
  • Doppler estimation unit Doppler Estimation
  • the distance processing module After the receiving antenna receives the reflected wave, the reflected wave is buffered to the analog-to-digital converter (ADC, Analog-to-Digital Converter) output buffer area, and then the millimeter wave device moves the reflected wave from the ADC output buffer area to In the local memory of the DSP, at this time, the distance processing module performs a 16-bit fixed-point 1-D window and a 16-bit fixed-point 1-D fast Fourier transform (FFT), and transmits the execution result to the Doppler estimation unit.
  • ADC Analog-to-Digital Converter
  • Capon Beamformer Use Equation (1) to reconstruct the source signal from the sensor array
  • Target detection unit The constant false-alarm (RAR) detection algorithm is used to process the first channel in the distance domain and the second channel in the angular domain, and the second channel adds the results of the first channel. Confirm to remove clutter and noise, and determine the detection point.
  • RAR constant false-alarm
  • Doppler estimation For each [distance, azimuth] pair, use the Capon beam weighting algorithm to filter the distance receiver, and then perform a peak search on the filtered distance receiver FFT to estimate Doppler.
  • the first millimeter wave is processed to obtain at least one set of signal characteristic values corresponding to the first millimeter wave, and each set of signal characteristic values in the at least one set of signal characteristic values corresponds to One frame signal in the first millimeter wave.
  • the terminal After the terminal receives the first millimeter wave through the millimeter wave device, the terminal processes the first millimeter wave to obtain at least one set of signal characteristic values corresponding to the first millimeter wave.
  • the terminal processes the first millimeter wave based on two types of time arrays to obtain a motion feature corresponding to a gesture action, where the motion feature represents displacement information of the gesture action; Thereafter, the terminal extracts at least one set of signal feature values from the motion features based on the Doppler estimation, wherein each set of signal feature values of the at least one set of signal feature values corresponds to a frame of signals in the motion feature representation.
  • the two types of time arrays include a fast time array and a slow time array.
  • the terminal processes the first millimeter wave into at least one beam, where each beam in the at least one beam corresponds to a received time point.
  • the first millimeter wave the terminal acquires at least one first information corresponding to at least one beam in the fast time array, and the at least one first information characterizes at least one frequency corresponding to the at least one beam; thereafter, the terminal in the slow time array, according to
  • the at least one first information determines the second information, and the second information represents a frequency change between the at least one beam; the second information is determined as an action characteristic.
  • the terminal when the terminal processes the first millimeter wave into at least one beam corresponding to each receiving time point, the terminal calculates a frequency corresponding to each beam in the at least one beam in a fast time array. In the slow time array, a frequency change between at least one beam is calculated according to a frequency corresponding to each beam in the at least one beam.
  • the frequency change represents displacement information of a gesture action, and the terminal determines the frequency change as a gesture action. feature.
  • the basic principle of terminal recognition of different hand movements is: assume the hand as a discrete dynamic scattering center, model the radio frequency (RF, Radio Frequency) response of the hand as a superposition of the response from the discrete dynamic scattering center, When the wavelength is smaller than the spatial range of the target, the scattering center model is consistent with the set theory of diffraction. Due to the short wavelength characteristic of the millimeter wave, the above assumption applies to millimeter wave hand motion perception.
  • RF Radio Frequency
  • This solution uses a generalized time-varying scattering center model and considers non-rigid hand dynamics, that is, each scattering center is parameterized by the composite reflectance parameter and radial distance from the sensor, and the composite reflectance parameter is frequency-dependent It varies with the local geometry of the hand, the direction relative to the radar, etc. Therefore, this application uses high time resolution sensing, that is, measures the response of the hand to the radar through a high frame rate, and then extracts subtle time signal changes corresponding to these hand movements to detect subtle and complex hand movements.
  • the terminal controls the millimeter wave device to send periodic modulated waveforms to implement the above concept.
  • the millimeter wave radar measures the corresponding received waveforms in each transmission cycle. Therefore, in order to implement the above solution, the present application defines two different time scales to analyze the reflected first millimeter wave, which are short-time scale perception and long-time scale perception, respectively.
  • the terminal uses short-term scale sensing in a fast time array, and uses long-term scale sensing in a slow time array.
  • the principle of short-term sensing is that the high radar repetition frequency associates the scattering center hand model with the signal processing method.
  • the scattering center model is within a single radar repetition interval.
  • the scattering center range and reflectance are functions that closely follow the short-term scale T.
  • the measurement waveform is composed of the reflections of each scattering center and is superimposed in fast time. Each individual reflection waveform has related scattering. The instantaneous reflectance and range modulation of the center.
  • the preprocessed received signal represents the superposition of the response from each scattering center.
  • the high radar repetition frequency can capture the fine phase change in the received signal corresponding to the dynamics of the scattering center in a slow time.
  • the principle of long-term sensing is that when the scattering center moves, the relative displacement of the scattering center will produce a phase change proportional to the wavelength.
  • the dependence of the phase change on the displacement allows the millimeter wave device to find the scattered scattering center in slow time based on its phase. Assuming that the speed of each scattering center is approximately constant over some coherent processing time that is greater than the radar repetition interval, the phase in the coherent processing time will generate a Doppler frequency, so each fast time window on the slow time window of coherent processing can be calculated
  • the spectrum of the waveform is used to resolve the Doppler frequencies of multiple scattering centers moving at different speeds.
  • the terminal processes the first millimeter wave into a motion feature corresponding to a gesture motion through short-term scale sensing and long-term scale sensing.
  • a certain number of continuous pre-processed radar signals are buffered in the fast time array and the slow time array, which are used to characterize motion characteristics.
  • each frame signal is composed of at least 11 feature values, as shown in FIG. 3.
  • the 11 feature values include: digital detection (num_detection), Doppler average (Doppler_average), and distance average (range_average). , Amplitude sum (magnitude_sum), active digital detection (positive num_detetion), distance index (range_index), inactive digital detection (negative num_detection), inactive Doppler mean (negtaive, doppler_average), distance display (range_disp), angle value ( angle_value), prediction result (predication_result).
  • the speed and Doppler frequency shift of the gesture motion are calculated using the Doppler effect, and the distance from the terminal to the gesture motion is calculated using the FMCW principle.
  • S103 Use a database of correspondences between standard feature values and control instructions to identify at least one set of signal feature values to obtain a first control instruction corresponding to a gesture action.
  • the terminal After the terminal obtains at least one set of signal characteristic values, the terminal recognizes at least one set of signal characteristic values by using a library of standard characteristic value and control instruction correspondences to obtain a first control instruction corresponding to a gesture action.
  • the correspondence database of standard feature values and control instructions is a relationship database obtained through preset neural networks.
  • the terminal uses the preset neural network to learn standard gesture actions to obtain at least one control instruction corresponding to at least A set of standard feature values.
  • the terminal composes the control instruction and the corresponding at least one set of standard feature values to form a library of correspondences between the standard feature values and the control instructions.
  • the terminal After the terminal obtains at least one set of signal feature values corresponding to the first millimeter wave, the terminal starts from In the correspondence database of standard characteristic values and control instructions, a first control instruction corresponding to at least one set of signal characteristic values is searched.
  • the preset neural network is a 6-layer residual network obtained after removing the last three layers of the residual network resnet18.
  • the terminal after receiving a standard gesture action corresponding to each control instruction, processes the standard gesture action to obtain a set of frame sequence signals (standard frame signals) corresponding to the standard gesture action, where a group of frames Each frame of the sequence signal corresponds to a set of feature values (a preset number of standard signal feature values), and the terminal inputs at least a set of feature values corresponding to a set of frame sequence signals into the 6-layer residual network, and uses the The 6-layer residual network learns at least one set of feature values to obtain the standard feature value group corresponding to each control instruction, and saves the control instruction and the corresponding standard feature value group as a trained network model in the format of .pkl,
  • a python script is called to import the trained network model. This script is called by the Matlab program. After the python script classifies and predicts at least one set of signal feature values, it returns the predicted classification results to Matlab program.
  • the terminal matches at least one set of signal feature values and standard feature values with a control instruction correspondence relationship database.
  • the terminal searches for the first control instruction corresponding to the first standard feature value group from the standard feature value and control instruction correspondence database.
  • the terminal obtains the gesture by using the standard feature value and control instruction correspondence database. The first control instruction corresponding to the action.
  • the first control instruction is used to control the camera to implement functions such as photographing and focusing, and is specifically selected according to actual conditions, which is not specifically limited in the embodiment of the present application.
  • the terminal when the terminal receives the initial state that the fingers of the right hand are naturally opened, the right arm forearm is lifted forward, and then the elbow joint of the right arm drives the forearm to be leveled with the elbow joint as the axial left side.
  • the terminal determines that the first control instruction is to control the camera to take a picture.
  • the terminal After the terminal obtains the first control instruction corresponding to the gesture action, the terminal uses the first control instruction to control the first application to implement a corresponding function.
  • the terminal after the terminal obtains the first control instruction, the terminal inputs the first control instruction into the Matlab program, and uses the Matlab program to complete the function of controlling the camera. Specifically, the terminal uses the Matlab program to complete the function of controlling the camera in the following manner: the terminal calls the Webcam module through the Matlab program. After the terminal obtains the first control instruction, the Matlab program sends a control value corresponding to the first control instruction to the Webcam module; after receiving the control value, the Webcam module controls the camera to implement different functions according to different control values.
  • Matlab programs runs through the entire system.
  • the Matlab program is used to store the signals collected by the millimeter wave device, call the prediction script of python for prediction, and control the camera after obtaining the predicted value.
  • the general architecture of gesture control is: a millimeter wave device receives a raw signal modulated by a standard gesture motion, processes the raw signal to obtain at least one set of feature values, and then, at least one set of features The value is input to the neural network analysis. After the prediction of the neural network, the camera is controlled to complete the corresponding function.
  • the terminal receives the first millimeter wave returned by the gesture through a millimeter wave device, and processes the first millimeter wave based on two types of time arrays based on the small wavelength of the first millimeter wave, and uses Doppler estimation To obtain at least one set of signal characteristic values corresponding to the processed first millimeter wave, and finally use the at least one set of signal characteristic values and a preset neural network to obtain a first control instruction corresponding to a gesture action, which can identify subtle gesture actions, thereby Improved the accuracy of gesture perception.
  • An embodiment of the present application provides a gesture recognition method, which is applied to a terminal, and a millimeter wave device is provided on the terminal. As shown in FIG. 5, the method may include:
  • the terminal receives the reflected signal through a millimeter wave device.
  • a gesture recognition method provided in the embodiment of the present application is applied to a scenario in which a user's gesture is sensed to implement non-contact photographing.
  • the terminal may transmit the wireless signal through the transmitting antenna of the millimeter wave device when the preset transmission time arrives, or may transmit the wireless signal through the transmitting antenna of the millimeter wave device when the first application is started.
  • the specific terminal The timing of transmitting wireless signals through the transmitting antenna of the millimeter wave device is selected according to the actual situation, which is not specifically limited in the embodiment of the present application.
  • the first application is a photographing application or a video capturing application, and is specifically selected according to an actual situation, and the embodiment of the present application is not specifically limited.
  • the terminal when a user clicks an application icon of a first application on an application icon display interface, the terminal receives a startup instruction for starting the first application. At this time, the terminal starts the first application and transmits a wireless signal using a millimeter wave device. .
  • the millimeter-wave device includes a transmitting antenna and a receiving antenna.
  • the millimeter-wave device uses the transmitting antenna to transmit a wireless signal. After the hand signal is modulated in the transmission range of the wireless signal, a reflected signal is formed, and then the reflected signal is transmitted by the millimeter-wave device. Receiving antenna capture.
  • the form of the wireless signal transmitted by the millimeter wave device is to periodically transmit FMCW waves, so that the frequency of the first millimeter wave and the second millimeter wave change the same, both are triangle wave laws, but there is a time difference, and the terminal can use This small time difference can calculate the target distance.
  • the terminal uses a beam synthesis algorithm to synthesize the reflected signal into a reflected wave.
  • the terminal After the terminal receives the reflected signal through the millimeter wave device, the terminal uses a beam combining algorithm to synthesize the reflected signal into a reflected wave.
  • the reflected wave is buffered into the ADC output buffer area, and then the millimeter wave device moves the reflected wave from the ADC output buffer area to the local memory of the DSP, and the terminal uses Capon beam synthesis
  • the reflector synthesizes the reflected signal into a reflected wave.
  • the formula (1) is used to reconstruct the source signal from the sensor array.
  • static debris is removed by removing the DC component of each distance receiver in the distance processing module, thereby eliminating reflections of static objects such as tables and chairs in the region of interest.
  • the terminal removes the clutter signal and the noise signal of the reflected wave to obtain a first millimeter wave.
  • the first millimeter wave is a second millimeter wave transmitted by the millimeter wave device and modulated by a gesture action.
  • the terminal After the terminal combines the reflected signal with the reflected wave, the terminal removes the clutter signal and the noise signal from the reflected wave, thereby obtaining the first millimeter wave.
  • a Constant False Alarm Rate (CFAR) detection algorithm is used to process the first channel in the distance domain and the second channel in the angular domain. The result is confirmed, thereby removing clutter and noise, and determining a detection point, thereby obtaining a first millimeter wave.
  • CFAR Constant False Alarm Rate
  • the terminal processes the first millimeter wave into at least one beam, and each beam in the at least one beam corresponds to the first millimeter wave received at a receiving time point.
  • the terminal After the terminal obtains the first millimeter wave, the terminal processes the first millimeter wave into at least one beam, where each beam in the at least one beam corresponds to the first millimeter wave received at a receiving time point.
  • the terminal divides the first millimeter wave into at least one beam corresponding to at least one receiving time point.
  • the terminal acquires at least one first information corresponding to the at least one beam, and the at least one first information represents at least one frequency corresponding to the at least one beam.
  • the terminal After the terminal processes the first millimeter wave into at least one beam, the terminal acquires at least one first information corresponding to the at least one beam in the fast time array.
  • the terminal calculates at least one first information corresponding to at least one beam based on a short-term scale sensing principle, wherein the at least one first information is used to characterize at least one frequency corresponding to at least one beam.
  • the principle of short-term sensing is that the high radar repetition frequency associates the scattering center hand model with the signal processing method.
  • the scattering center model is within a single radar repetition interval Approximately constant, the scattering center range and reflectance are functions that closely follow the short-term scale T.
  • the hand In each transmission period, the hand is illuminated with a single wide beam, and all scattering centers on the hand reflect the signal at the same time.
  • the measurement waveform is composed of the reflections of each scattering center and is superimposed in fast time. Each individual reflection waveform has related scattering. The instantaneous reflectance and range modulation of the center.
  • the preprocessed received signal represents the superposition of the response from each scattering center.
  • the high radar repetition frequency can capture the fine phase change in the received signal corresponding to the dynamics of the scattering center in a slow time.
  • the terminal determines the second information according to the at least one first information, and the second information represents a frequency change between the at least one beam.
  • the terminal After the terminal obtains at least one first information corresponding to at least one beam in the fast time array, the terminal determines the second information according to the at least one first information in the slow time array, wherein the second information represents at least one Frequency variation between beams.
  • the terminal calculates the second information characterizing the frequency change between at least one beam based on the long-term scale sensing principle.
  • the principle of long-term sensing is that when the scattering center moves, the relative displacement of the scattering center will produce a phase change proportional to the wavelength.
  • the dependence of the phase change on the displacement allows the millimeter wave device to find the scattered scattering center in slow time based on its phase. Assuming that the speed of each scattering center is approximately constant over some coherent processing time that is greater than the radar repetition interval, the phase in the coherent processing time will generate a Doppler frequency, so you can calculate each fast time window on the coherent processing slow time window
  • the spectrum of the waveform is used to resolve the Doppler frequencies of multiple scattering centers moving at different speeds.
  • the terminal applies FFT to each fast time array on the slow time array to obtain frequency information.
  • the resulting fast time-frequency map is transformed into distance and speed by transformation. Fine adjustments can be made for desired hand dynamics and desired sensing performance based on SNR, speed resolution, and Doppler aliasing. Thereby, frequency change information of at least one beam is determined, and the frequency change information represents the distance and rate of the multi-center of the hand over time.
  • S207 The terminal determines the second information as an action feature.
  • the terminal determines the second information as an action characteristic corresponding to the first millimeter wave.
  • the terminal uses the distance and rate of the multi-center of the hand as a function of time.
  • the terminal buffers a certain number of continuously pre-processed radar signals in the fast time array and the slow time array, which are used to characterize the motion characteristics.
  • the terminal uses the Doppler effect to sequentially extract speed information and Doppler frequency shift information corresponding to a frame signal from the motion characteristics.
  • the terminal uses the Doppler effect to extract the speed information and Doppler frequency shift information corresponding to a frame of signal from the motion feature.
  • the terminal determines the motion characteristic as at least one frame signal, and then sequentially processes one frame signal of the at least one frame signal by using the Doppler effect to sequentially extract corresponding speed information from the one frame signal. And Doppler frequency shift information.
  • the Doppler effect means that the wavelength of an object's radiation changes due to the relative movement of the light source and the observer.
  • the wave In front of the moving wave source, the wave is compressed, the wavelength becomes shorter, and the frequency becomes higher. Behind the moving wave source, the opposite effect occurs, the wavelength becomes longer, and the frequency becomes lower. The higher the speed of the wave source, the greater the effect. According to the degree of red / blue shift of the light wave, the speed information of the wave source moving in the observation direction can be calculated.
  • the millimeter wave device since the millimeter wave device transmits a frequency-modulated continuous wave, the frequency variation rules of the second millimeter wave and the first millimeter wave both conform to the triangle wave law. Therefore, according to the Doppler effect, the frequency difference is shown in FIG. 6 In the frequency-time coordinate, the solid line is the frequency curve of the transmitted wave, and the dashed line is the frequency curve of the received wave, where f b is the frequency difference when the detected object is stationary, and f d is the Doppler when the detected object is moving. Frequency shift.
  • the terminal uses the FM continuous wave principle to sequentially extract distance information corresponding to a frame of signals from the motion characteristics.
  • the terminal uses the FM continuous wave principle to sequentially extract distance information corresponding to a frame of signal from the motion feature.
  • the terminal determines the motion characteristic as at least one frame signal, and then uses the FM continuous wave principle to sequentially process one frame signal in the at least one frame signal to sequentially extract the corresponding distance from the one frame signal. information.
  • the millimeter wave device sends the millimeter wave form to calculate the distance between the targets.
  • the basic principle is that the transmitted wave is a high-frequency continuous wave, and its frequency changes with time according to the law of triangular waves.
  • the frequency of the echo received by the radar is the same as that of the transmitted frequency, both of which are triangle wave laws, but there is a time difference.
  • the distance information can be calculated by using this small time difference.
  • the dashed line is the frequency curve of the transmitted wave
  • the solid line is the frequency curve of the received wave
  • td is the time difference between ft and fr
  • ft is the frequency of the transmitted wave
  • fr is the frequency of the received wave
  • S208 and S209 are two parallel steps after S207, and the specific execution order is selected according to the actual situation, which is not specifically limited in this embodiment of the present application.
  • the terminal determines at least speed information, Doppler frequency shift information, and distance information as a set of signal characteristic values corresponding to one frame of signals.
  • the terminal After the terminal extracts the speed information, Doppler frequency shift information, and distance information, the terminal determines at least the speed information, Doppler frequency shift information, and distance information as a set of signal characteristic values corresponding to one frame of signals.
  • a frame signal includes a set of signal characteristic values related to speed information, Doppler frequency shift information, and distance information.
  • each frame signal is composed of at least 11 characteristic values related to speed information, Doppler frequency shift information, and distance information.
  • the 11 characteristic values include: num_detection , Doppler_average, range_average, amplitude_sum, active num_detetion, range index, negative num_detection, inactive number negtaive (doppler_average), distance display (range_disp), angle value (angle_value), prediction result (predication_result).
  • the terminal composes each set of signal characteristic values corresponding to each frame of signals into at least one set of signal characteristic values corresponding to at least one frame of signals.
  • the terminal After the terminal determines a set of signal characteristic values corresponding to a frame signal, the terminal forms each set of signal characteristic values corresponding to each frame signal to form at least one set of signal characteristic values corresponding to at least one frame signal.
  • the terminal determines each set of signal feature values corresponding to each frame signal in turn, and then forms each set of signal feature values corresponding to each frame signal into at least one set of signal features corresponding to at least one frame signal. value.
  • the terminal uses a library of correspondences between standard feature values and control instructions to identify at least one set of signal feature values to obtain a first control instruction corresponding to a gesture action.
  • the terminal After the terminal obtains the correspondence database of standard feature values and control instructions, the terminal performs classification prediction on at least one set of signal feature values by using the correspondence database of standard feature values and control instructions.
  • the terminal searches for a first standard signal characteristic value corresponding to at least one set of signal characteristic values from a standard relationship database of standard characteristic values and control instructions, and determines a first control instruction corresponding to the first standard signal characteristic value.
  • the database of correspondences between standard feature values and control instructions is a relationship database obtained through preset neural networks.
  • the terminal uses a Matlab program to call a python script to import a trained network model (a library of correspondences between standard feature values and control instructions).
  • the python script returns the predicted classification result value to the Matlab program after performing classification prediction on at least one set of signal feature values.
  • the terminal uses the first control instruction to control the first application to implement a corresponding function.
  • the terminal After the terminal obtains the first control instruction, the terminal uses the first control instruction to control the first application to implement a corresponding function.
  • the terminal after the terminal obtains the first control instruction, the terminal inputs the first control instruction into the Matlab program, and uses the Matlab program to complete the function of controlling the camera. Specifically, the terminal uses the Matlab program to complete the function of controlling the camera in the following manner: the terminal calls the Webcam module through the Matlab program. After the terminal obtains the first control instruction, the Matlab program sends a control value corresponding to the first control instruction to the Webcam module; after receiving the control value, the Webcam module controls the camera to implement different functions according to different control values.
  • Matlab programs runs through the entire system.
  • the Matlab program is used to store the signals collected by the millimeter wave device, call the prediction script of python for prediction, and control the camera after obtaining the predicted value.
  • the use of the Matlab program runs through the entire system process.
  • the millimeter wave device transmits the collected signals to the Matlab program, and the signals collected by the millimeter wave device are stored in the Matlab program and called.
  • Python's prediction script performs predictions and controls the camera after obtaining the predicted values.
  • the terminal receives the first millimeter wave returned by the gesture through a millimeter wave device, and processes the first millimeter wave based on two types of time arrays based on the small wavelength of the first millimeter wave, and uses Doppler estimation To obtain at least one set of signal characteristic values corresponding to the processed first millimeter wave, and finally use the at least one set of signal characteristic values and a preset neural network to obtain a first control instruction corresponding to a gesture action, which can identify subtle gesture actions, thereby Improved the accuracy of gesture perception.
  • the terminal when the terminal performs gesture recognition, it also uses a preset neural network to learn in real time to obtain a library of correspondences between standard feature values and control instructions.
  • the method for terminal to perform gesture recognition may further include: The following steps:
  • the terminal obtains a preset number of standard frame signals corresponding to a standard gesture action.
  • the terminal determines a period of time required for acquisition of a standard gesture, and then determines the number of standard frame signals corresponding to the period of time.
  • a standard gesture action needs to be collected for 2 seconds, and the number of standard frame signals that can be collected in 2 seconds is 60.
  • the terminal acquires four standard gestures that control the camera to implement different functions, as shown in FIG. 10.
  • Gesture 1-Focusing Bend the right middle finger, ring finger, and little finger so that the tip of the finger touches the palm of the hand, extend the forefinger and thumb and enclose the two into an ellipse. Bevel, move the wrist up and down to drive the right hand up and down twice.
  • Gesture 3-Zoom out Fingers open naturally, hands move forward towards the millimeter wave device, and fists are held during the movement.
  • the terminal determines a preset number of standard signal feature values corresponding to a preset number of standard frame signals.
  • the terminal After the terminal obtains a preset number of standard frame signals corresponding to the standard gesture action, the terminal determines a preset number of standard signal feature values corresponding to the preset number of standard frame signals.
  • the terminal processes a standard gesture action to obtain a set of frame sequence signals corresponding to the standard gesture action, wherein each frame sequence signal in the set of frame sequence signals corresponds to a set of feature values.
  • each frame sequence signal contains 11 feature values related to angle, distance, and Doppler frequency shift.
  • the terminal uses a preset neural network to learn a preset number of standard signal feature values, and obtains a database of correspondences between the standard feature values and the control instructions.
  • the terminal After the terminal determines the preset number of standard signal feature values corresponding to the preset number of standard frame signals, the terminal uses the preset neural network to learn the preset number of standard signal feature values to obtain the standard feature value corresponding to the control instruction. Relationship library.
  • the preset neural network is a 6-layer residual network obtained after removing the last three layers of the residual network resnet18.
  • the terminal inputs at least one set of feature values corresponding to a set of frame sequence signals into the 6-layer residual network, and uses the 6-layer residual network to learn at least one set of feature values to obtain each control instruction.
  • the corresponding standard feature value group, and the control instruction and the corresponding standard feature value group are saved as a trained network model in the format of .pkl.
  • FIG. 11 is a schematic structural diagram of a terminal according to an embodiment of the present application.
  • a millimeter wave device is provided on the terminal.
  • the terminal 1 in the embodiment of the application includes: a processor 10, a receiver 11, a memory 12, and a communication bus 13.
  • the processor 10 may be an application-specific integrated circuit (ASIC, Application Specific Integrated Circuit), a digital signal processor (DSP, Digital Signal Processor), or a digital signal processing terminal (DSPD, Digital Signal Processing).
  • ASIC Application-specific integrated circuit
  • DSP Digital Signal Processor
  • DSPD Digital Signal Processing terminal
  • At least one of a device PLC
  • PLC programmable logic terminal
  • FPGA field programmable gate array
  • CPU central processing unit
  • controller a microcontroller
  • microprocessor a microprocessor
  • the communication bus 13 is used to implement connection and communication between the processor 10, the receiver 11, and the memory 12.
  • the receiver 11 is configured to receive a first millimeter wave through the millimeter wave device.
  • the first millimeter wave is a reflected wave modulated by a gesture motion of a second millimeter wave transmitted by the millimeter wave device;
  • the processor 10 is configured to execute a running program stored in the memory 12 to implement the following steps:
  • the first millimeter wave is processed to obtain at least one set of signal characteristic values corresponding to the first millimeter wave, and each of the at least one set of signal characteristic values
  • the signal characteristic value corresponds to a frame signal in the first millimeter wave
  • the standard characteristic value and control instruction correspondence database is used to identify the at least one set of signal characteristic values to obtain a first control instruction corresponding to the gesture action ; Using the first control instruction to control the first application to implement a corresponding function.
  • the processor 10 is further configured to process the first millimeter wave based on the two types of time arrays to obtain an action feature corresponding to the gesture action, and the action feature Characterizing displacement information of the gesture action; based on the Doppler estimation, the at least one set of signal feature values is extracted from the action feature, and each set of signal feature values of the at least one set of signal feature values Corresponds to a frame signal in the motion feature representation.
  • the two types of time arrays include a fast time array and a slow time array; the processor 10 is further configured to process the first millimeter wave into at least one beam, and the at least one Each beam in a beam corresponds to the first millimeter wave received at a receiving time point; in the fast time array, at least one first information corresponding to the at least one beam is acquired, and the at least one first information is characterized At least one frequency corresponding to the at least one beam; in the slow time array, second information is determined according to the at least one first information, and the second information characterizes a frequency between the at least one beam Change; determining the second information as the action characteristic.
  • the processor 10 is further configured to use the Doppler effect to sequentially extract speed information and Doppler frequency shift information corresponding to the one-frame signal from the motion feature; Using the FM continuous wave principle, the distance information corresponding to the one-frame signal is sequentially extracted from the motion characteristics; at least the speed information, the Doppler frequency shift information, and the distance information are determined as the The set of signal characteristic values corresponding to one frame signal; and each set of signal characteristic values corresponding to each frame signal to form at least one set of signal characteristic values corresponding to the at least one frame signal.
  • the processor 10 is further configured to obtain a preset number of standard frame signals corresponding to a standard gesture action; and determine a preset number of standard signals corresponding to the preset number of standard frame signals.
  • Eigenvalues using a preset neural network to learn the preset number of standard signal eigenvalues to obtain a database of correspondences between the standard eigenvalues and control instructions.
  • the processor 10 is further configured to receive a reflected signal through the millimeter wave device; use the beam synthesis algorithm to combine the reflected signal into a reflected wave; and remove the clutter of the reflected wave Signal and noise signal to obtain the first millimeter wave.
  • the database of the correspondence between the standard feature values and the control instructions is a relationship database obtained through a preset neural network learning.
  • the terminal proposed in the embodiment of the present application receives a first millimeter wave through a millimeter wave device, where the first millimeter wave is a reflected wave modulated by a gesture motion of a second millimeter wave transmitted by the millimeter wave device; based on two types of time arrays and Doppler Le estimates that the first millimeter wave is processed to obtain at least one set of signal characteristic values corresponding to the first millimeter wave, and each set of signal characteristic values of the at least one set of signal characteristic values corresponds to a frame signal in the first millimeter wave;
  • the standard feature value and control instruction correspondence database is used to identify at least one set of signal characteristic values to obtain a first control instruction corresponding to a gesture action; and the first control instruction is used to control the first application to implement a corresponding function.
  • the terminal receives the first millimeter wave modulated by the gesture action through the millimeter wave device. Processing, and using Doppler estimation, to obtain at least one set of signal feature values corresponding to the processed first millimeter wave, and finally using at least one set of signal feature values and standard feature value and control command correspondence database to obtain the corresponding gesture action
  • the first control instruction can thereby recognize subtle gesture actions, thereby improving the accuracy of gesture perception.
  • An embodiment of the present application provides a storage medium.
  • the storage medium stores one or more programs.
  • the one or more programs can be executed by one or more processors and applied to a terminal.
  • the programs are implemented when the programs are executed by the processors.
  • the method is the same as in the first embodiment to the second embodiment.
  • the methods in the above embodiments can be implemented by means of software plus a necessary universal hardware platform, and of course, also by hardware, but in many cases the former is better.
  • Implementation Based on such an understanding, the technical solution of the present invention, in essence, or a part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, The optical disc) includes several instructions for causing a terminal (which may be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to execute the methods described in the embodiments of the present invention.
  • a terminal which may be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.
  • the terminal receives the first millimeter wave returned by the gesture action through the millimeter wave device, and processes the first millimeter wave based on two types of time arrays based on the characteristics of the small wavelength of the first millimeter wave, and uses Doppler Le estimates that at least one set of signal feature values corresponding to the processed first millimeter wave is obtained, and finally the first control instruction corresponding to the gesture action is obtained by using the at least one set of signal feature values and a preset neural network, which can identify subtle gesture actions , Thereby improving the accuracy of gesture perception.

Landscapes

  • Engineering & Computer Science (AREA)
  • Radar, Positioning & Navigation (AREA)
  • Remote Sensing (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • General Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • Electromagnetism (AREA)
  • Radar Systems Or Details Thereof (AREA)

Abstract

一种手势识别方法及终端、存储介质，该方法可以包括：通过毫米波装置接收第一毫米波，第一毫米波为通过毫米波装置发射的第二毫米波经过手势动作调制后的反射波（S101）；基于两类时间阵列和多普勒估计，对第一毫米波进行处理，得到第一毫米波对应的至少一组信号特征值，至少一组信号特征值中的每一组信号特征值对应第一毫米波中的一帧信号（S102）；利用标准特征值与控制指令对应关系库，对至少一组信号特征值进行识别，得到手势动作对应的第一控制指令（S103）；利用第一控制指令，控制第一应用实现相应的功能（S104）。

Description

一种手势识别方法及终端、存储介质 技术领域
本申请涉及电子应用领域，尤其涉及一种手势识别方法及终端、存储介质。
背景技术
近年来，随着智能终端的快速发展，终端的功能变得越来越丰富，而用户对终端的控制也不仅限于在终端的显示界面上点击、滑动等操作方式，未来，手势感知会成为用户控制终端的发展趋势，用户无需接触终端，而只需在与终端的一定范围内变换不同的手势，从而使终端识别不同的手势来实现对应的不同功能，扩展了用户控制终端的形式。现有的手势识别方案包括声波手势识别和基于可见光摄像头图像分析的手势识别，现有的手势识别方案存在手势识别的准确率低的问题。
以声波手势识别方案为例，该方案中终端根据用户的手势动作产生的超声波信号重建出该手势动作，然而，在嘈杂环境中声波手势识别方案的手势识别的准确度会大幅度降低。以基于可见光摄像头图像分析的手势识别方案为例，该方案中终端根据通过摄像头采集的多角度手势图像重建出手势动作，然而，在弱光或无光的环境下该方案的手势识别的准确度会比较低。
发明内容
本申请实施例期望提供一种手势识别方法及终端、存储介质，能够提高手势识别的准确率。
本申请实施例提供一种手势识别方法，应用于终端，所述终端上设置有毫米波装置，所述方法包括：
通过所述毫米波装置接收第一毫米波，所述第一毫米波为通过所述毫米波装置发射的第二毫米波经过手势动作调制后的反射波；
基于两类时间阵列和多普勒估计，对所述第一毫米波进行处理，得到所述第一毫米波对应的至少一组信号特征值，所述至少一组信号特征值中的每一组信号特征值对应所述第一毫米波中的一帧信号；
利用标准特征值与控制指令对应关系库，对所述至少一组信号特征值进行识别，得到所述手势动作对应的第一控制指令；
利用所述第一控制指令，控制第一应用实现相应的功能。
本申请实施例提供一种终端，所述终端包括：处理器、接收器、存储器及通信总线，所述终端上设置有毫米波装置，所述接收器，用于通过毫米波装置接收手势动作返回的第一毫米波，所述第一毫米波为通过所述毫米波装置发射的第二毫米波经过手势动作调制后的反射波；所述处理器用于执行所述存储器中存储的运行程序，以实现以下步骤：
基于两类时间阵列和多普勒估计，对所述第一毫米波进行处理，得到所述第一毫米波对应的至少一组信号特征值，所述至少一组信号特征值中的每一组信号特征值对应所述第一毫米波中的一帧信号；利用标准特征值与控制指令对应关系库，对所述至少一组信号特征值进行识别，得到所述手势动作对应的第一控制指令；利用所述第一控制指令，控制第一应用实现相应的功能。
本申请实施例提供一种存储介质，其上存储有计算机程序，应用于终端，该计算机程序被处理器执行时实现如上述任一项手势识别方法。
本申请实施例提供一种手势识别方法及终端、存储介质，该方法包括：通过毫米波装置接收第一毫米波，第一毫米波为通过毫米波装置发射的第二毫米波经过手势动作调制后的反射波；基于两类时间阵列和多普勒估计，对第一毫米波进行处理，得到第一毫米波对应的至少一组信号特征值，至少一组信号特征值中的每一组信号特征值对应第一毫米波中的一帧信号；利用标准特征值与控制指令对应关系库，对至少一组信号特征值进行识别，得到手势动作对应的第一控制指令；利用第一控制指令，控制第一应用实现相应的功能。采用上述方案，终端通过毫米波装置接收经过手势动作调制的第一毫米波，根据第一毫米波的波长小的特性，基于两类时间阵列对第一毫米波进行处理，并利用多普勒估计，得到处理后的第一毫米波对应的至少一组信号特征值，最后利用至少一组信号特征值和标准特征值与控制指令对应关系库，得到手势动作对应的第一控制指令，由此能够识别出细微的手势动作，从而提高了手势感知的准确率。
附图说明
图1为本申请实施例提供的一种手势识别方法的流程图一；
图2为本申请实施例提供的一种示例性的终端的结构组成图；
图3为本申请实施例提供的一种示例性的一帧信号对应的特征值的显示图；
图4为本申请实施例提供的一种示例性的手势控制的架构图；
图5为本申请实施例提供的一种手势识别方法的流程图二；
图6为本申请实施例提供的一种示例性的调频连续波的原理图；
图7为本申请实施例提供的一种示例性的多普勒频移的原理图；
图8为本申请实施例提供的一种示例性的卷积神经网络模型的示意图；
图9为本申请实施例提供的一种示例性的基于Matlab程序的手势识别流程图；
图10为本申请实施例提供的一种示例性的手势动作示意图；
图11为本申请实施例提供的一种终端的结构示意图。
具体实施方式
为了能够更加详尽地了解本申请实施例的特点与技术内容，下面结合附图对本申请实施例的实现进行详细阐述，所附附图仅供参考说明之用，并非用来限定本申请实施例。
毫米波是指30-300千兆赫的频段，由于该频段的可用带宽十分充裕，使得在毫米波频段上进行 数据传输时的传输速率非常大；毫米波由于大带宽和高速率等特性，成为第五代无线通信技术(5G，5th-Generation)所使用的通讯频段，利用毫米波进行数据传输，可大幅度提高无线网络速率。例如，运行在60GHz频段的IEEE 802.11ad支持高达6.7Gbps的数据传输速率，而其演进标准IEEE802.11ay将提供20Gbps的数据传输速率。因此，毫米波无线电有望使无线网络接入进入到multi-Gbps时代。因此毫米波无线电模块将会被广泛安装在手机、可穿戴、智能硬件或更广泛的物联网设备上，成为一种主流的通讯技术。毫米波除了高速率链接，毫米波的短波长、大带宽、有向波束等特点，也使得高分辨率、高健壮性的人员手势感知成为可能。
毫米波技术感知技术能够提供更智能、便捷、有趣人机交互应用体验。其基本原理是使用毫米波射频模块发射毫米波，接收模块收到手势动作的反射波，通过反射波推测手势过程中的距离、角度、速度和能量大小，以此进行动作分类。毫米波支持距离测量、手势检测、接近检测、人数探测、距离测量、存在性检测等多种感知功能，可应用在如下场景中：
来电、闹钟等铃声场景，用户可通过特定手势(例如接近手机)，降低铃声音量，直至静音。
在自拍的过程中，可以通过一系列手势“告诉”手机拍照时机、明暗度调节、焦距调节等，从而省去触摸手机屏幕的不便捷操作。
在屏幕上方左/右/上/下滑，查看上一个/下一个应用，返回桌面及进入多任务。
在屏幕上方上滑，或多指捏合，进入多任务或某特定模式。
识别用户在手机边框附近的精细手势，来执行按钮操作，如隔空滑页、调节视频音量和亮度、切换音乐、切换相机滤镜。
手在屏幕上方悬空拍打，截取屏幕截图。
在不方便点击屏幕时(如带手套时)，可悬空点击，模拟屏幕点击动作。
识别手的运动轨迹，来添加一些视频、照片特效。
远距离摄影时，利用手势来切换相机滤镜、调节相机焦距、暂停与继续、删除已拍摄内容等。
以下为本申请实施例在拍照场景下利用毫米波进行手势识别的场景。
实施例一
本申请实施例提供的一种手势识别方法，应用于终端，该终端上设置有毫米波装置，如图1所示，该方法可以包括：
S101、通过毫米波装置接收第一毫米波，第一毫米波为通过毫米波装置发射的第二毫米波经过手势动作调制后的反射波。
本申请实施例提供的一种手势识别方法应用于对用户的手势进行感知来实现非接触式拍照的场景下。
本申请实施例中，上述终端可以为任何具备通信和存储功能的设备，例如：平板电脑、手机、电子阅读器、遥控器、个人计算机(Personal Computer，PC)、笔记本电脑、车载设备、网络电视、可穿戴设备等设备，具体的根据实际情况进行选择，本申请实施例不做具体的限定。
本申请实施例中，终端的屏幕内部设置有毫米波装置，毫米波装置包括发射天线和接收天 线。
可以理解的是，由于毫米波无线电可穿透塑料等非金属类材料，使得毫米波装置被隐藏部署在终端屏幕内部，不会更改终端的外观，从而对终端外形设计意义重大。
本申请实施例中，终端通过毫米波装置的发送天线发射无线信号(第二毫米波)，在无线信号的发射范围内经过手部动作调制后形成反射信号(第一毫米波)，之后反射信号被毫米波装置的接收天线捕获。
本申请实施例中，终端可以在预设发射时间到达时，通过毫米波装置的发送天线发射无线信号，也可以在启动如拍照应用、视频拍摄应用等第一应用时，通过毫米波装置的发送天线发射无线信号，具体的终端通过毫米波装置的发送天线发射无线信号的时机根据实际情况进行选择，本申请实施例不做具体的限定。
本申请实施例中，毫米波装置发射无线信号的形式为周期性的发射调频连续波(FMCW，Frequency Modulated Continuous Wave)，使得第一毫米波频率与第二毫米波变化规律相同，都是三角波规律，只是有一个时间差，终端可以利用这个微小的时间差可计算出目标距离。
示例性的，如图2所示，终端上设置有数字信号处理器(DSP，Digital Signal Processing)，其中，DSP由距离处理模块、Capon波束形成器(Capon Beam Former)、目标检测单元(Object Detection)和多普勒估计单元(Doppler Estimation)四个部分组成，其中，
距离处理模块：当接收天线接收到反射波之后，反射波被缓存到模数转换器(ADC，Analog-to-Digital Converter)输出缓存区，之后毫米波装置将反射波从ADC输出缓存区移动到DSP的本地内存中，此时，距离处理模块执行16位定点1-D窗口和16位定点1-D快速傅立叶变换(FFT，Fast Fourier Transformation)，并将执行结果传送至多普勒估计单元。
Capon波束形成器：利用公式(1)，从传感器阵列重构源信号
X(t)＝A(θ)s(t)+n(t)           (1)
其中，s(t)是混合基带信号后的输入信号；
通过移除距离处理模块中的每个距离接收器的直流元件去除静态杂物，从而消除了例如桌子、椅子等静态对象在感兴趣区域的反射；
使用帧内多重线性调频计算每一个距离接收器的空间协方差矩阵R n，并对R n求逆得到R n -1，将每一个距离接收器的R n -1的上对角线存储在内存中，之后对每一距离接收器计算Capon束形成器输出，并将角度谱图存储在存储器，以构建[距离，方位]热图，最后将[距离，方位]热图传输至多普勒估计单元。
目标检测单元：利用恒虚警率(CFAR，Constant False-Alarm Rate)检测算法，对距离域中的第一通道和角度域中的第二通道进行处理，第二通道对第一通道的结果加以确认，从而去除杂波和噪声，并确定出检测点。
多普勒估计：对于每一[距离，方位]对，利用Capon波束加权算法过滤距离接收器，然后在滤波后的距离接收器的FFT上进行峰值搜索，以此来估计多普勒。
S102、基于两类时间阵列和多普勒估计，对第一毫米波进行处理，得到第一毫米波对应的至少一组信号特征值，至少一组信号特征值中的每一组信号特征值对应第一毫米波中的一帧信号。
当终端通过毫米波装置接收到第一毫米波之后，终端对第一毫米波进行处理，得到第一毫米波对应的至少一组信号特征值。
本申请实施例中，在终端接收到第一毫米波之后，终端基于两类时间阵列，对第一毫米波进行处理，得到手势动作对应的动作特征，其中，动作特征表征手势动作的位移信息；之后，终端基于多普勒估计，从动作特征中提取到至少一组信号特征值，其中，至少一组信号特征值中的每一组信号特征值对应动作特征表征中的一帧信号。
本申请实施例中，两类时间阵列包括快速时间阵列和慢速时间阵列，终端将第一毫米波处理成至少一个波束，其中，至少一个波束中每个波束对应在一个接收时间点所接收的第一毫米波；终端在快速时间阵列中，获取至少一个波束对应的至少一个第一信息，至少一个第一信息表征至少一个波束对应的至少一个频率；之后，终端在慢速时间阵列中，根据至少一个第一信息，确定出第二信息，第二信息表征至少一个波束之间的频率变化；将第二信息确定为动作特征。
本申请实施例中，终端在将第一毫米波处理成每个接收时间点对应的至少一个波束，终端在快速时间阵列中计算出至少一个波束中的每一个波束对应的频率，之后，终端在慢速时间阵列中根据至少一个波束中的每一个波束对应的频率，计算出至少一个波束之间的频率变化，该频率变化表征手势动作的位移信息，终端将该频率变化确定为手势动作的动作特征。
需要说明的是，终端识别不同手部动作的基本原理为：将手假设为离散动态散射中心，将手的射频(RF，Radio Frequency)响应，建模为来自离散动态散射中心的响应的叠加，当波长小于目标的空间范围时，散射中心模型与衍射的集合理论是一致的，由于毫米波的波长短的特性，上述假设适用于毫米波手部动作感知。本方案采用广义时变散射中心模型，并考虑非刚性手部动力学，即每个散射中心通过来自传感器的复合反射率参数和径向距离来进行参数化，而复合反射率参数是频率依赖性的，随着手的局部几何形状，相对于雷达的方向等而变化。因此，本申请采用高时间分辨率的感知，即通过高帧率测量手对雷达的响应，然后提取与这些手部运动相对应的细微时间信号变化，来检测细微而复杂的手部动作。终端控制毫米波设备发送周期性的调制波形来实现上述概念，毫米波雷达在每个传输周期分别测量相应的接收波形。因此，为了实现上述方案，本申请定义了两种不同的时间尺度来分析反射的第一毫米波，分别为短时尺度感知和长时尺度感知。
本申请实施例中，终端在快速时间阵列中使用短时尺度感知，在慢速时间阵列中使用长时尺度感知。
其中，短时尺度感知的原理是高雷达重复频率将散射中心手部模型与信号处理方法联系起来，对于足够高的高速雷达频率和手部相对慢的动作，散射中心模型在单个雷达重复间隔内近似恒定，散射中心范围和反射率是紧随短时尺度T变化的函数。在每个传输周期内以单一的宽波束照射手，手上的所有散射中心同时反射信号，测量波形由每个散射中心的反射组成，并以快速时 间叠加，每个单独的反射波形有相关散射中心的瞬时反射率和范围调制，在RF解调和调制特定滤波之后，预处理的接收信号表示来自每个散射中心响应的叠加。高雷达重复频率能够在慢速时间内捕获与散射中心动态相对应的接收信号中的精细相位变化。
其中，长时尺度感知的原理是当散射中心移动时，散射中心的相对位移会产生与波长成比例的相位变化。相位变化对位移的依赖使毫米波装置能根据其相位找到慢时间内的分散散射中心。假设每个散射中心的速度在大于雷达重复间隔的一些相干处理时间上近似恒定，相干处理时间上的相位则会产生多普勒频率，因此可以通过计算相干处理慢时间窗口上每个快速时间窗中的波形的频谱来解析以不同速度移动的多个散射中心的多普勒频率。
本申请实施例中，终端通过短时尺度感知和长时尺度感知将第一毫米波处理成手势动作对应的动作特征。
本申请实施例中，在快速时间阵列和慢速时间阵列中缓冲一定数目个连续预处理雷达信号，用于表征动作特征。
本申请实施例中，每一帧信号至少由11个特征值组成，如图3所示，该11个特征值包括：数字检测(num_detection)、多普勒均值(Doppler_average)、距离均值(range_average)、振幅和(magnitude_sum)、活跃数字检测(positive num_detetion)、距离索引(range_index)、非活跃数字检测(negative num_detection)、非活跃多普勒均值(negtaive doppler_average)、距离显示(range_disp)、角度值(angle_value)、预测结果(predication_result)。
本申请实施例中，利用多普勒效应计算手势动作的速度和多普勒频移、利用FMCW原理计算手势动作距终端的距离。
S103、利用标准特征值与控制指令对应关系库，对至少一组信号特征值进行识别，得到手势动作对应的第一控制指令。
当终端获取到至少一组信号特征值之后，终端利用标准特征值与控制指令对应关系库，对至少一组信号特征值进行识别，得到手势动作对应的第一控制指令。
本申请实施例中，标准特征值与控制指令对应关系库是经过预设神经网络学习得到的关系库，具体的，终端利用预设神经网络对标准手势动作进行学习，得到一个控制指令对应的至少一组标准特征值，终端将控制指令以及对应的至少一组标准特征值组成标准特征值与控制指令对应关系库，当终端获取到第一毫米波对应的至少一组信号特征值之后，终端从标准特征值与控制指令对应关系库中，查找至少一组信号特征值对应的第一控制指令。
可选的，预设神经网络为去除残差网络resnet18的后三层layer层之后，得到的6层残差网络。
本申请实施例中，终端接收每一个控制指令对应的标准手势动作之后，对该标准手势动作进行处理，得到该标准手势动作对应的一组帧序列信号(标准帧信号)，其中，一组帧序列信号中的每一帧序列信号对应一组特征值(预设数目的标准信号特征值)，终端将一组帧序列信号对应的至少一组特征值输入该6层残差网络中，利用该6层残差网络对至少一组特征值进行学习， 得到每一个控制指令对应的标准特征值组，并将控制指令和对应的标准特征值组作为训练好的网络模型保存为.pkl的格式，当终端对新手势动作进行预测时，调用python脚本来导入训练好的网络模型，该脚本由Matlab程序进行调用，python脚本在对至少一组信号特征值进行分类预测之后，将预测分类结果返还给Matlab程序。
本申请实施例中，终端将至少一组信号特征值和标准特征值与控制指令对应关系库进行匹配，当至少一组信号特征值和标准特征值与控制指令对应关系库中的第一标准特征值组匹配成功时，终端从标准特征值与控制指令对应关系库中查找第一标准特征值组对应的第一控制指令，此时，终端利用标准特征值与控制指令对应关系库，得到了手势动作对应的第一控制指令。
本申请实施例中，第一控制指令用于控制相机实现拍照、调焦等功能，具体的根据实际情况进行选择，本申请实施例不做具体的限定。
示例性的，当终端接收到初始状态为右手手指自然打开，向前举起右手小臂，之后右手臂肘关节带动小臂以肘关节为轴向左侧放平，且放平过程的同时右手握拳的手势变化时，终端确定出第一控制指令为控制相机进行拍照。
S104、利用第一控制指令，控制第一应用实现相应的功能。
当终端得到手势动作对应的第一控制指令之后，终端利用第一控制指令，控制第一应用实现相应的功能。
本申请实施例中，终端在得到第一控制指令之后，将第一控制指令输入Matlab程序中，并利用Matlab程序完成控制相机的功能。具体的，终端利用Matlab程序完成控制相机的功能采用的方式是：终端通过Matlab程序调用Webcam模块。当终端获取到第一控制指令之后，Matlab程序就会给Webcam模块发送第一控制指令对应的一个control值；Webcam模块在接收到control值之后，根据不同的control值，控制相机的实现不同功能。
需要说明的是，Matlab程序的使用贯穿整个系统。Matlab程序被用来存储毫米波装置收集到的信号，调用python的预测脚本进行预测，在获取到预测值后控制相机。
示例性的，如图4所示，手势控制的总架构为：毫米波设备接收标准手势动作调制后的原始信号，并对原始信号进行处理得到至少一组特征值，之后，将至少一组特征值输入神经网络分析，在经过神经网络的预测之后，控制相机完成相应的功能。
可以理解的是，终端通过毫米波装置接收手势动作返回的第一毫米波，根据第一毫米波的波长小的特性，基于两类时间阵列对第一毫米波进行处理，并利用多普勒估计，得到处理后的第一毫米波对应的至少一组信号特征值，最后利用至少一组信号特征值和预设神经网络得到手势动作对应的第一控制指令，能够识别出细微的手势动作，从而提高了手势感知的准确率。
实施例二
本申请实施例提供一种手势识别方法，应用于终端，终端上设置有毫米波装置，如图5所示，该方法可以包括：
S201、终端通过毫米波装置接收反射信号。
本申请实施例提供的一种手势识别方法应用于对用户的手势进行感知来实现非接触式拍照的场景下。
本申请实施例中，终端可以在预设发射时间到达时，通过毫米波装置的发送天线发射无线信号，也可以在启动第一应用时，通过毫米波装置的发送天线发射无线信号，具体的终端通过毫米波装置的发送天线发射无线信号的时机根据实际情况进行选择，本申请实施例不做具体的限定。
本申请实施例中，第一应用为拍照应用或者视频拍摄应用等，具体的根据实际情况进行选择，本申请实施例不做具体的限定。
本申请实施例中，当用户在应用图标显示界面点击第一应用的应用图标时，终端接收到启动第一应用的启动指令，此时，终端启动第一应用，并利用毫米波装置发射无线信号。
本申请实施例中，毫米波装置包括发射天线和接收天线，毫米波装置利用发射天线发射无线信号，在无线信号的发射范围内经过手部动作调制后形成反射信号，之后反射信号被毫米波装置的接收天线捕获。
本申请实施例中，毫米波装置发射无线信号的形式为周期性的发射FMCW波，使得第一毫米波频率与第二毫米波变化规律相同，都是三角波规律，只是有一个时间差，终端可以利用这个微小的时间差可计算出目标距离。
S202、终端利用波束合成算法，将反射信号合成反射波。
当终端通过毫米波装置接收到反射信号之后，终端利用波束合成算法，将反射信号合成反射波。
本申请实施例中，当接收天线接收到反射波之后，反射波被缓存到ADC输出缓存区，之后毫米波装置将反射波从ADC输出缓存区移动到DSP的本地内存中，终端利用Capon波束合成器将反射信号合成反射波。
本申请实施例中，利用公式(1)，从传感器阵列重构源信号
X(t)＝A(θ)s(t)+n(t)         (1)
其中，s(t)是混合基带信号后的输入信号。
本申请实施例中，通过移除距离处理模块中的每个距离接收器的直流元件去除静态杂物，从而消除了例如桌子、椅子等静态对象在感兴趣区域的反射。
S203、终端去除反射波的杂波信号和噪声信号，得到第一毫米波，第一毫米波为通过毫米波装置发射的第二毫米波经过手势动作调制后的反射波。
当终端将反射信号合成反射波之后，终端去除反射波中的杂波信号和噪声信号，从而得到第一毫米波。
本申请实施例中，利用恒虚警率(CFAR，Constant False-Alarm Rate)检测算法，对距离域中的第一通道和角度域中的第二通道进行处理，第二通道对第一通道的结果加以确认，从而去除杂波和噪声，并确定出检测点，进而得到第一毫米波。
S204、终端将第一毫米波处理成至少一个波束，至少一个波束中每个波束对应在一个接收 时间点所接收的第一毫米波。
当终端得到第一毫米波之后，终端将第一毫米波处理成至少一个波束，其中，至少一个波束中每个波束对应在一个接收时间点所接收的第一毫米波。
本申请实施例中，终端将第一毫米波分割成至少一个接收时间点对应的至少一个波束。
S205、在快速时间阵列中，终端获取至少一个波束对应的至少一个第一信息，至少一个第一信息表征至少一个波束对应的至少一个频率。
当终端将第一毫米波处理成至少一个波束之后，终端在快速时间阵列中，获取至少一个波束对应的至少一个第一信息。
本申请实施例中，终端基于短时尺度感知原理，计算出至少一个波束对应的至少一个第一信息，其中，至少一个第一信息用于表征至少一个波束对应的至少一个频率。
其中，短时尺度感知的原理是高雷达重复频率将散射中心手部模型与信号处理方法联系起来，对于足够高的高速雷达频率和手部相对慢的动作，散射中心模型在单个雷达重复间隔内近似恒定，散射中心范围和反射率是紧随短时尺度T变化的函数。在每个传输周期内以单一的宽波束照射手，手上的所有散射中心同时反射信号，测量波形由每个散射中心的反射组成，并以快速时间叠加，每个单独的反射波形有相关散射中心的瞬时反射率和范围调制，在RF解调和调制特定滤波之后，预处理的接收信号表示来自每个散射中心响应的叠加。高雷达重复频率能够在慢速时间内捕获与散射中心动态相对应的接收信号中的精细相位变化。
S206、在慢速时间阵列中，终端根据至少一个第一信息，确定出第二信息，第二信息表征至少一个波束之间的频率变化。
当终端在快速时间阵列中获取到至少一个波束对应的至少一个第一信息之后，终端在慢速时间阵列中，根据至少一个第一信息，确定出第二信息，其中，第二信息表征至少一个波束之间的频率变化。
本申请实施例中，终端基于长时尺度感知原理，计算出表征至少一个波束之间的频率变化的第二信息。
其中，长时尺度感知的原理是当散射中心移动时，散射中心的相对位移会产生与波长成比例的相位变化。相位变化对位移的依赖使毫米波装置能根据其相位找到慢时间内的分散散射中心。假设每个散射中心的速度在大于雷达重复间隔的一些相干处理时间上近似恒定，相干处理时间上的相位则会产生多普勒频率，因此可以通过计算相干处理慢时间窗口上每个快速时间窗中的波形的频谱来解析以不同速度移动的多个散射中心的多普勒频率。
具体的，终端在慢速时间阵列上对每个快速时间阵列应用FFT获得频率信息。由此产生的快速时频映射通过变换转换成距离和速度。可以根据SNR，速度分辨率和多普勒混叠针对期望的手动力学和期望的感测性能进行精细调整。从而确定出至少一个波束的频率变化信息，该频率变化信息表征手部多中心随时间变化的距离和速率。
S207、终端将第二信息确定为动作特征。
当终端确定出第二信息之后，终端将第二信息确定为第一毫米波对应的动作特征。
本申请实施例中，终端将手部多中心随时间变化的距离和速率成为动作特征。
本申请实施例中，终端在快速时间阵列和慢速时间阵列中缓冲一定数目个连续预处理雷达信号，用于表征动作特征。
S208、终端利用多普勒效应，依次从动作特征中提取到一帧信号对应的速度信息和多普勒频移信息。
当终端将第二信息确定为动作特征之后，终端利用多普勒效应，从动作特征中提取到一帧信号对应的速度信息和多普勒频移信息。
本申请实施例中，终端将动作特征确定为至少一帧信号，之后利用多普勒效应依次对至少一帧信号中的一帧信号进行处理，以依次从一帧信号中提取出对应的速度信息和多普勒频移信息。
本申请实施例中，多普勒效应是指物体辐射的波长因为光源和观测者的相对运动而产生变化。在运动的波源前面，波被压缩，波长变得较短，频率变得较高，在运动的波源后面，产生相反的效应，波长变得较长，频率变得较低。波源的速度越高，所产生的效应越大，根据光波红/蓝移的程度，可以计算出波源循着观测方向运动的速度信息。
本申请实施例中，由于毫米波装置发射的是调频连续波，第二毫米波和第一毫米波的频率变化规律均符合三角波规律，故，根据多普勒效应，其频差如图6所示，频率-时间坐标中的实线为发射波频率变化曲线，虚线为接收波频率变化曲线，其中f b为被探测物体静止时的频差，f d是被探测物体移动时的多普勒频移。
S209、终端利用调频连续波原理，依次从动作特征中提取到一帧信号对应的距离信息。
当终端将第二信息确定为动作特征之后，终端利用调频连续波原理，依次从动作特征中提取到一帧信号对应的距离信息。
本申请实施例中，终端将动作特征确定为至少一帧信号，之后利用调频连续波原理，依次对至少一帧信号中的一帧信号进行处理，以依次从一帧信号中提取出对应的距离信息。
本申请实施例中，毫米波装置发送毫米波形式可以计算出相对目标之间的距离。其基本原理为，发射波为高频连续波，其频率随时间按照三角波规律变化。雷达接收的回波的频率与发射的频率变化规律相同，都是三角波规律，只是有一个时间差，利用这个微小的时间差可计算出距离信息。
如图7所示，虚线为发射波频率变化曲线，实线为接收波频率变化曲线，其中，td为ft和fr的时间差，ft为发射波频率，fr为接收波频率。
S208和S209为S207之后的两个并列的步骤，具体的执行顺序根据实际情况进行选择，本申请实施例不做具体的限定。
S210、终端至少将速度信息、多普勒频移信息和距离信息，确定为一帧信号对应的一组信号特征值。
当终端提取出速度信息、多普勒频移信息和距离信息之后，终端至少将速度信息、多普勒频移 信息和距离信息，确定为一帧信号对应的一组信号特征值。
本申请实施例中，一帧信号包括与速度信息、多普勒频移信息和距离信息有关的一组信号特征值。
本申请实施例中，每一帧信号至少由11个与速度信息、多普勒频移信息和距离信息有关特征值组成，如图3所示，该11个特征值包括：数字检测(num_detection)、多普勒均值(Doppler_average)、距离均值(range_average)、振幅和(magnitude_sum)、活跃数字检测(positive num_detetion)、距离索引(range_index)、非活跃数字检测(negative num_detection)、非活跃普勒均值(negtaive doppler_average)、距离显示(range_disp)、角度值(angle_value)、预测结果(predication_result)。
S211、终端将每一帧信号对应的每一组信号特征值，组成至少一帧信号对应的至少一组信号特征值。
当终端确定出一帧信号对应的一组信号特征值之后，终端将每一帧信号对应的每一组信号特征值，组成至少一帧信号对应的至少一组信号特征值。
本申请实施例中，终端依次确定出每一帧信号对应的每一组信号特征值，之后，将每一帧信号对应的每一组信号特征值组成至少一帧信号对应的至少一组信号特征值。
S212、终端利用标准特征值与控制指令对应关系库，对至少一组信号特征值进行识别，得到手势动作对应的第一控制指令。
当终端获取到标准特征值与控制指令对应关系库之后，终端利用标准特征值与控制指令对应关系库，对至少一组信号特征值进行分类预测。
本申请实施例中，终端从标准特征值与控制指令对应关系库中查找与至少一组信号特征值对应的第一标准信号特征值，并确定第一标准信号特征值对应的第一控制指令，其中，标准特征值与控制指令对应关系库是经过预设神经网络学习得到的关系库。
本申请实施例中，终端利用Matlab程序调用python脚本，以导入训练好的网络模型(标准特征值与控制指令对应关系库)。python脚本在对至少一组信号特征值进行分类预测之后，将预测分类结果值返还给Matlab程序。
S213、终端利用第一控制指令，控制第一应用实现相应的功能。
当终端得到第一控制指令之后，终端利用第一控制指令，控制第一应用实现相应的功能。
本申请实施例中，终端在得到第一控制指令之后，将第一控制指令输入Matlab程序中，并利用Matlab程序完成控制相机的功能。具体的，终端利用Matlab程序完成控制相机的功能采用的方式是：终端通过Matlab程序调用Webcam模块。当终端获取到第一控制指令之后，Matlab程序就会给Webcam模块发送第一控制指令对应的一个control值；Webcam模块在接收到control值之后，根据不同的control值，控制相机的实现不同功能。
需要说明的是，Matlab程序的使用贯穿整个系统。Matlab程序被用来存储毫米波装置收集到的信号，调用python的预测脚本进行预测，在获取到预测值后控制相机。
示例性的，如图9所示，Matlab程序(矩阵工厂)的使用贯穿整个系统过程，毫米波装置将采 集到的信号传输给Matlab程序，Matlab程序中存储毫米波设备收集到的信号，并调用python的预测脚本进行预测，在获取到预测值后控制相机。
可以理解的是，终端通过毫米波装置接收手势动作返回的第一毫米波，根据第一毫米波的波长小的特性，基于两类时间阵列对第一毫米波进行处理，并利用多普勒估计，得到处理后的第一毫米波对应的至少一组信号特征值，最后利用至少一组信号特征值和预设神经网络得到手势动作对应的第一控制指令，能够识别出细微的手势动作，从而提高了手势感知的准确率。
基于上述实施例二，在本申请的实施例中，上述终端在进行手势识别时还实时利用预设神经网络学习，得到标准特征值与控制指令对应关系库，终端进行手势识别的方法还可以包括以下步骤：
S301、终端获取标准手势动作对应的预设数目的标准帧信号。
本申请实施例中，终端预先确定出一个标准手势动作所需采集的时间段，之后确定该时间段对应的标准帧信号的数目。
示例性的，一个标准的手势动作需要采集2秒，2秒中可以采集到的标准帧信号的数量为60个。
示例性的，终端获取到控制相机实现把不同功能的4种标准手势，如图10所示。
手势1–调焦：弯曲右手中指、无名指、小指，使指尖接触掌心，伸出食指与拇指并使二者围成椭圆，两根手指指尖分开，虎口打开，右手小臂与地面呈一定斜角，上下活动手腕从而带动右手上下敲击两次。
手势2–放大：手握拳，手向前朝着TI设备水平移动，移动过程中手指打开。
手势3–缩小：手指自然打开，手向前朝着毫米波设备移动，移动过程中手握拳。
手势4–拍照：右手手指自然打开，向前举起右手小臂，使小臂后侧正对毫米波设备，右手臂肘关节带动小臂以肘关节为轴向左侧放平，放平过程的同时右手握拳。
S302、终端确定预设数目的标准帧信号对应的预设数目的标准信号特征值。
当终端获取到标准手势动作对应的预设数目的标准帧信号之后，终端确定预设数目的标准帧信号对应的预设数目的标准信号特征值。
本申请实施例中，终端对标准手势动作进行处理，得到该标准手势动作对应的一组帧序列信号，其中，一组帧序列信号中的每一帧序列信号对应一组特征值。
示例性的，每一帧序列信号含有11个与角度、距离、多普勒频移有关的特征值。
S303、终端利用预设神经网络，对预设数目的标准信号特征值进行学习，得到标准特征值与控制指令对应关系库。
当终端确定出预设数目的标准帧信号对应的预设数目的标准信号特征值之后，终端利用预设神经网络，对预设数目的标准信号特征值进行学习，得到标准特征值与控制指令对应关系库。
可选的，预设神经网络为去除残差网络resnet18的后三层layer层之后，得到的6层残差网络。
本申请实施例中，终端将一组帧序列信号对应的至少一组特征值输入该6层残差网络中，利用该6层残差网络对至少一组特征值进行学习，得到每一个控制指令对应的标准特征值组，并将控制 指令和对应的标准特征值组作为训练好的网络模型保存为.pkl的格式。
实施例三
图11为本申请实施例提出的终端的组成结构示意图一，终端上设置有毫米波装置，在实际应用中，基于实施例一和实施例二的同一发明构思下，如图11所示，本申请实施例的终端1包括：处理器10、接收器11、存储器12及通信总线13。在具体的实施例的过程中，上述处理器10可以为特定用途集成电路(ASIC，Application Specific Integrated Circuit)、数字信号处理器(DSP，Digital Signal Processor)、数字信号处理终端(DSPD，Digital Signal Processing Device)、可编程逻辑终端(PLD，Programmable Logic Device)、现场可编程门阵列(FPGA，Field Programmable Gate Array)、CPU、控制器、微控制器、微处理器中的至少一种。可以理解地，对于不同的设备，用于实现上述处理器功能的电子器件还可以为其它，本申请实施例不作具体限定。
在本申请的实施例中，上述通信总线13用于实现处理器10、接收器11和存储器12之间的连接通信；上述接收器11，用于通过所述毫米波装置接收第一毫米波，所述第一毫米波为通过所述毫米波装置发射的第二毫米波经过手势动作调制后的反射波；上述处理器10用于执行存储器12中存储的运行程序，以实现以下步骤：
基于两类时间阵列和多普勒估计，对所述第一毫米波进行处理，得到所述第一毫米波对应的至少一组信号特征值，所述至少一组信号特征值中的每一组信号特征值对应所述第一毫米波中的一帧信号；利用标准特征值与控制指令对应关系库，对所述至少一组信号特征值进行识别，得到所述手势动作对应的第一控制指令；利用所述第一控制指令，控制所述第一应用实现相应的功能。
在本申请实施例中，进一步地，上述处理器10，还用于基于所述两类时间阵列，对所述第一毫米波进行处理，得到所述手势动作对应的动作特征，所述动作特征表征所述手势动作的位移信息；基于所述多普勒估计，从所述动作特征中提取到所述至少一组信号特征值，所述至少一组信号特征值中的每一组信号特征值对应所述动作特征表征中的一帧信号。
在本申请实施例中，进一步地，所述两类时间阵列包括快速时间阵列和慢速时间阵列；上述处理器10，还用于将所述第一毫米波处理成至少一个波束，所述至少一个波束中每个波束对应在一个接收时间点所接收的第一毫米波；在所述快速时间阵列中，获取所述至少一个波束对应的至少一个第一信息，所述至少一个第一信息表征所述至少一个波束对应的至少一个频率；在所述慢速时间阵列中，根据所述至少一个第一信息，确定出第二信息，所述第二信息表征所述至少一个波束之间的频率变化；将所述第二信息确定为所述动作特征。
在本申请实施例中，进一步地，上述处理器10，还用于利用多普勒效应，依次从所述动作特征中提取到所述一帧信号对应的速度信息和多普勒频移信息；利用调频连续波原理，依次从所述动作特征中提取到所述一帧信号对应的距离信息；至少将所述速度信息、所述多普勒频移信息和所述距离信息，确定为所述一帧信号对应的所述一组信号特征值；将每一帧信号对应的每一组信号特征值，组成所述至少一帧信号对应的至少一组信号特征值。
在本申请实施例中，进一步地，上述处理器10，还用于获取标准手势动作对应的预设数目的标准帧信号；确定所述预设数目的标准帧信号对应的预设数目的标准信号特征值；利用预设神经网络，对所述预设数目的标准信号特征值进行学习，得到所述标准特征值与控制指令对应关系库。
在本申请实施例中，进一步地，上述处理器10，还用于通过所述毫米波装置接收反射信号；利用波束合成算法，将所述反射信号合成反射波；去除所述反射波的杂波信号和噪声信号，得到所述第一毫米波。
在本申请实施例中，进一步地，所述标准特征值与控制指令对应关系库为经过预设神经网络学习得到的关系库。
本申请实施例提出的终端，通过毫米波装置接收第一毫米波，第一毫米波为通过毫米波装置发射的第二毫米波经过手势动作调制后的反射波；基于两类时间阵列和多普勒估计，对第一毫米波进行处理，得到第一毫米波对应的至少一组信号特征值，至少一组信号特征值中的每一组信号特征值对应第一毫米波中的一帧信号；利用标准特征值与控制指令对应关系库，对至少一组信号特征值进行识别，得到手势动作对应的第一控制指令；利用第一控制指令，控制第一应用实现相应的功能。由此可见，本申请实施例提出的终端，终端通过毫米波装置接收经过手势动作调制的第一毫米波，根据第一毫米波的波长小的特性，基于两类时间阵列对第一毫米波进行处理，并利用多普勒估计，得到处理后的第一毫米波对应的至少一组信号特征值，最后利用至少一组信号特征值和标准特征值与控制指令对应关系库，得到手势动作对应的第一控制指令，由此能够识别出细微的手势动作，从而提高了手势感知的准确率。
本申请实施例提供一种存储介质，上述存储介质存储有一个或者多个程序，上述一个或者多个程序可被一个或者多个处理器执行，应用于终端中，该程序被处理器执行时实现如实施例一至实施例二的方法。
需要说明的是，在本文中，术语“包括”、“包含”或者其任何其他变体意在涵盖非排他性的包含，从而使得包括一系列要素的过程、方法、物品或者系统不仅包括那些要素，而且还包括没有明确列出的其他要素，或者是还包括为这种过程、方法、物品或者系统所固有的要素。在没有更多限制的情况下，由语句“包括一个……”限定的要素，并不排除在包括该要素的过程、方法、物品或者系统中还存在另外的相同要素。
上述本申请实施例序号仅仅为了描述，不代表实施例的优劣。
通过以上的实施方式的描述，本领域的技术人员可以清楚地了解到上述实施例方法可借助软件加必需的通用硬件平台的方式来实现，当然也可以通过硬件，但很多情况下前者是更佳的实施方式。基于这样的理解，本发明的技术方案本质上或者说对现有技术做出贡献的部分可以以软件产品的形式体现出来，该计算机软件产品存储在一个存储介质(如ROM/RAM、磁碟、光盘)中，包括若干指令用以使得一台终端(可以是手机，计算机，服务器，空调器，或者网络设备等)执行本发明各个实施例所述的方法。
上面结合附图对本发明的实施例进行了描述，但是本发明并不局限于上述的具体实施方式，上述的具体实施方式仅仅是示意性的，而不是限制性的，本领域的普通技术人员在本发明的启示下，在不脱离本发明宗旨和权利要求所保护的范围情况下，还可做出很多形式，这些均属于本发明的保护之内。
工业实用性
在本申请实施例中，终端通过毫米波装置接收手势动作返回的第一毫米波，根据第一毫米波的波长小的特性，基于两类时间阵列对第一毫米波进行处理，并利用多普勒估计，得到处理后的第一毫米波对应的至少一组信号特征值，最后利用至少一组信号特征值和预设神经网络得到手势动作对应的第一控制指令，能够识别出细微的手势动作，从而提高了手势感知的准确率。

Claims (15)

  1. 一种手势识别方法，应用于终端，所述终端上设置有毫米波装置，所述方法包括：
    通过所述毫米波装置接收第一毫米波，所述第一毫米波为通过所述毫米波装置发射的第二毫米波经过手势动作调制后的反射波；
    基于两类时间阵列和多普勒估计，对所述第一毫米波进行处理，得到所述第一毫米波对应的至少一组信号特征值，所述至少一组信号特征值中的每一组信号特征值对应所述第一毫米波中的一帧信号；
    利用标准特征值与控制指令对应关系库，对所述至少一组信号特征值进行识别，得到所述手势动作对应的第一控制指令；
    利用所述第一控制指令，控制第一应用实现相应的功能。
  2. 根据权利要求1所述的方法，其特征在于，所述基于两类时间阵列和多普勒估计，对所述第一毫米波进行处理，得到所述第一毫米波对应的至少一组信号特征值，包括：
    基于所述两类时间阵列，对所述第一毫米波进行处理，得到所述手势动作对应的动作特征，所述动作特征表征所述手势动作的位移信息；
    基于所述多普勒估计，从所述动作特征中提取到所述至少一组信号特征值，所述至少一组信号特征值中的每一组信号特征值对应所述动作特征表征中的一帧信号。
  3. 根据权利要求2所述的方法，其特征在于，所述两类时间阵列包括快速时间阵列和慢速时间阵列；所述基于两类时间阵列，对所述第一毫米波进行预处理，得到所述手势动作对应的动作特征，包括：
    将所述第一毫米波处理成至少一个波束，所述至少一个波束中每个波束对应在一个接收时间点所接收的第一毫米波；
    在所述快速时间阵列中，获取所述至少一个波束对应的至少一个第一信息，所述至少一个第一信息表征所述至少一个波束对应的至少一个频率；
    在所述慢速时间阵列中，根据所述至少一个第一信息，确定出第二信息，所述第二信息表征所述至少一个波束之间的频率变化；
    将所述第二信息确定为所述动作特征。
  4. 根据权利要求2所述的方法，其特征在于，所述基于所述多普勒估计，从所述动作特征中提取到所述至少一组信号特征值，包括：
    利用多普勒效应，依次从所述动作特征中提取到所述一帧信号对应的速度信息和多普勒频移信息；
    利用调频连续波原理，依次从所述动作特征中提取到所述一帧信号对应的距离信息；
    至少将所述速度信息、所述多普勒频移信息和所述距离信息，确定为所述一帧信号对应的所述一组信号特征值；
    将每一帧信号对应的每一组信号特征值，组成所述至少一帧信号对应的所述至少一组信号特征值。
  5. 根据权利要求1所述的方法，其特征在于，所述方法还包括：
    获取标准手势动作对应的预设数目的标准帧信号；
    确定所述预设数目的标准帧信号对应的预设数目的标准信号特征值；
    利用预设神经网络，对所述预设数目的标准信号特征值进行学习，得到所述标准特征值与控制指令对应关系库。
  6. 根据权利要求1所述的方法，其特征在于，所述通过毫米波装置接收手势动作返回的第一毫米波，包括：
    通过所述毫米波装置接收反射信号；
    利用波束合成算法，将所述反射信号合成反射波；
    去除所述反射波的杂波信号和噪声信号，得到所述第一毫米波。
  7. 根据权利要求1或5所述的方法，其特征在于，所述标准特征值与控制指令对应关系库为经过预设神经网络学习得到的关系库。
  8. 一种终端，其特征在于，所述终端包括：处理器、接收器、存储器及通信总线，所述终端上设置有毫米波装置，所述接收器，用于通过所述毫米波装置接收第一毫米波，所述第一毫米波为通过所述毫米波装置发射的第二毫米波经过手势动作调制后的反射波；所述处理器用于执行所述存储器中存储的运行程序，以实现以下步骤：
    基于两类时间阵列和多普勒估计，对所述第一毫米波进行处理，得到所述第一毫米波对应的至少一组信号特征值，所述至少一组信号特征值中的每一组信号特征值对应所述第一毫米波中的一帧信号；利用标准特征值与控制指令对应关系库，对所述至少一组信号特征值进行识别，得到所述手势动作对应的第一控制指令；利用所述第一控制指令，控制第一应用实现相应的功能。
  9. 根据权利要求8所述的终端，其特征在于，
    所述处理器，还用于基于所述两类时间阵列，对所述第一毫米波进行处理，得到所述手势动作对应的动作特征，所述动作特征表征所述手势动作的位移信息；基于所述多普勒估计，从所述动作特征中提取到所述至少一组信号特征值，所述至少一组信号特征值中的每一组信号特征值对应所述动作特征表征中的一帧信号。
  10. 根据权利要求9所述的终端，其特征在于，所述两类时间阵列包括快速时间阵列和慢速时间阵列；
    所述处理器，还用于将所述第一毫米波处理成至少一个波束，所述至少一个波束中每个波束对应在一个接收时间点所接收的第一毫米波；在所述快速时间阵列中，获取所述至少一个波束对应的至少一个第一信息，所述至少一个第一信息表征所述至少一个波束对应的至少一个频率；在所述慢速时间阵列中，根据所述至少一个第一信息，确定出第二信息，所述第二信息表征所述至少一个波束之间的频率变化；将所述第二信息确定为所述动作特征。
  11. 根据权利要求9所述的终端，其特征在于，
    所述处理器，还用于利用多普勒效应，依次从所述动作特征中提取到所述一帧信号对应的速度信息和多普勒频移信息；利用调频连续波原理，依次从所述动作特征中提取到所述一帧信号对应的距离信息；至少将所述速度信息、所述多普勒频移信息和所述距离信息，确定为所述一帧信号对应的所述一组信号特征值；将每一帧信号对应的每一组信号特征值，组成所述至少一帧信号对应的至少一组信号特征值。
  12. 根据权利要求8所述的终端，其特征在于，
    所述处理器，还用于获取标准手势动作对应的预设数目的标准帧信号；确定所述预设数目的标准帧信号对应的预设数目的标准信号特征值；利用预设神经网络，对所述预设数目的标准信号特征值进行学习，得到所述标准特征值与控制指令对应关系库。
  13. 根据权利要求8所述的终端，其特征在于，
    所述处理器，还用于通过所述毫米波装置接收反射信号；利用波束合成算法，将所述反射信号合成反射波；去除所述反射波的杂波信号和噪声信号，得到所述第一毫米波。
  14. 根据权利要求8或12所述的终端，其特征在于，所述标准特征值与控制指令对应关系库为经过预设神经网络学习得到的关系库。
  15. 一种存储介质，其上存储有计算机程序，应用于终端，该计算机程序被处理器执行时实现如权利要求1-7任一项所述的方法。
PCT/CN2018/103362 2018-08-30 2018-08-30 一种手势识别方法及终端、存储介质 WO2020042121A1 (zh)

Priority Applications (4)

Application Number Priority Date Filing Date Title
EP18923764.7A EP3640674B1 (en) 2018-08-30 2018-08-30 Gesture recognition method, terminal, and storage medium
PCT/CN2018/103362 WO2020042121A1 (zh) 2018-08-30 2018-08-30 一种手势识别方法及终端、存储介质
CN201880038905.4A CN110799927B (zh) 2018-08-30 2018-08-30 一种手势识别方法及终端、存储介质
US16/826,073 US11061115B2 (en) 2018-08-30 2020-03-20 Method for gesture recognition, terminal, and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2018/103362 WO2020042121A1 (zh) 2018-08-30 2018-08-30 一种手势识别方法及终端、存储介质

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US16/826,073 Continuation US11061115B2 (en) 2018-08-30 2020-03-20 Method for gesture recognition, terminal, and storage medium

Publications (1)

Publication Number Publication Date
WO2020042121A1 true WO2020042121A1 (zh) 2020-03-05

Family

ID=69426526

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2018/103362 WO2020042121A1 (zh) 2018-08-30 2018-08-30 一种手势识别方法及终端、存储介质

Country Status (4)

Country Link
US (1) US11061115B2 (zh)
EP (1) EP3640674B1 (zh)
CN (1) CN110799927B (zh)
WO (1) WO2020042121A1 (zh)

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112130133A (zh) * 2020-08-26 2020-12-25 南京熊猫电子制造有限公司 一种基于毫米波雷达的人机交互方法
CN112507955A (zh) * 2020-12-21 2021-03-16 西南交通大学 一种婴儿手部精细动作识别方法及系统
CN112782662A (zh) * 2021-01-30 2021-05-11 湖南森鹰智造科技有限公司 一种动态手势识别监测设备
CN113093905A (zh) * 2021-03-31 2021-07-09 惠州华阳通用电子有限公司 一种应用程序启动控制方法及装置
US11061115B2 (en) 2018-08-30 2021-07-13 Guangdong Oppo Mobile Telecommunications Corp., Ltd. Method for gesture recognition, terminal, and storage medium
CN113177581A (zh) * 2021-04-16 2021-07-27 杭州环木信息科技有限责任公司 一种基于毫米波感知的液晶屏幕内容远程感知方法

Families Citing this family (18)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111401166A (zh) * 2020-03-06 2020-07-10 中国科学技术大学 基于肌电信息解码的鲁棒手势识别方法
CN111427031B (zh) * 2020-04-09 2021-12-07 浙江大学 一种基于雷达信号的身份及手势识别方法
CN111652040A (zh) * 2020-04-14 2020-09-11 广西万云科技有限公司 一种基于射频及信号自适应卷积神经网络进行人体活动识别的方法与系统
CN112230213B (zh) * 2020-08-26 2024-02-02 北京航空航天大学杭州创新研究院 毫米波雷达及其信号处理方法、装置、计算机可读存储介质
CN112034446A (zh) * 2020-08-27 2020-12-04 南京邮电大学 一种基于毫米波雷达的手势识别系统
CN112068127B (zh) * 2020-09-14 2024-03-29 上海栩讷科技有限公司 一种基于高频毫米波雷达的无接触控制方法
CN112069483A (zh) * 2020-09-14 2020-12-11 中国科学技术大学 一种智能可穿戴设备的用户识别认证方法
CN112565973B (zh) * 2020-12-21 2023-08-01 Oppo广东移动通信有限公司 终端、终端控制方法、装置及存储介质
CN112741618B (zh) * 2020-12-22 2022-03-22 浙江大学 一种基于fmcw雷达的舌姿检测系统与方法
CN114661142A (zh) * 2020-12-22 2022-06-24 华为技术有限公司 一种手势识别方法以及装置
CN112987926B (zh) * 2021-03-05 2023-01-24 佛山科学技术学院 一种凌空手势轨迹重构方法及系统
CN112965639A (zh) * 2021-03-17 2021-06-15 北京小米移动软件有限公司 手势识别方法及装置、电子设备、存储介质
CN113031777A (zh) * 2021-03-26 2021-06-25 深圳市华杰智通科技有限公司 基于手势识别的密码输入控制装置
CN113406610B (zh) * 2021-06-16 2023-06-23 深圳大学 一种目标检测方法、装置、设备和存储介质
WO2022266931A1 (zh) * 2021-06-24 2022-12-29 京东方科技集团股份有限公司 交互控制装置和交互系统
CN113900513B (zh) * 2021-09-26 2022-10-11 上海交通大学 基于可见光信号的隔空手势控制方法
CN116027919B (zh) * 2022-07-27 2023-10-31 荣耀终端有限公司 触摸屏的控制方法、电子设备、芯片及存储介质
CN116559818B (zh) * 2023-07-04 2023-09-12 南昌大学 一种人体姿态识别方法、系统、计算机及可读存储介质

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105094298A (zh) * 2014-05-13 2015-11-25 华为技术有限公司 终端以及基于该终端的手势识别方法
CN107961035A (zh) * 2017-12-29 2018-04-27 深圳开立生物医疗科技股份有限公司 一种超声探头以及控制超声诊断仪的方法和装置
CN108200706A (zh) * 2018-01-25 2018-06-22 宁波隔空智能科技有限公司 一种基于微波雷达手势识别技术的照明灯具及其控制方法
CN108361780A (zh) * 2018-01-25 2018-08-03 宁波隔空智能科技有限公司 基于微波雷达手势识别技术的抽油烟机控制器及控制方法

Family Cites Families (20)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20100202656A1 (en) * 2009-02-09 2010-08-12 Bhiksha Raj Ramakrishnan Ultrasonic Doppler System and Method for Gesture Recognition
US9335825B2 (en) * 2010-01-26 2016-05-10 Nokia Technologies Oy Gesture control
US9557413B2 (en) * 2010-12-06 2017-01-31 The University Of Memphis Research Foundation Surveillance and tracking system and method
EP2986997A4 (en) * 2013-04-18 2017-02-08 California Institute of Technology Life detecting radars
WO2015184406A1 (en) * 2014-05-30 2015-12-03 Texas Tech University System Hybrid fmcw-intererometry radar for positioning and monitoring and methods of using the same
US9939524B2 (en) * 2015-02-04 2018-04-10 Honeywell International Inc. Systems and methods for measuring velocity with a radar altimeter
US10481696B2 (en) * 2015-03-03 2019-11-19 Nvidia Corporation Radar based user interface
CN107466389B (zh) * 2015-04-30 2021-02-12 谷歌有限责任公司 用于确定类型不可知的rf信号表示的方法和装置
US9568595B2 (en) * 2015-06-29 2017-02-14 Echocare Technologies Ltd. Ultra-wide band antenna arrays and related methods in personal emergency response systems
CN105306819B (zh) 2015-10-15 2018-09-04 广东欧珀移动通信有限公司 一种基于手势控制拍照的方法及装置
CN105718064A (zh) * 2016-01-22 2016-06-29 南京大学 基于超声波的手势识别系统与方法
US10917767B2 (en) * 2016-03-31 2021-02-09 Intel Corporation IOT device selection
CN106446801B (zh) * 2016-09-06 2020-01-07 清华大学 基于超声主动探测的微手势识别方法及系统
US10466772B2 (en) * 2017-01-09 2019-11-05 Infineon Technologies Ag System and method of gesture detection for a remote device
CN107132913A (zh) * 2017-03-31 2017-09-05 南京理工大学 一种基于毫米波的多用户人机交互方法
CN107024685A (zh) * 2017-04-10 2017-08-08 北京航空航天大学 一种基于距离‑速度特征的手势识别方法
CN108344995A (zh) * 2018-01-25 2018-07-31 宁波隔空智能科技有限公司 一种基于微波雷达技术的手势识别装置及手势识别方法
US10576328B2 (en) * 2018-02-06 2020-03-03 Infineon Technologies Ag System and method for contactless sensing on a treadmill
US10775493B2 (en) * 2018-03-28 2020-09-15 Infineon Technologies Ag System and method for controlling access to a trunk of a vehicle using a radar sensor
WO2020042121A1 (zh) 2018-08-30 2020-03-05 Oppo广东移动通信有限公司 一种手势识别方法及终端、存储介质

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105094298A (zh) * 2014-05-13 2015-11-25 华为技术有限公司 终端以及基于该终端的手势识别方法
CN107961035A (zh) * 2017-12-29 2018-04-27 深圳开立生物医疗科技股份有限公司 一种超声探头以及控制超声诊断仪的方法和装置
CN108200706A (zh) * 2018-01-25 2018-06-22 宁波隔空智能科技有限公司 一种基于微波雷达手势识别技术的照明灯具及其控制方法
CN108361780A (zh) * 2018-01-25 2018-08-03 宁波隔空智能科技有限公司 基于微波雷达手势识别技术的抽油烟机控制器及控制方法

Cited By (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11061115B2 (en) 2018-08-30 2021-07-13 Guangdong Oppo Mobile Telecommunications Corp., Ltd. Method for gesture recognition, terminal, and storage medium
CN112130133A (zh) * 2020-08-26 2020-12-25 南京熊猫电子制造有限公司 一种基于毫米波雷达的人机交互方法
CN112507955A (zh) * 2020-12-21 2021-03-16 西南交通大学 一种婴儿手部精细动作识别方法及系统
CN112507955B (zh) * 2020-12-21 2023-04-18 西南交通大学 一种婴儿手部精细动作识别方法及系统
CN112782662A (zh) * 2021-01-30 2021-05-11 湖南森鹰智造科技有限公司 一种动态手势识别监测设备
CN113093905A (zh) * 2021-03-31 2021-07-09 惠州华阳通用电子有限公司 一种应用程序启动控制方法及装置
CN113093905B (zh) * 2021-03-31 2024-05-07 惠州华阳通用电子有限公司 一种应用程序启动控制方法及装置
CN113177581A (zh) * 2021-04-16 2021-07-27 杭州环木信息科技有限责任公司 一种基于毫米波感知的液晶屏幕内容远程感知方法

Also Published As

Publication number Publication date
EP3640674A4 (en) 2020-08-26
US11061115B2 (en) 2021-07-13
EP3640674A1 (en) 2020-04-22
US20200217930A1 (en) 2020-07-09
CN110799927A (zh) 2020-02-14
EP3640674B1 (en) 2023-11-01
CN110799927B (zh) 2022-02-15

Similar Documents

Publication Publication Date Title
WO2020042121A1 (zh) 一种手势识别方法及终端、存储介质
Hayashi et al. RadarNet: Efficient gesture recognition technique utilizing a miniature radar sensor
JP7296415B2 (ja) レーダーシステムを備えるスマートフォン、システムおよび方法
CN110765974B (zh) 基于毫米波雷达和卷积神经网络的微动手势识别方法
Ling et al. Ultragesture: Fine-grained gesture sensing and recognition
US10817070B2 (en) RF-based micro-motion tracking for gesture tracking and recognition
CN110741385B (zh) 手势识别的方法和设备、定位追踪的方法和设备
CN107466389B (zh) 用于确定类型不可知的rf信号表示的方法和装置
WO2021218753A1 (zh) 一种手势识别方法以及相关装置
US10921866B2 (en) Access to high frame-rate radar data via a circular buffer
Zhang et al. A survey of mmWave-based human sensing: Technology, platforms and applications
Wu et al. Dynamic hand gesture recognition using FMCW radar sensor for driving assistance
US20230333209A1 (en) Gesture recognition method and apparatus
CN113064483A (zh) 一种手势识别方法以及相关装置
Cai et al. Efficient convolutional neural network for fmcw radar based hand gesture recognition
Ali et al. End-to-end dynamic gesture recognition using mmWave radar
Kong et al. Gesture recognition system based on ultrasonic FMCW and ConvLSTM model
Regani et al. Handwriting tracking using 60 GHz mmWave radar
WO2022031299A1 (en) Smart-device-based radar system performing angular position estimation
Chmurski et al. Novel Radar-based Gesture Recognition System using Optimized CNN-LSTM Deep Neural Network for Low-power Microcomputer Platform.
Wu et al. Towards natural virtual mouse with mm-wave radar
Lan et al. A low cost solution of hand gesture recognition using a three-dimensional radar array
Daim et al. Hand Gesture Speed Recognition and Classification using IR-UWB Radar Sensor
Qiu et al. A Survey of Gesture Recognition Using Frequency Modulated Continuous Wave Radar
Dong et al. Radar gesture recognition based on lightweight convolutional neural network

Legal Events

Date Code Title Description
ENP Entry into the national phase

Ref document number: 2018923764

Country of ref document: EP

Effective date: 20200103

NENP Non-entry into the national phase

Ref country code: DE