WO2018076371A1 - Procédé de reconnaissance de geste, procédé d'apprentissage de réseau, appareil et équipement - Google Patents

Procédé de reconnaissance de geste, procédé d'apprentissage de réseau, appareil et équipement Download PDF

Info

Publication number
WO2018076371A1
WO2018076371A1 PCT/CN2016/104121 CN2016104121W WO2018076371A1 WO 2018076371 A1 WO2018076371 A1 WO 2018076371A1 CN 2016104121 W CN2016104121 W CN 2016104121W WO 2018076371 A1 WO2018076371 A1 WO 2018076371A1
Authority
WO
WIPO (PCT)
Prior art keywords
data
motion
action
image
frequency domain
Prior art date
Application number
PCT/CN2016/104121
Other languages
English (en)
Chinese (zh)
Inventor
崔健
Original Assignee
深圳市大疆创新科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 深圳市大疆创新科技有限公司 filed Critical 深圳市大疆创新科技有限公司
Priority to PCT/CN2016/104121 priority Critical patent/WO2018076371A1/fr
Priority to CN201680029871.3A priority patent/CN107735796A/zh
Publication of WO2018076371A1 publication Critical patent/WO2018076371A1/fr

Links

Images

Classifications

    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
    • G05D1/0011—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots associated with a remote control arrangement
    • G05D1/0016—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots associated with a remote control arrangement characterised by the operator's input device
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/20—Movements or behaviour, e.g. gesture recognition
    • G06V40/28—Recognition of hand or arm movements, e.g. recognition of deaf sign language
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
    • G05D1/0011—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots associated with a remote control arrangement
    • G05D1/0033—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots associated with a remote control arrangement by having the operator tracking the vehicle either by direct line of sight or via one or more cameras located remotely from the vehicle
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
    • G05D1/10—Simultaneous control of position or course in three dimensions
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting

Definitions

  • the present application relates to the field of communications technologies, and in particular, to a motion recognition method, a network training method, an apparatus, and a device.
  • the current gesture recognition is identified by identifying the start point and the end point of the data waveform. In this mode, if the user does not make gestures, there may be various motions, and it is difficult to distinguish them. The start point and the end point result in error or unrecognizable gesture recognition, and the accuracy and reliability of gesture recognition are low.
  • the embodiment of the invention provides a motion recognition method, a network training method, a device and a device, which can improve the accuracy and reliability of recognizing a user's gesture action.
  • an embodiment of the present invention provides a motion recognition apparatus, including: an acquisition module and a processing module;
  • An acquiring module configured to acquire motion data detected by an external device for the current motion
  • a processing module configured to convert the motion data acquired by the acquiring module into frequency domain data, and use the frequency domain data to identify an action corresponding to the frequency domain data.
  • the embodiment of the present invention further provides a motion recognition apparatus, including: a first acquisition module, a second acquisition module, a fusion module, and a processing module;
  • a first acquiring module configured to acquire motion data detected by the external device for the current motion, and acquire feature data corresponding to the current motion according to the motion data
  • a second acquiring module configured to acquire an image collected for the current action, and process the image to obtain image recognition data corresponding to the current action
  • a fusion module configured to fuse feature data corresponding to the current action and image recognition data to obtain fusion data
  • a processing module configured to identify an action corresponding to the merged data.
  • the embodiment of the present invention further provides a network training device based on motion recognition, including: a first acquiring module, a second acquiring module, a determining module, and a processing module;
  • a first acquiring module configured to acquire motion data detected by an external device for a preset motion
  • a second acquiring module configured to acquire an image collected for the preset action, and process the image to obtain image recognition data corresponding to the preset action
  • a determining module configured to identify an action corresponding to the motion data
  • a processing module configured to perform supervised learning on the image identification data acquired by the second obtaining module by using the action identified by the determining module, and perform pre-determined network model based on the image recognition data after the supervised learning training.
  • the embodiment of the present invention further provides a motion recognition method, including:
  • an embodiment of the present invention further provides a motion recognition method, including:
  • the embodiment of the present invention further provides a network training method based on motion recognition, including:
  • the image recognition data is supervised and learned by the identified action, and the preset network model is trained based on the supervised learning image recognition data.
  • the embodiment of the present invention further provides a motion recognition device, including: a processor and a communication interface, where the processor is connected to the communication interface;
  • the communication interface is configured to acquire motion data detected by an external device for a current motion
  • the processor is configured to convert the motion data into frequency domain data, and use the frequency domain data to identify an action corresponding to the frequency domain data.
  • the embodiment of the present invention further provides a motion recognition device, including: a processor, a communication interface, and an image acquisition device, where the processor is respectively connected to the image acquisition device and the communication interface, where
  • the communication interface is configured to acquire motion data detected by an external device for a current motion
  • the image obtaining device is configured to collect an image for the current motion
  • the processor is configured to acquire an image acquired by the image acquiring device for the current action, and process an image acquired by the image acquiring device to obtain image recognition data corresponding to the current action, according to the motion
  • the data is acquired by the feature data corresponding to the current action, and the feature data corresponding to the current action and the image recognition data are merged to obtain the merged data, and the action corresponding to the merged data is identified.
  • the embodiment of the present invention further provides a network training device based on motion recognition, comprising: an image acquiring device, a processor, and a communication interface, wherein the processor is respectively connected to the image acquiring device and the communication interface ,among them,
  • the communication interface is configured to acquire motion data detected by an external device for a preset motion
  • the image obtaining device is configured to collect an image for the preset action
  • the processor is configured to acquire an image acquired by the image acquiring device for the preset action, and process an image acquired by the image acquiring device to obtain image recognition data corresponding to the preset action; The action corresponding to the motion data; performing supervised learning on the image recognition data by using the identified action, and training the preset network model based on the image recognition data after the supervised learning.
  • an embodiment of the present invention further provides an aircraft, including
  • the motion recognition device for identifying an action.
  • an embodiment of the present invention further provides an aircraft, including
  • the motion recognition device for identifying an action.
  • an embodiment of the present invention further provides an aircraft, including
  • the motion recognition-based network training device is for training a network model for motion recognition.
  • the embodiment of the present invention can obtain the motion corresponding to the frequency domain data by using the motion data of the external device, and convert the motion data into the frequency domain data, or by using the motion data and the image identification data. Integrating to obtain the fused data, thereby using the fused data to identify an action corresponding to the fused data, or by determining an action corresponding to the motion data, using the image recognition data and the determined action to supervise the image recognition data to enhance The accuracy and reliability of motion recognition are good.
  • FIG. 1 is a schematic diagram of an aircraft control system according to an embodiment of the present invention.
  • FIG. 2 is a schematic flowchart of a motion recognition method according to an embodiment of the present invention.
  • FIG. 3 is a schematic flowchart of another motion recognition method according to an embodiment of the present invention.
  • FIG. 4 is a schematic flowchart of a network training method based on motion recognition according to an embodiment of the present invention
  • FIG. 5 is a schematic structural diagram of a motion recognition apparatus according to an embodiment of the present invention.
  • FIG. 6 is a schematic structural diagram of another motion recognition apparatus according to an embodiment of the present invention.
  • FIG. 7 is a schematic structural diagram of a network training apparatus based on motion recognition according to an embodiment of the present invention.
  • FIG. 8 is a schematic structural diagram of a motion recognition device according to an embodiment of the present invention.
  • FIG. 9 is a schematic structural diagram of another motion recognition apparatus according to an embodiment of the present invention.
  • FIG. 10 is a schematic structural diagram of a network training device based on motion recognition according to an embodiment of the present invention.
  • FIG. 1 provides a schematic diagram of an aircraft control system including an aircraft such as a drone 110 and a wearable device 120, wherein the drone 110 includes a flight body, a pan/tilt head, and an imaging device 130.
  • the flying body includes a plurality of rotors and a rotor motor that drives the rotor to rotate, thereby providing the power required for the drone 110 to fly.
  • the imaging device 130 is mounted on the flying body through the pan/tilt.
  • the imaging device 130 is used for image or video capture during flight of the drone 110, including but not limited to multi-spectral imagers, hyperspectral imagers, visible light cameras, infrared cameras, and the like.
  • the pan/tilt is a multi-axis transmission and stabilization system, including multiple rotating shafts and pan/tilt motors.
  • the pan/tilt motor compensates for the photographing angle of the image forming apparatus 130 by adjusting the rotation angle of the rotating shaft, and prevents or reduces the shake of the image forming apparatus 130 by setting an appropriate buffer mechanism.
  • imaging device 130 can be mounted on the flying body either directly or by other means.
  • the wearable device 120 is worn by the operator and communicates with the drone 110 by wireless communication, thereby setting the flight process and imaging of the drone 110.
  • the photographing process of the preparation 130 is controlled.
  • the wearable device 120 has a built-in motion sensor.
  • the motion sensor senses the movement of the hand and outputs corresponding motion data, and according to the movement.
  • the data controls the drone accordingly.
  • the imaging device of the device on the drone can also capture the image data of the human body motion, and recognize the limb motion according to the image data, and the aircraft can be correspondingly controlled according to the recognized limb motion.
  • the embodiment of the invention discloses a motion recognition method, a network training method based on the motion recognition method, a related device and related equipment, which can improve the accuracy and reliability of the motion recognition, and has good robustness. The details are explained below.
  • FIG. 2 is a schematic flowchart diagram of a motion recognition method according to an embodiment of the present invention. Specifically, as shown in FIG. 2, the motion recognition method in the embodiment of the present invention may include the following steps:
  • the technical solution of the embodiment of the present invention may be specifically applied to an external device, or may be specifically applied to a controlled device corresponding to the motion recognition, such as an aircraft, or may be specifically applied to other independent motion recognition devices, and the present invention
  • the embodiment is not limited.
  • the external device may be a wearable device or a handheld device, such as a wristband, a watch, a smart ring, etc., and the external device is configured with a motion sensor such as an Inertial Measurement Unit (IMU), when the external device When moving or making an action such as a gesture, the motion sensor configured by the external device outputs corresponding motion data, and the external device detects the motion data, wherein the motion data may be one or both of angular acceleration and acceleration.
  • IMU Inertial Measurement Unit
  • the using the frequency domain data to identify an action corresponding to the frequency domain data may be specifically: inputting the frequency domain data into a network model, to identify and the frequency domain by using the network model The action corresponding to the data.
  • the feature of the frequency domain data may be further extracted, for example, the feature of the frequency domain data is obtained by extracting and superimposing, and the extracted feature is input into a network model to identify and The action corresponding to the feature to improve the efficiency of the action recognition.
  • the frequency domain data may be obtained by subjecting the motion data to Fourier transform.
  • the network model may be a neural network, or other network models. The embodiment of the present invention uses a neural network as an example for description.
  • the acquired motion data and/or the frequency domain data may also be normalized.
  • the current gesture recognition is based on time domain data for identification, and since the gesture actions performed within a certain time cannot be aligned to the same length of time, it is difficult to distinguish various gestures from the time domain waveform, and each person There is a big difference in doing the same gesture data. Therefore, the recognition effect is poor when the gesture motion recognition is directly performed in the time domain.
  • the embodiment of the present invention uses the frequency domain identification method to perform motion recognition. Since the frequency domain has no time axis, the gestures are aligned, and the various frequency components of each person making the same gesture are very similar, which greatly improves the gesture. The recognition effect of motion recognition.
  • the motion data acquired by the external device tends to be noisy, and the frequency range of the motion data is very low, resulting in inaccurate recognition of gesture gestures corresponding to the motion data.
  • the obtained original motion data can be subjected to low-pass filtering processing with a relatively low frequency band, and the data amount of the acquired motion data can be reduced by reducing the sampling rate of the motion data, thereby reducing the computational cost of the algorithm.
  • the acquired frequency domain data may be normalized to facilitate the network model to perform data processing.
  • the motion data may include data obtained by sampling data output by the motion sensor of the external device within a preset time period; or the motion data may also include data outputted by the motion sensor of the external device.
  • the obtained data is obtained by sampling a preset number of times.
  • the data output by the motion sensor can be collected according to a preset time period. For example, after a large number of tests, it is found that the gesture of the person generally does not exceed 5 s, and the network model can be continuously performed on the motion data collected every 5 s. Train or come to gesture gesture recognition.
  • the embodiment of the invention performs the gesture recognition by converting the motion data into the frequency domain data, that is, the entire segment of the data recognition gesture, and does not identify the start point and the end point of the gesture within 5s, which improves the accuracy of the motion recognition. Sex.
  • the motion data detected by the external device for the preset action may also be acquired; the motion data corresponding to the preset action is converted into frequency domain data, and the frequency domain corresponding to the preset action is utilized.
  • the data and the preset action train the network model.
  • the network model is trained before using the network model to identify the action.
  • several stable gesture actions can be defined in advance, and different users can perform these gesture actions, and collect motion data obtained by a large number of different users to perform these gestures (such as data output by the watch IMU), and convert them.
  • the frequency domain data is utilized to train a network model such as a neural network.
  • the characteristics of the frequency domain data may be further extracted, for example, by extracting, superimposing, etc.
  • the characteristics of the frequency domain data, and the extracted features are taken as inputs, and the preset action is used as an output to train the network model such as a neural network, thereby improving the stability and reliability of the network model through a large amount of network training. And improve the reliability of motion recognition based on the network model.
  • a regularization process may be employed on the motion data and/or the frequency domain data during training to reduce overfitting.
  • the network model is trained by various gesture actions performed by the user using an external device, thereby improving the recognition rate of the gesture action and reducing the false detection rate of the action.
  • the aircraft may also be controlled according to the identified action.
  • the control instruction corresponding to the gesture action may be generated, and the control instruction is sent to the controlled device, such as an aircraft, to enable the aircraft to perform an operation corresponding to the gesture action.
  • a plurality of gestures such as a gesture action 1, a gesture action 2, a gesture action 3, and the like, may be predefined, and a control function corresponding to each gesture action may be further preset to control the corresponding controlled device.
  • a control function corresponding to each gesture action may be further preset to control the corresponding controlled device.
  • the external device as the watch and taking the control aircraft as an example, it is assumed that when the user wears the hand of the watch to perform the gesture action 1, the aircraft can automatically start and take off; when the aircraft is in flight, the user can perform the gesture action 2, the aircraft can enter the surround Selfie function.
  • a mode switching control between the plurality of self-timer modes by the gesture action 1 may be set, and in the self-timer mode, the user may perform the gesture action 2 again to exit the self-timer mode, and the like.
  • the user simply raises the finger with the watch toward the aircraft and performs a gesture action 1 at which time the aircraft can enter which mode to fly, and the aircraft takes the user as the center of the ball.
  • the aircraft is currently flying in a spherical plane with a radius of a person, and the aircraft flies according to the position pointed by the user's finger.
  • the aircraft can adjust the flight radius, and the aircraft can automatically detect whether the user is pointing to the ground and safely control the height to prevent squatting.
  • the user can perform a gesture action 1 by a preset gesture action, such as pointing to the aircraft, and exiting which mode.
  • a gesture action 3 to control the aircraft to make a safe landing. This enables flight control of the aircraft without the need to use the remote control for control, enhancing the user experience.
  • the motion data of the external device is acquired, and the motion data is converted into the frequency domain data, so that the frequency domain data is used to identify the action corresponding to the frequency domain data, which is a good avoidance.
  • the false detection rate is low, and the accuracy and reliability of motion recognition are further improved, and the robustness is good.
  • FIG. 3 is a schematic flowchart diagram of another motion recognition method according to an embodiment of the present invention. Specifically, as shown in FIG. 3, the motion recognition method in the embodiment of the present invention may include the following steps:
  • the technical solution of the embodiment of the present invention may be specifically applied to the controlled device corresponding to the motion recognition, such as an aircraft, or may be specifically applied to other independent motion recognition devices, which is not limited in the embodiment of the present invention.
  • the external device may be a wearable device or a handheld device such as a wristband, a watch, a smart ring, or the like.
  • the acquired motion data may be data collected by a motion sensor such as an IMU disposed in the external device.
  • the feature data may refer to frequency domain data converted from the acquired motion data; or may be obtained by converting the obtained motion data into frequency domain data, and further extracting the frequency data.
  • the characteristics of the domain data such as by extraction, superposition, etc., obtain the characteristics of the frequency domain data.
  • the image identification data may be an image acquisition device such as a camera, and specifically may be a camera disposed on the aircraft, and the image data obtained by detecting the current motion (ie, capturing an image corresponding to the motion), and the image is The data obtained by processing the data.
  • the fusion data includes features of the motion data and features in the image recognition data, so that features of the data for performing motion recognition can be added, thereby avoiding actions (such as gestures) when only the image recognition action is performed.
  • the user of the action is not in the image acquired by the image acquisition device, or the user is small in the image, causing the motion recognition to be erroneous, or even unrecognizable, or the user cannot perform recognition when the action is recognized by the external device.
  • the identifying the action corresponding to the fused data may be specifically: inputting the fused data into a network model, to identify an action corresponding to the fused data by using the network model.
  • the network model may be a neural network, or may be another network model.
  • the embodiment of the present invention uses a neural network as an example for description.
  • the motion data in the embodiment of the present invention may include data obtained by sampling data output by the motion sensor of the external device in a preset time period; or pre-predicting data output by the motion sensor of the external device.
  • the obtained data is obtained by sampling the number of times.
  • the acquiring the feature data corresponding to the current action according to the motion data may be specifically: converting the motion data into frequency domain data, and acquiring the current action according to the frequency domain data.
  • the merging the feature data and the image identification data corresponding to the current action to obtain the fused data may be specifically: merging the frequency domain data and the image identification data to obtain fused data.
  • the obtained motion data such as IMU data
  • the feature data may be determined based on the frequency domain data, for example, directly using the frequency domain data as Feature data, or feature extraction of the frequency domain data, such as data obtained by processing such as extraction, superposition, etc., as the feature data.
  • the fusion data including the feature data and the image recognition data may be generated, which is equivalent to the characteristics of the two data sources (image recognition data and motion data). Fusion, so that gesture recognition can be performed based on the fusion data, and device control can be performed based on the recognized gesture motion.
  • the motion data detected by the external device for the preset motion may be acquired, and the feature data corresponding to the preset action is acquired according to the motion data; and the image identification data obtained by detecting the preset motion is acquired. And combining the image identification data corresponding to the preset action with the feature data corresponding to the preset action to obtain fusion data corresponding to the preset action; and using the fusion data corresponding to the preset action And training the network model with the preset action.
  • the obtained motion data such as the IMU data
  • the frequency domain data is obtained, and the feature data is determined based on the frequency domain data, for example, directly the frequency domain data.
  • the feature data the frequency domain data may be subjected to feature extraction, such as data obtained by processing such as extraction, superposition, etc., as the feature data.
  • the image learns gestures through deep learning.
  • the middle layer is the learned image recognition data, and the special data from the IMU can be
  • the levy data is inserted into an intermediate layer to obtain the fused data of the two, and continues to learn, which is equivalent to merging the characteristics of the two data sources (image recognition data and motion data).
  • the training is performed so that the gesture motion can be recognized by the integrated motion data even if the image is difficult to recognize the gesture, and the device control can be further performed based on the gesture motion.
  • the aircraft may also be controlled according to the identified action.
  • a plurality of gestures may be predefined, and a control function corresponding to each gesture action may be further preset to control a corresponding controlled device such as an aircraft.
  • a control aircraft as an example, the operation of the aircraft can be controlled by gesture gesture recognition to control the takeoff, landing, self-timer, and which flight, thereby enhancing the user experience.
  • the motion data of the acquired external device and the image identification data may be merged, and specifically, the motion data may be converted into frequency domain data, and the feature data of the current action is determined based on the frequency domain data, and The fusion data is obtained by fusing the feature data with the image identification data, so that the fusion data can be used to identify the action corresponding to the fusion data, thereby improving the accuracy and reliability of the motion recognition, and the robustness is good.
  • the problem that the motion data of only the external device is used for identification, or the motion recognition using only the image recognition is low or even unrecognizable is avoided.
  • FIG. 4 is a schematic flowchart diagram of a network training method based on motion recognition according to an embodiment of the present invention.
  • the network training method in the embodiment of the present invention may include the following steps:
  • the technical solution of the embodiment of the present invention may be specifically applied to the controlled device corresponding to the action identification, such as an aircraft, or may be specifically applied to other independent network training devices, which is not limited in the embodiment of the present invention.
  • the external device may be a wearable device or a handheld device such as a wristband, a watch, a smart ring, or the like.
  • the acquired motion data may be data collected by a motion sensor such as an IMU disposed in the external device. Further, in the embodiment of the present invention, the acquiring according to the motion data
  • the feature data corresponding to the preset action is also referred to as feature data corresponding to the motion data, and the feature data is used to determine a specific gesture action.
  • the motion data in the embodiment of the present invention may include data obtained by sampling data acquired by the motion sensor of the external device in a preset time period, or preset data output by the motion sensor of the external device. The number of samples is taken to obtain the data.
  • the image identification data may be an image acquisition device such as a camera, and specifically may be a camera disposed on the aircraft, and the image data obtained by detecting the current motion (ie, capturing an image corresponding to the motion), and the image is The data obtained by processing the data.
  • the network model may be a neural network, or may be another network model.
  • the embodiment of the present invention uses a neural network as an example for description.
  • the performing the supervised learning on the image identification data by using the identified action may be specifically: using depth learning, using the image recognition data as an input, and performing the action as a target output.
  • Supervise learning since the feature dimension collected by the image is large, the depth of the image recognition feature dimension can be reduced by deep learning, thereby improving the stability and reliability of the network model based on the image recognition network training after the learning.
  • the network model may be a network model corresponding to the image recognition action, and the network model may be trained by using the action corresponding to the identified motion data and the image recognition data.
  • the motion data feature acquired by the wristband can be pre-trained (may be data obtained by processing, filtering, etc. the motion data, or the motion data may be subjected to Fourier The corresponding relationship between the transformed frequency domain data, and the like) and the gesture action.
  • the motion data and the image recognition data collected by the wristband are synchronously acquired, the characteristics of the motion data are acquired, and the corresponding relationship between the motion data feature and the gesture motion collected by the wristband is recognized, and the currently collected The action corresponding to the motion data feature of the bracelet.
  • deep learning can be used to take image recognition data as input, and the recognized action is the target output to supervise the image recognition data.
  • the network model can be trained by using the reduced dimension image recognition data, and the corresponding relationship between the image recognition data and the gesture action is determined to perform the gesture action in the subsequent manner.
  • the current image recognition data can be recognized by the acquired current image and the current gesture action can be quickly and accurately recognized by the network model to perform device control based on the gesture action.
  • the network model may be a network model corresponding to the manner of the image recognition action based on the motion data recognition action, and the network model may be trained by using the acquired motion data and the fusion data of the image recognition data.
  • the feature data corresponding to the preset action may be acquired according to the motion data.
  • the image recognition data after the supervised learning is used to train the preset network model, which may be specifically: The feature data is merged with the image recognition data after the supervised learning to obtain fusion data; and the preset network model is trained by using the fusion data.
  • the external device is still used as an example of a wristband, and the corresponding relationship between the motion data feature and the gesture motion collected by the wristband can be pre-trained.
  • the motion data and the image recognition data collected by the wristband are synchronously acquired, the characteristics of the motion data are acquired, and the corresponding relationship between the motion data feature and the gesture motion collected by the wristband is recognized, and the currently collected The action corresponding to the motion data feature of the bracelet. Further, the depth learning is used to take the image recognition data as an input, and the recognized motion is the target output to supervise and learn the image recognition data.
  • the network model can be trained by using the reduced image identification data and the feature data of the motion data, that is, using the fusion data of the two to network
  • the model is trained to input the fusion data of the learned current image recognition data acquired for a certain gesture action and the feature data of the current motion data to the network model, so as to be fast and accurate.
  • the current gesture action is recognized, and device control can be further performed based on the gesture action.
  • the acquiring the feature data corresponding to the current action according to the motion data may be specifically: converting the motion data into frequency domain data, and acquiring, according to the frequency domain data, the current action Characteristic data.
  • the feature data is determined based on the frequency domain data, for example, the converted frequency domain data may be directly used as feature data, or the frequency domain data may be subjected to feature extraction, such as after being extracted, superimposed, etc. The data is used as the feature data.
  • the current image identification data may also be obtained based on the current motion collection, and based on the The neural network trained by the above training method recognizes the gesture action corresponding to the current image recognition data, so that the controlled device such as the aircraft needs to be determined based on the pre-defined gesture action and the control function corresponding to each gesture action.
  • the operations performed, such as gesture recognition, can be used to control the aircraft to take off, land, self-timer, and fly, thereby enhancing the user experience.
  • the motion data and the image recognition data are simultaneously collected when the user performs the gesture action, and the current gesture action is recognized based on the correspondence between the motion data and the gesture action, thereby being able to be identified by using the deep learning method.
  • the gesture action supervises and learns the image recognition data, which improves the accuracy and reliability of motion recognition and is robust.
  • FIG. 5 is a schematic structural diagram of a motion recognition apparatus according to an embodiment of the present invention.
  • the motion recognition device in the embodiment of the present invention may be specifically configured in an external device, or may be specifically disposed in a controlled device, such as an aircraft, corresponding to the motion recognition, or may be specifically configured in another independent motion recognition device. ,and many more.
  • the motion recognition apparatus 10 of the embodiment of the present invention may include an acquisition module 11 and a processing module 12. among them,
  • the obtaining module 11 is configured to acquire motion data detected by an external device for the current motion
  • the processing module 12 is configured to convert the motion data acquired by the acquiring module 11 into frequency domain data, and use the frequency domain data to identify an action corresponding to the frequency domain data.
  • the external device may be a wearable device or a handheld device such as a wristband, a watch, a smart ring, or the like.
  • the external device is configured with a motion sensor such as an IMU.
  • the motion sensor configured by the external device outputs corresponding motion data, and the external device detects the motion data.
  • the processing module 12 is specifically configured to input the frequency domain data into a network model to identify an action corresponding to the frequency domain data by using the network model.
  • the processing module 12 may further extract features of the frequency domain data, such as acquiring, acquiring, or superimposing the characteristics of the frequency domain data, and inputting the extracted features into the network.
  • the model identifies the action corresponding to the feature to improve the efficiency of the action recognition.
  • the frequency domain data may be obtained by subjecting the motion data to Fourier transform.
  • the network model may be a neural network, or other network models, which are not limited in the embodiment of the present invention.
  • the obtaining module 11 is further configured to acquire motion data detected by the external device for the preset motion
  • the processing module 12 is further configured to convert the motion data corresponding to the preset action into frequency domain data, and use the frequency domain data corresponding to the preset action and the preset action to the network.
  • the model is trained.
  • the processing module 12 is further configured to perform regularization processing on the frequency domain data to reduce over-fitting of the frequency domain data.
  • the processing module 12 is further configured to perform normalization processing on the motion data.
  • the motion data may include data obtained by sampling data acquired by the motion sensor of the external device in a preset time period, or sampling the data output by the motion sensor of the external device by a preset number of times. The data.
  • the processing module 12 is further configured to control the aircraft according to the identified action.
  • the processing module 12 may further generate a control instruction corresponding to the gesture action, and send the control instruction to the controlled device, such as an aircraft, to enable the aircraft to perform an operation corresponding to the gesture action.
  • the controlled device such as an aircraft
  • the motion data of the external device is acquired, and the motion data is converted into the frequency domain data, so that the frequency domain data is used to identify the action corresponding to the frequency domain data, which is a good avoidance.
  • Open the recognition start and end points of the gesture data waveform improve the recognition rate of the motion recognition, reduce the false detection rate, and further improve the accuracy and reliability of the motion recognition, and the robustness is better.
  • FIG. 6 is a schematic structural diagram of another motion recognition apparatus according to an embodiment of the present invention.
  • the motion recognition device in the embodiment of the present invention may be specifically disposed in a controlled device corresponding to the motion recognition, such as an aircraft, or may be specifically disposed in another independent motion recognition device, etc., specifically, as shown in FIG. 6.
  • the action recognition apparatus 20 of the embodiment of the present invention may include a first acquisition module 21, a second acquisition module 22, a fusion module 23, and a processing module 24. among them,
  • the first obtaining module 21 is configured to acquire motion data detected by the external device for the current motion, and acquire feature data corresponding to the current motion according to the motion data;
  • the second acquiring module 22 is further configured to acquire an image collected for the current action, and process the image to obtain image recognition data corresponding to the current action;
  • the merging module 23 is configured to combine feature data corresponding to the current action and image identification data to obtain fused data.
  • the processing module 24 is configured to identify an action corresponding to the fused data.
  • the external device may be a wearable device or a handheld device such as a wristband, a watch, a smart ring, or the like.
  • the acquired motion data may be data collected by a motion sensor such as an IMU disposed in the external device.
  • the feature data may refer to frequency domain data obtained by converting the acquired motion data (such as transforming by Fourier transform), and may also refer to converting the obtained motion data into frequency.
  • the domain data, and further extracting the characteristics of the frequency domain data for example, by extracting, superimposing, etc., obtaining the characteristics of the frequency domain data.
  • the processing module 24 is specifically configured to input the merged data into a network model to identify an action corresponding to the merged data by using the network model.
  • the network model may be a neural network, or other network models, which are not limited in the embodiment of the present invention.
  • the first obtaining module 21 may be specifically configured to convert the motion data into frequency domain data when acquiring feature data corresponding to the current motion according to the motion data, to use the frequency domain data as Feature data corresponding to the current action;
  • the merging module 23 may be specifically configured to combine the frequency domain data and the image identification data acquired by the second acquiring module 22 to obtain fused data.
  • the first obtaining module 21 is further configured to acquire motion data detected by the external device for the preset action, and acquire feature data corresponding to the preset action according to the motion data;
  • the second acquiring module 22 is further configured to acquire an image collected for the preset action, and process the collected image to obtain image recognition data corresponding to the preset action;
  • the merging module 23 is further configured to combine the image identification data corresponding to the preset action with the feature data corresponding to the preset action to obtain fused data corresponding to the preset action;
  • the processing module 24 is further configured to train the network model by using the fused data corresponding to the preset action and the preset action.
  • the motion data may include data obtained by sampling data acquired by the motion sensor of the external device in a preset time period, or sampling the data output by the motion sensor of the external device by a preset number of times. The data.
  • the processing module 24 is further configured to control the aircraft according to the identified action.
  • the motion data of the acquired external device and the image identification data may be merged, and specifically, the motion data may be converted into frequency domain data, and the feature data of the current action is determined based on the frequency domain data, and The fusion data is obtained by fusing the feature data with the image identification data, so that the fusion data can be used to identify the action corresponding to the fusion data, thereby improving the accuracy and reliability of the motion recognition, and the robustness is good.
  • the problem of low recognition accuracy or even unrecognizable motion recognition caused by using only external device recognition or using only image recognition is avoided.
  • FIG. 7 is a schematic structural diagram of a network training apparatus based on motion recognition according to an embodiment of the present invention.
  • the motion recognition device in the embodiment of the present invention may be specifically disposed in a controlled device, such as an aircraft, corresponding to the motion recognition, or may be specifically configured in another independent network training device.
  • the network training device 30 of the embodiment of the present invention may include a first obtaining module 31, a second obtaining module 32, a determining module 33, and a processing module 34. among them,
  • the first obtaining module 31 is configured to acquire motion data detected by an external device for a preset motion
  • the second acquiring module 32 is further configured to acquire an image collected for the preset action, and process the image to obtain image recognition data corresponding to the preset action;
  • the determining module 33 is configured to identify an action corresponding to the motion data
  • the processing module 34 is configured to perform supervised learning on the image identification data acquired by the second obtaining module 32 by using the action identified by the determining module 33, and based on the image recognition data after the supervised learning Set up a network model for training.
  • the external device may be a wearable device or a handheld device such as a wristband, a watch, a smart ring, or the like.
  • the acquired motion data may be data collected by a motion sensor such as an IMU disposed in the external device.
  • the network model can be a neural network or other network models. The embodiment of the present invention uses a neural network as an example for description.
  • the processing module 34 is specifically configured to use the depth learning to input the image recognition data acquired by the second acquiring module 32, and use the action as a target output to perform supervised learning.
  • the processing module 34 can reduce the image recognition feature dimension through deep learning, so as to improve the stability and reliability of the network model obtained based on the learned image recognition network training.
  • the first obtaining module 31 is further configured to acquire feature data corresponding to the preset action according to the motion data;
  • the processing module 34 is further configured to fuse the feature data with the image data after the supervised learning to obtain the fused data, and use the fused data to train the preset network model.
  • the first obtaining module 31 is specifically configured to: when acquiring feature data corresponding to the current action according to the motion data, specifically, converting the motion data into frequency domain data, to use the frequency domain data As feature data corresponding to the current action.
  • the feature data is determined based on the frequency domain data.
  • the acquiring module 31 may directly use the converted frequency domain data as feature data, or perform feature extraction on the frequency domain data, such as extraction, superposition, and the like.
  • the data obtained after the processing is taken as the feature data.
  • the motion data and the image recognition data are simultaneously collected when the user performs the gesture action, and the current gesture action is recognized based on the correspondence between the motion data and the gesture action, thereby being able to be identified by using the deep learning method.
  • the gesture action supervises and learns the image recognition data, which improves the accuracy and reliability of motion recognition and is robust.
  • the embodiment of the present invention further provides a computer storage medium, wherein the computer storage medium stores program instructions, and the program may include some or all of the steps of the motion recognition method in the corresponding embodiment of FIG. 2 .
  • the embodiment of the invention further provides a computer storage medium, wherein the computer storage medium stores program instructions, and the program execution may include a part of the motion recognition method in the corresponding embodiment of FIG. Or all steps.
  • the embodiment of the present invention further provides a computer storage medium, where the computer storage medium stores program instructions, and the program execution may include some or all steps of the motion recognition based network training method in the corresponding embodiment of FIG. 4 . .
  • FIG. 8 is a schematic structural diagram of a motion recognition device according to an embodiment of the present invention.
  • the motion recognition device in the embodiment of the present invention may be an external device such as a wristband, a watch, a ring, or the like, or may be a controlled device such as an aircraft, or may be another independent motion recognition device, or the like.
  • the motion recognition device 1 in the embodiment of the present invention may include: a communication interface 300, a memory 200, and a processor 100, and the processor 100 may be respectively connected to the communication interface 300 and the memory 200.
  • the motion recognition device 1 may further include a motion sensor, a camera, and the like.
  • the communication interface 300 can include a wired interface, a wireless interface, and the like, and can be used to receive data transmitted by an external device, such as receiving motion data collected by an external device for a certain gesture action of the user, or for transmitting motion data acquired by the external device. and many more.
  • the memory 200 may include a volatile memory such as a random-access memory (RAM); the memory 200 may also include a non-volatile memory such as a flash. A flash memory or the like; the memory 200 may further include a combination of the above types of memories.
  • RAM random-access memory
  • flash non-volatile memory
  • a flash memory or the like; the memory 200 may further include a combination of the above types of memories.
  • the processor 100 may be a central processing unit (CPU), a graphics processing unit (GPU), or the like.
  • the processor may further include a hardware chip.
  • the hardware chip may be an application-specific integrated circuit (ASIC), a programmable logic device (PLD) or a combination thereof.
  • the PLD may be a complex programmable logic device (CPLD) or a field-programmable gate array (FPGA).
  • the memory 200 is further configured to store program instructions.
  • the processor 100 can invoke the program instructions to implement the motion recognition method as shown in the embodiment of FIG. 2 of the present application.
  • the communication interface 300 can be configured to acquire motion data detected by an external device for current motion detection
  • the processor 100 can invoke program instructions stored in the memory 200 for executing:
  • the processor 100 is specifically configured to input the frequency domain data into a network model to identify an action corresponding to the frequency domain data by using the network model.
  • the network model may comprise a neural network.
  • the communication interface 300 is further configured to acquire motion data detected by the external device for the preset action
  • the processor 100 is further configured to convert motion data corresponding to the preset action into frequency domain data, and use the frequency domain data corresponding to the preset action and the preset action on the network.
  • the model is trained.
  • the processor 100 is further configured to perform regularization processing on the frequency domain data to reduce over-fitting of the frequency domain data.
  • the processor 100 is further configured to perform normalization processing on the motion data.
  • the motion data may include data obtained by sampling data acquired by a motion sensor of an external device within a preset time period, or sampling the data output by the motion sensor of the external device by a preset number of times. The data obtained.
  • the processor 100 is further configured to control the aircraft according to the identified action.
  • An embodiment of the present invention further provides an aircraft, including
  • the motion recognition device according to any of the above embodiments of FIG. 8 is configured to recognize an action.
  • FIG. 9 is a schematic structural diagram of another motion recognition device according to an embodiment of the present invention.
  • the motion recognition device in the embodiment of the present invention may be a controlled device such as an aircraft, or may be another independent motion recognition device, and the like.
  • the motion recognition device 2 in the embodiment of the present invention may include: a communication interface 700, an image acquisition device 600, a memory 500, and a processor 400.
  • the processor 400 may be respectively connected to the communication interface 700 and image.
  • the device 600 and the memory 500 are connected.
  • the motion recognition device 2 may further include a motion sensor.
  • the image acquisition device 600 may include a camera for acquiring an image, such as an image when the user performs a gesture.
  • the communication interface 700 can include a wired interface, a wireless interface, and the like, and can be used to receive data transmitted by the external device, such as receiving motion data collected by the external device for a certain gesture action of the user.
  • the memory 500 may include a volatile memory, such as a random-access memory (RAM); the memory 500 may also include a non-volatile memory, such as a flash. A flash memory or the like; the memory 500 may further include a combination of memories of the above kind.
  • RAM random-access memory
  • non-volatile memory such as a flash.
  • flash memory or the like the memory 500 may further include a combination of memories of the above kind.
  • the processor 400 may be a central processing unit (CPU), a graphics processing unit (GPU), or the like.
  • the processor may further include a hardware chip.
  • the hardware chip may be an application-specific integrated circuit (ASIC), a programmable logic device (PLD) or a combination thereof.
  • the PLD may be a complex programmable logic device (CPLD) or a field-programmable gate array (FPGA).
  • the memory 500 is further configured to store program instructions.
  • the processor 400 can invoke the program instructions to implement the motion recognition method as shown in the embodiment of FIG. 3 of the present application.
  • the communication interface 700 is configured to acquire motion data detected by an external device for a current motion.
  • the image obtaining device 600 is configured to collect an image for the current motion
  • the processor 400 is configured to process an image acquired by the image acquiring device, obtain image recognition data, acquire feature data corresponding to the current action according to the motion data, and select a feature corresponding to the current action.
  • the data and the image identification data are fused to obtain fused data, and an action corresponding to the fused data is identified.
  • the processor 400 is specifically configured to input the merged data into a network model to identify an action corresponding to the merged data by using the network model.
  • the network model comprises a neural network.
  • the processor 400 is specifically configured to convert the motion data into frequency domain data, and fuse the frequency domain data and the image identification data to obtain fused data.
  • the obtained motion data such as the IMU data
  • the frequency domain data is obtained, and the feature data is determined based on the frequency domain data, for example, directly
  • the frequency domain data obtained by the conversion is used as feature data, or the frequency domain data is subjected to feature extraction, such as data obtained by processing after extraction, superposition, etc., as the feature data.
  • the communication interface 700 is further configured to acquire motion data detected by the external device for the preset action
  • the image obtaining device 600 is further configured to collect an image for the preset action
  • the processor 400 is further configured to process an image acquired by the image acquiring device to obtain image recognition data corresponding to the preset action, and acquire feature data corresponding to the preset action according to the motion data; Combining the image identification data corresponding to the preset action with the feature data corresponding to the preset action to obtain fusion data corresponding to the preset action; using the fusion data corresponding to the preset action and The preset action trains the network model.
  • the motion data may include data obtained by sampling data acquired by a motion sensor of an external device within a preset time period, or sampling the data output by the motion sensor of the external device by a preset number of times. The data obtained.
  • the processor 400 is further configured to control the aircraft according to the identified action.
  • An embodiment of the present invention further provides an aircraft, including
  • the motion recognition device is for identifying an action.
  • FIG. 10 is a schematic structural diagram of a network training device based on motion recognition according to an embodiment of the present invention.
  • the network training device in the embodiment of the present invention may be a controlled device such as an aircraft, or may be other independent network training devices, and the like.
  • the network training device 3 in the embodiment of the present invention may include: a communication interface 1100, an image acquisition device 1000, a memory 900, and a processor 800, and the processor 800 may be respectively associated with the communication interface 1100.
  • the image acquisition device 1000 and the memory 900 are connected.
  • the motion recognition device 3 may further include a motion sensor or the like.
  • the image acquisition device 1000 may include a camera for acquiring an image, such as an image when the user performs a gesture.
  • the communication interface 1100 can include a wired interface, a wireless interface, and the like, and can be used to receive data transmitted by the external device, such as receiving motion data collected by the external device for a certain gesture action of the user.
  • the memory 900 may include a volatile memory, such as a random-access memory (RAM); the memory 900 may also include a non-volatile memory, such as a flash. A flash memory or the like; the memory 900 may further include a combination of the above types of memories.
  • RAM random-access memory
  • flash non-volatile memory
  • the processor 800 can be a central processing unit (CPU), a graphics processing unit (GPU), and the like.
  • the processor may further include a hardware chip.
  • the hardware chip may be an application-specific integrated circuit (ASIC), a programmable logic device (PLD) or a combination thereof.
  • the PLD may be a complex programmable logic device (CPLD) or a field-programmable gate array (FPGA).
  • the memory 900 is further configured to store program instructions.
  • the processor 800 can invoke the program instructions to implement a motion recognition based network training method as shown in the embodiment of FIG. 4 of the present application.
  • the communication interface 1100 is configured to acquire motion data detected by an external device for a preset action
  • the image obtaining device 1000 is configured to collect an image for the preset action
  • the processor 800 is configured to process an image acquired by the image acquiring device to obtain image recognition data, identify an action corresponding to the motion data, and supervise the image identification data by using the identified action Learning, and training the preset network model based on the image recognition data after the supervised learning.
  • the processor 800 is specifically configured to perform the supervised learning by using the image recognition data as an input and using the action as a target output.
  • the processor 800 is further configured to acquire feature data corresponding to the preset action according to the motion data, and fuse the feature data with the image recognition data after the supervised learning to obtain a fusion. Data; using the fusion data to train a preset network model.
  • the processor 800 is specifically configured to convert the motion data into frequency domain data to use the frequency domain data as feature data corresponding to the current action.
  • the obtained motion data can be transformed from the time domain to the frequency domain by Fourier transform. Go to the frequency domain data, and determine the feature data based on the frequency domain data, for example, directly using the frequency domain data as feature data, or performing feature extraction on the frequency domain data, such as data obtained after processing by extraction, superposition, and the like.
  • the feature data For example, directly using the frequency domain data as feature data, or performing feature extraction on the frequency domain data, such as data obtained after processing by extraction, superposition, and the like.
  • the network model may comprise a neural network.
  • the motion data of the external device may be acquired, and the motion data is converted into frequency domain data, thereby using the frequency domain data to identify an action corresponding to the frequency domain data, or by using motion data and
  • the image recognition data is fused to obtain the fused data, thereby using the fused data to identify an action corresponding to the fused data, or by determining an action corresponding to the motion data, using the image recognition data and the determined action to supervise the image recognition data
  • the robustness is better.
  • An embodiment of the present invention further provides an aircraft, including
  • the motion recognition-based network training device according to any one of the foregoing embodiments of FIG. 10 is configured to train the network model of motion recognition.
  • the disclosed apparatus and method may be implemented in other manners.
  • the device embodiments described above are merely illustrative.
  • the division of the modules is only a logical function division.
  • there may be another division manner for example, multiple modules or components may be combined or Can be integrated into another system, or some features can be ignored or not executed.
  • the mutual coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some interface, device or module, and may be electrical, mechanical or otherwise.
  • the modules described as separate components may or may not be physically separated.
  • the components displayed as modules may or may not be physical modules, that is, may be located in one place, or may be distributed to multiple network modules. . Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the embodiment.
  • each functional module in each embodiment of the present invention may be integrated into one processing module, or each module may exist physically separately, or two or more modules may be integrated into one module. in.
  • the above integrated modules can be implemented in the form of hardware or in the form of hardware plus software function modules.
  • the above-described integrated modules implemented in the form of software function modules can be stored in a computer readable storage medium.
  • the software function modules described above are stored in a storage medium and include instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) or a processor to perform the methods of the various embodiments of the present invention. Part of the steps.
  • the foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, and the like, which can store program code. .

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Automation & Control Theory (AREA)
  • Remote Sensing (AREA)
  • Radar, Positioning & Navigation (AREA)
  • Aviation & Aerospace Engineering (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Computing Systems (AREA)
  • Data Mining & Analysis (AREA)
  • Mathematical Physics (AREA)
  • Human Computer Interaction (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • General Health & Medical Sciences (AREA)
  • Psychiatry (AREA)
  • Social Psychology (AREA)
  • Evolutionary Computation (AREA)
  • Multimedia (AREA)
  • Health & Medical Sciences (AREA)
  • Evolutionary Biology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Artificial Intelligence (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • General Engineering & Computer Science (AREA)
  • Image Analysis (AREA)
  • User Interface Of Digital Computer (AREA)

Abstract

La présente invention concerne un procédé de reconnaissance de geste, un procédé d'apprentissage de réseau, un appareil et un équipement, le procédé consistant : à acquérir des données de mouvement détectées par un dispositif externe pour un geste actuel (101) ; à convertir les données de mouvement en données de domaine de fréquence, et à utiliser les données de domaine de fréquence pour reconnaître un geste correspondant aux données de domaine de fréquence (102). Par conséquent, la précision et la fiabilité de la reconnaissance de geste peuvent être améliorées.
PCT/CN2016/104121 2016-10-31 2016-10-31 Procédé de reconnaissance de geste, procédé d'apprentissage de réseau, appareil et équipement WO2018076371A1 (fr)

Priority Applications (2)

Application Number Priority Date Filing Date Title
PCT/CN2016/104121 WO2018076371A1 (fr) 2016-10-31 2016-10-31 Procédé de reconnaissance de geste, procédé d'apprentissage de réseau, appareil et équipement
CN201680029871.3A CN107735796A (zh) 2016-10-31 2016-10-31 动作识别方法、网络训练方法、装置及设备

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2016/104121 WO2018076371A1 (fr) 2016-10-31 2016-10-31 Procédé de reconnaissance de geste, procédé d'apprentissage de réseau, appareil et équipement

Publications (1)

Publication Number Publication Date
WO2018076371A1 true WO2018076371A1 (fr) 2018-05-03

Family

ID=61201295

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2016/104121 WO2018076371A1 (fr) 2016-10-31 2016-10-31 Procédé de reconnaissance de geste, procédé d'apprentissage de réseau, appareil et équipement

Country Status (2)

Country Link
CN (1) CN107735796A (fr)
WO (1) WO2018076371A1 (fr)

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2020037594A1 (fr) * 2018-08-23 2020-02-27 合刃科技（深圳）有限公司 Procédé et appareil de reconnaissance de geste basée sur l'imagerie hyperspectrale
CN111050266B (zh) * 2019-12-20 2021-07-30 朱凤邹 一种基于耳机检测动作进行功能控制的方法及系统
CN116311539B (zh) * 2023-05-19 2023-07-28 亿慧云智能科技（深圳）股份有限公司 基于毫米波的睡眠动作捕捉方法、装置、设备及存储介质

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101807245A (zh) * 2010-03-02 2010-08-18 天津大学 基于人工神经网络的多源步态特征提取与身份识别方法
CN104484037A (zh) * 2014-12-12 2015-04-01 三星电子（中国）研发中心 通过可穿戴设备进行智能控制的方法及该可穿戴设备
CN204945794U (zh) * 2015-08-24 2016-01-06 武汉理工大学 基于手势识别的无线遥控小车
CN105817037A (zh) * 2016-05-19 2016-08-03 深圳大学 一种基于肌电控制的玩具飞行器及其控制方法
CN105867355A (zh) * 2016-06-03 2016-08-17 深圳市迪瑞特科技有限公司 一种智能车机系统
CN105955306A (zh) * 2016-07-20 2016-09-21 西安中科比奇创新科技有限责任公司 可穿戴设备、基于可穿戴设备的无人机控制方法及系统

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101807245A (zh) * 2010-03-02 2010-08-18 天津大学 基于人工神经网络的多源步态特征提取与身份识别方法
CN104484037A (zh) * 2014-12-12 2015-04-01 三星电子（中国）研发中心 通过可穿戴设备进行智能控制的方法及该可穿戴设备
CN204945794U (zh) * 2015-08-24 2016-01-06 武汉理工大学 基于手势识别的无线遥控小车
CN105817037A (zh) * 2016-05-19 2016-08-03 深圳大学 一种基于肌电控制的玩具飞行器及其控制方法
CN105867355A (zh) * 2016-06-03 2016-08-17 深圳市迪瑞特科技有限公司 一种智能车机系统
CN105955306A (zh) * 2016-07-20 2016-09-21 西安中科比奇创新科技有限责任公司 可穿戴设备、基于可穿戴设备的无人机控制方法及系统

Also Published As

Publication number Publication date
CN107735796A (zh) 2018-02-23

Similar Documents

Publication Publication Date Title
US11892859B2 (en) Remoteless control of drone behavior
US11914370B2 (en) System and method for providing easy-to-use release and auto-positioning for drone applications
US11861069B2 (en) Gesture operated wrist mounted camera system
JP6732317B2 (ja) 顔活動検出方法および装置、ならびに電子デバイス
US20230136669A1 (en) Event camera-based gaze tracking using neural networks
CN107239728B (zh) 基于深度学习姿态估计的无人机交互装置与方法
JP6503070B2 (ja) 携帯型デバイスの位置を決定するための方法
US11715231B2 (en) Head pose estimation from local eye region
CN110083202A (zh) 与近眼显示器的多模交互
WO2018001245A1 (fr) Commande de robot à l'aide de gestes
CN104820488A (zh) 用户定向式个人信息助理
EP3188128A1 (fr) Dispositif de traitement d'informations, procédé de traitement d'informations et programme
US10971152B2 (en) Imaging control method and apparatus, control device, and imaging device
WO2018076371A1 (fr) Procédé de reconnaissance de geste, procédé d'apprentissage de réseau, appareil et équipement
Abate et al. Remote 3D face reconstruction by means of autonomous unmanned aerial vehicles
WO2022082440A1 (fr) Procédé, appareil et système pour déterminer une stratégie de suivi de cible et dispositif et support de stockage
Tu et al. Face and gesture based human computer interaction
CN115393962A (zh) 动作识别方法、头戴显示设备和存储介质
US20220265168A1 (en) Real-time limb motion tracking
WO2023151551A1 (fr) Procédé et appareil de traitement d'image vidéo, dispositif électronique et support de stockage
US20230143443A1 (en) Systems and methods of fusing computer-generated predicted image frames with captured images frames to create a high-dynamic-range video having a high number of frames per second
US20200341556A1 (en) Pattern embeddable recognition engine and method
Martins A human-machine interface using augmented reality glasses for applications in assistive robotics.
Mishra A review on learning-based algorithms for human activity recognition
KR20230168094A (ko) 손톱 분석을 위해 이미지를 처리하는 방법, 시스템 및 비일시성의 컴퓨터 판독 가능한 기록 매체

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 16919618

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 16919618

Country of ref document: EP

Kind code of ref document: A1