WO2020211447A1 - 基于手部速度与轨迹分布的跟随机器人手势轨迹识别方法 - Google Patents

基于手部速度与轨迹分布的跟随机器人手势轨迹识别方法 Download PDF

Info

Publication number
WO2020211447A1
WO2020211447A1 PCT/CN2019/128938 CN2019128938W WO2020211447A1 WO 2020211447 A1 WO2020211447 A1 WO 2020211447A1 CN 2019128938 W CN2019128938 W CN 2019128938W WO 2020211447 A1 WO2020211447 A1 WO 2020211447A1
Authority
WO
WIPO (PCT)
Prior art keywords
hand
grid
descriptor
trajectory
vector
Prior art date
Application number
PCT/CN2019/128938
Other languages
English (en)
French (fr)
Inventor
赵昕玥
高淦
何再兴
张博伦
张树有
谭建荣
Original Assignee
赵昕玥
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 赵昕玥 filed Critical 赵昕玥
Priority to EP19924763.6A priority Critical patent/EP3951643B1/en
Priority to US17/279,062 priority patent/US11847803B2/en
Publication of WO2020211447A1 publication Critical patent/WO2020211447A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/20—Movements or behaviour, e.g. gesture recognition
    • G06V40/28—Recognition of hand or arm movements, e.g. recognition of deaf sign language
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G06F18/2413—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on distances to training or reference patterns
    • G06F18/24133—Distances to prototypes
    • G06F18/24137—Distances to cluster centroïds
    • G06F18/2414—Smoothing the distance, e.g. radial basis function networks [RBFN]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G06F18/2413—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on distances to training or reference patterns
    • G06F18/24147—Distances to closest patterns, e.g. nearest neighbour classification
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/017—Gesture based interaction, e.g. based on a set of recognized hand gestures
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
    • G06T17/20—Finite element generation, e.g. wire-frame surface description, tesselation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/70—Denoising; Smoothing
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
    • G06T7/251—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments involving models
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/60—Type of objects
    • G06V20/64—Three-dimensional objects
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/107—Static hand or arm
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
    • B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
    • B25J19/00—Accessories fitted to manipulators, e.g. for monitoring, for viewing; Safety devices combined with or specially adapted for use in connection with manipulators
    • B25J19/02—Sensing devices
    • B25J19/021—Optical sensing devices
    • B25J19/023—Optical sensing devices including video camera means
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
    • B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
    • B25J9/00—Programme-controlled manipulators
    • B25J9/0003—Home robots, i.e. small robots for domestic use
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10024—Color image
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10028—Range image; Depth image; 3D point clouds
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30196—Human being; Person
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30241—Trajectory
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30248—Vehicle exterior or interior
    • G06T2207/30252—Vehicle exterior; Vicinity of vehicle

Definitions

  • the invention relates to the field of gesture recognition, in particular to a method for recognizing a gesture track of a follower robot based on a histogram of hand speed and track distribution.
  • the follower robot has a wide range of application backgrounds in daily life, and can assist passengers in transporting luggage or helping specific customers during shopping.
  • a follower robot with only a tracking function cannot be competent in the actual working environment.
  • the determination and switching of the tracking object; the end of the tracking task; receiving the instruction of the tracking object during the tracking process, etc. all require the follower robot to have the function of human-computer interaction.
  • the continuous trajectory is segmented by the angle threshold, the position and speed characteristics at the segmentation point are extracted by the sliding window, and the SVM classifier is used to detect the speed reduction to verify the correctness of the segmentation.
  • the weighted dynamic time planning method identifies the trajectory. Mahmoud et al. calculated the angle between the front and rear points of the gesture trajectory and discrete, established the HMM model of the posture and the non-posture trajectory, and performed sliding window matching on the continuous trajectory. If the difference between the probability of a certain posture and the non-posture probability is negative, it is meaningful. The start point of the trajectory is detected, and the difference between positive and negative represents the detection of the end point of the posture, and finally the meaning of the trajectory between the start point and the end point is recognized.
  • the above method of gesture classification requires the establishment of a training set containing a large amount of data for a single action, which requires a large amount of preliminary work; it is sensitive to the time sequence of gesture completion, which results in gestures that must be completed in a set order to interact with the robot, which lacks flexibility; and The computational consumption is large, which is not conducive to rapidity, and has higher requirements for hardware.
  • the present invention proposes a method for recognizing the gesture trajectory of a following robot based on hand speed and trajectory distribution.
  • the invention uses kinect to sample the hand movement trajectory and smooth it; count the speed direction and amplitude information of the trajectory to form a hand movement speed descriptor; count the direction and amplitude information of the distribution of the trajectory points relative to the center of gravity of the trajectory to form the shape description of the hand trajectory Sub; the standard description of the gesture is given by the expression of the trajectory shape or the method of demonstrating the template trajectory; the cosine similarity of the velocity and the shape descriptor and the standard velocity and the shape descriptor is calculated, and the comprehensive similarity is obtained after weighting, as the gesture type Judgment basis.
  • the technical solution adopted by the present invention includes the following steps:
  • Step 1 The kinect camera samples and shoots towards the operator to obtain the 3D position information of the skeletal joints. According to the 3D position information of the skeletal joints and the relative position of the operator and the kinect camera, a projection plane is established. The kinect camera faces the operator to obtain the hand of the human hand. Three-dimensional position, during the process from the operator's start sign gesture to the end sign gesture, the movement data of the three-dimensional position of the hand is recorded, and the movement data of the three-dimensional position of the hand is projected to the projection plane to obtain the projection plane data of the hand;
  • a kinect camera and an upper computer are installed on the following robot, and four wheels are installed at the bottom.
  • the continuous images of the operator's gestures are captured by the kinect camera and sent to the upper computer for processing and judgment, and the gesture track is obtained by recognition.
  • the three-dimensional position of the hand is the three-dimensional position of the center point of the human hand.
  • Step 2 Carry out the sliding average smoothing processing on the projection plane data of the hand.
  • the two adjacent three-dimensional hand positions are connected by a vector from the previous three-dimensional hand position to the next three-dimensional hand position.
  • This vector is used as the speed Vector
  • the angle of the velocity vector is divided into n categories according to the neighborhood method
  • the amplitude of the velocity vector is divided into m categories according to the magnitude.
  • Each velocity vector is expressed as follows:
  • q represents the type result of the angle of the velocity vector classified by the neighborhood method
  • j represents the type result of the magnitude of the velocity vector classified by the size
  • classifying the angle of the velocity vector by the neighborhood method means: shifting the starting point of the velocity vector to the coordinate origin of the projection plane, and dividing all the angular intervals of the velocity vector between 0 and 360 degrees on the projection plane, etc. Divided into n areas, the end point of the velocity vector falls in the qth area, and the angle of the velocity vector is classified as q according to the neighborhood method.
  • classifying the amplitude of the speed vector according to the size refers to: dividing the amplitude of the speed vector into m intervals, and the amplitude of the speed vector falls in the j-th interval, then the amplitude of the speed vector is classified according to the size The result is j.
  • Step 3 Process the angle and amplitude category of each velocity vector to obtain the hand motion vector, and cyclically shift each component in the hand motion vector to obtain the hand motion descriptor; use the cyclic shift to make the largest component at the top of the vector, Make the hand motion descriptor have rotation invariance.
  • Step 4 Establish a hand movement area around the hand in the hand projection plane data, grid the hand movement area, traverse the three-dimensional position of the hand in all frames according to the timing of sampling and shooting, and calculate the three-dimensional position of the hand in each frame Assign values to the grid where it is located, and then calculate the centroid positions of all assigned grids;
  • Step 5 Make a vector from the position of the centroid to each assigned grid as the centroid pointing vector, and then classify the angle and amplitude in the same way as in step 2.
  • the classification method is the same as the speed vector classification in step 2.
  • the method is the same, that is, the angle of the centroid pointing vector is divided into multiple categories according to the neighborhood method, and the amplitude of the centroid pointing vector is divided into multiple categories according to the size; then, the angle and amplitude category of each centroid pointing vector are processed to obtain the hand shape Vector, cyclically shift each component in the hand shape vector to obtain the hand trajectory shape descriptor; make the largest component at the first place of the vector.
  • Step 6 Calculate the cosine of the angle between the hand motion descriptor and the standard hand motion descriptor, calculate the cosine of the angle between the hand trajectory shape descriptor and the standard hand trajectory shape descriptor, and integrate the two angle cosine values
  • the common similarity between the motion descriptor, the track shape descriptor and the standard descriptor is obtained by processing, and the standard gesture with the largest common similarity is taken as the result of gesture recognition.
  • the standard hand motion descriptor and the standard hand trajectory shape descriptor are both standard descriptors.
  • the method for obtaining the standard descriptor is as follows: collect data through kinect to obtain standard template gestures, and then process the standard hand movements through the above steps Descriptor and standard hand trajectory shape descriptor.
  • step 2 the moving data of the three-dimensional position of the hand after the projection is subjected to moving average smoothing processing, which is specifically processed by the following formula:
  • P i represents the i-th hand three-dimensional position
  • P i (x i, y i), x i, y i represent the i-th hand the three-dimensional position of horizontal and vertical coordinates on the projection plane
  • P i ' the i-th hand three-dimensional position represented by a moving average smoothing after
  • P i ' (x i ', y i ')
  • x i', y i ' denote D i-th hand portion after moving average smoothing
  • the abscissa and ordinate of the position on the projection plane, i ⁇ N, N represents the total number of three-dimensional hand positions taken by sampling.
  • processing the angle and amplitude categories of each velocity vector to obtain the hand motion vector means: adding the amplitude classification results of the velocity vectors with the same angle classification result, and the result of the addition is used as the hand motion vector
  • the i-th ordinal component, i is the same as the angle classification result.
  • the grid assignment in step 4 specifically deals with the three-dimensional position of the hand in each frame as the current frame:
  • H' is taken as the largest integer value smaller than the absolute value of the slope of the connecting segment; if the ordinate is closer to the grid where the three-dimensional position of the hand is in the current frame than the row coordinate, then H'is taken to be greater than the absolute value of the slope of the connecting segment The smallest integer value.
  • step 4.1.4 Repeat the above step 4.1.3 processing continuously until the assignment grid reaches the grid where the three-dimensional position of the hand is in the current frame;
  • All the assigned grids constitute the motion trajectory of the gesture center.
  • the assignment is added as a mark, and then the centroid positions of all assigned grids are calculated.
  • the centroid position can be a decimal.
  • the method of processing the angle and amplitude category of each centroid pointing vector in the step 5 to obtain the hand shape vector is the same as the method of processing the angle and amplitude category of each velocity vector to obtain the hand motion vector in step 3.
  • the magnitude classification results of the centroid pointing vectors with the same classification result are added, and the sum result is used as the i-th ordinal component of the hand shape vector, and i is the same as the angle classification result.
  • step 6 the calculation method of the similarity between the integrated motion descriptor and track shape descriptor and the standard descriptor is:
  • S 1 represents the similarity between the motion descriptor and the standard descriptor
  • S 2 represents the similarity between the trajectory shape descriptor and the standard descriptor
  • S 0 represents the common similarity of the motion descriptor, the trajectory shape descriptor and the standard descriptor
  • ⁇ 1 represents the weight of the similarity between the motion descriptor and the standard descriptor in the common similarity of the motion descriptor, the trajectory shape descriptor and the standard descriptor
  • ⁇ 2 represents the weight of the trajectory shape descriptor and the standard descriptor. The weight of similarity in the common similarity of motion descriptors, trajectory shape descriptors and standard descriptors.
  • the invention processes the data collected by the kinect camera, can accurately identify and obtain the type of human gestures, is insensitive to the timing of the translation, zoom, rotation, and trajectory of the gesture trajectory, and has high flexibility.
  • the invention does not require training of a large number of samples, saving time and energy.
  • the invention has high recognition speed and low resource consumption.
  • Fig. 1 is a flowchart of gesture trajectory recognition according to the present invention
  • Figure 2 is a diagram of the hand movement position between the intercepted start and end marks
  • Figure 3 is an effect diagram after sliding smooth filtering of the hand movement position
  • Figure 4 is an effect diagram of the speed vector angle and amplitude classified under the polar coordinate diagram
  • Figure 5 is a schematic diagram of a hand motion descriptor
  • Figure 6 is a schematic diagram of grid assignment where the three-dimensional position of the hand is located
  • Fig. 7 is a schematic diagram of a descriptor of a hand trajectory shape.
  • Step 1 The kinect camera samples and shoots towards the operator to obtain the 3D position information of the skeletal joints.
  • a projection plane is established.
  • the movement data of the three-dimensional position of the hand is recorded during the process from the operator performing the start marking gesture to the end marking gesture, and the movement data of the three-dimensional position of the hand is projected to the projection plane to obtain the projection plane data of the hand. Mark the hand projection plane data obtained in step 1 with ‘*’, and then connect them with straight lines in sequence, as shown in Figure 2.
  • Step 2 Carry out moving average smoothing processing on the projection plane data of the hand, and the processed effect is shown in Figure 3.
  • the two adjacent three-dimensional positions of the hands are connected by a vector that starts from the three-dimensional position of the hand in the previous frame and points to the three-dimensional position of the hand in the next frame.
  • This vector is used as the velocity vector.
  • the angle of the velocity vector is divided into n according to the neighborhood method.
  • Class, the amplitude of the speed vector is divided into m classes according to the size, and each speed vector is expressed as follows:
  • q represents the type result of the angle of the velocity vector classified by the neighborhood method
  • j represents the type result of the magnitude of the velocity vector classified by the size.
  • the velocity vector starts from -22.5 degrees and every 45 degrees is classified into 8 types from 0 to 360 degrees; for the amplitude value, it is divided into one type for every 0.01 meter, which is divided into m 0 types.
  • m 0 depends on the magnitude of the maximum velocity vector.
  • the discrimination conditions for angle classification are shown in Table 1; the discrimination conditions for amplitude classification are shown in Table 2. Move the starting point of all velocity vectors to the origin of the polar coordinate diagram, as shown in Figure 4.
  • Step 3 Processing the angle and amplitude category of each velocity vector to obtain the hand motion vector, and cyclically shift each component in the hand motion vector to obtain the hand motion descriptor.
  • the hand motion vector obtained through the above steps is: [116; 74; 73; 108; 71; 79; 102; 59]; the hand motion descriptor is: [116; 74; 73; 108; 71; 79; 102 ;59];
  • the histogram corresponding to the hand motion descriptor is shown in Figure 5.
  • Step 4 Establish a hand movement area around the hand in the hand projection plane data, grid the hand movement area, traverse the three-dimensional position of the hand in all frames according to the timing of sampling and shooting, and calculate the three-dimensional position of the hand in each frame Calculate the center of mass of all assigned grids, and then take two points (21, 4) and (25, 10) on the trajectory to illustrate the assignment process, as shown in Table 3 and Figure 6.
  • the grid column that is closest to the grid where the three-dimensional position of the hand in the current frame is located after the assignment is called the previous grid column; it will be sent from the previous grid, and is adjacent to the grid in the current frame where the three-dimensional position of the hand is located.
  • a column of grid is called the current grid column.
  • centroid calculation result of all points is: (15.35, 25.75).
  • Step 5 Create a vector from the position of the centroid to each assigned grid as the centroid pointing vector, and then classify the angle and amplitude in the same way as in step 2, and process the angle and amplitude category of each centroid pointing vector
  • the hand shape vector is obtained, and the components in the hand shape vector are cyclically shifted; the hand shape vector obtained through the above steps is: [45;137;162;50;168;136;90;136]; hand
  • the trajectory shape descriptor is: [168; 136; 90; 136; 45; 137; 162; 50]; the histogram corresponding to the hand trajectory shape descriptor is shown in Figure 6.
  • Step 6 Calculate the cosine of the angle between the hand motion descriptor and the standard hand motion descriptor, calculate the cosine of the angle between the hand trajectory shape descriptor and the standard hand trajectory shape descriptor, and integrate the two angle cosine values
  • the common similarity between the motion descriptor, the track shape descriptor and the standard descriptor is obtained by processing, and the standard gesture with the largest common similarity is taken as the result of gesture recognition.
  • the method of obtaining the standard hand motion descriptor in this embodiment is as follows: Given the expression of the trajectory shape, take points on the analytical formula at a certain density, and obtain the standard hand motion according to the method described in steps 1-5 for the obtained points. Descriptor and standard hand trajectory shape descriptor.
  • the descriptor is: [1; 1; 1; 1; 1; 1; 1; 1].
  • the cosine of the angle between the hand motion descriptor and the standard hand motion descriptor is 0.9755; the cosine of the angle between the hand trajectory shape descriptor and the standard hand trajectory shape descriptor is 0.9320.
  • the track recognition result is a circle.
  • the present invention processes the data collected by the kinect camera, and can accurately identify the type of human gestures without training a large number of samples.
  • the timing of the translation, zoom, rotation, and trajectory of the gesture trajectory is not Sensitive, highly flexible, very saving time and energy, fast recognition speed and low resource consumption.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Multimedia (AREA)
  • Human Computer Interaction (AREA)
  • General Engineering & Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Social Psychology (AREA)
  • Health & Medical Sciences (AREA)
  • Psychiatry (AREA)
  • General Health & Medical Sciences (AREA)
  • Evolutionary Biology (AREA)
  • Artificial Intelligence (AREA)
  • Evolutionary Computation (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Computer Graphics (AREA)
  • Geometry (AREA)
  • Software Systems (AREA)
  • Image Analysis (AREA)
  • Image Processing (AREA)

Abstract

本发明公开了一种基于手部速度与轨迹分布的跟随机器人手势轨迹识别方法。kinect相机朝向操作者采样拍摄，获得手部投影平面数据；对手部投影平面数据进行滑动平均平滑处理，建立速度向量，处理速度向量获得手部运动描述子；建立手部运动区域，按采样拍摄的时序遍历所有帧的手部三维位置，将每帧的手部三维位置所在的网格赋值，再计算所有被赋值网格的质心位置；建立质心指向向量，处理质心指向向量得到手部轨迹形状描述子；综合两个夹角余弦值处理获得运动描述子、轨迹形状描述子与标准描述子的共同相似度，取共同相似度最大为结果。本发明能准确识别获得人手势的类型，对手势轨迹的平移、缩放、旋转、轨迹的时序不敏感，灵活性高，节省时间和精力。

Description

基于手部速度与轨迹分布的跟随机器人手势轨迹识别方法 技术领域
本发明涉及姿态识别领域，具体涉及一种基于手部速度与轨迹分布直方图的跟随机器人手势轨迹识别方法。
背景技术
跟随机器人在日常生活中有广泛的应用背景，可协助旅客运送行李，或购物时帮助特定顾客。但是，仅具有追踪功能的跟随机器人无法在实际工作环境中胜任。追踪对象的确定，切换；追踪任务的结束；追踪过程中接收追踪对象的指令等都要求跟随机器人具有人机交互的功能。遥控交互的方式受限于一定物理器材，使用起来不方便；语音交互由于跟随机器人工作环境嘈杂，追踪人与跟随机器人又有一定距离，外部噪声的干扰太大，也无法应用；而手势作为一种自然的人机交互方式，因其具有不依赖外部媒介，表意直观，可远距离传递信息的特点，适合用作跟随机器人的人机交互方式。
但是，不同人甚至同一个人在不同时间做同一手势，轨迹都有较大的不同，这种不确定性要求跟随机器人的手势交互技术可以对手势轨迹作出区分。近年来，有众多学者对手势轨迹识别都有所研究，Beh等人提出用轨迹相邻两点角度变化是否超过阈值对手势轨迹分段，使用基于Von-Mise分布的HMM分类器对轨迹建模，对字母数字手势轨迹数据集最佳识别率达到97.11％。Jingren Tang等人提出结构化的动态时间规划方法，首先利用角度阈值分割连续轨迹，使用滑动窗提取分割点处的位置和速度特征，使用SVM分类器检测速度降低以验证分割的正确性，再用加权的动态时间规划方法识别轨迹。Mahmoud等计算手势轨迹的前后点之间角度并离散，建立姿态和非姿态轨迹的HMM模型，在连续轨迹上做滑动窗匹配，如果某姿态的概率与非姿态概率之差负为正则有意义姿态轨迹起始点检出，差值正变负代表姿态结束点检出，最后识别开始点和结束点之间的轨迹含义。
以上手势分类的方法需要针对单一动作建立包含大量数据的训练集，先期工作量大；对手势完成的时间序列敏感，导致必须按照设定好的顺序完成手势才能与机器人交互，缺乏灵活性；且计算消耗大，不利于快速性，对硬件有较高的要求。
发明内容
针对以上问题，为了解决背景技术中存在的问题，本发明提出一种基于手部速度及轨迹分布的跟随机器人手势轨迹识别方法。
本发明利用kinect采样手部运动轨迹并平滑；统计轨迹的速度方向和幅值信息，形成手部运动速度描述符；统计轨迹点相对轨迹重心分布的方向和幅值信息，形成手部轨迹形状描述子；手势的标准描述通过轨迹形状的表达式或者示范模板轨迹的方法给定；计算速度和形状描述符与标准速度与形状描述符的余弦相似度，加权后得到综合相似度，作为手势类型的判别依据。
如图1所示，本发明所采用的技术方案是包括以下步骤：
步骤1：kinect相机朝向操作者采样拍摄获得骨骼关节的三维位置信息，根据骨骼关节的三维位置信息以及操作者和kinect相机的相对位置建立投影平面，kinect相机朝向操作者拍摄获得人手部的手部三维位置，从操作者进行开始标志手势到进行结束标志手势之间的过程中，记录手部三维位置的运动数据，将手部三维位置的运动数据投影到投影平面，获得手部投影平面数据；
本发明在跟随机器人上安装有kinect相机和上位机，底部安装由四个轮子，通过kinect相机拍摄操作者的手势连续图像发送到上位机中进行处理判断，识别获得手势轨迹。
所述的手部三维位置为人手部中心点的三维位置。
步骤2：对手部投影平面数据进行滑动平均平滑处理，相邻的两个手部三维位置用一个从上一帧手部三维位置出发指向下一帧手部三维位置的向量连接，该向量作为速度向量，对速度向量的角度按邻域方法分为n类，对速度向量的幅值按照大小分为m类，将每个速度向量表示如下：
(q,j),q≤n,j≤m        (1)
其中，q表示速度向量的角度按邻域的方法分类的类型结果，j表示速度向量的幅值按大小分类的类型结果；
所述步骤2中，对速度向量的角度按邻域方法分类是指：将速度向量的起点平移到投影平面的坐标原点，将投影平面上0到360度之间的速度向量的所有角度区间等分成n个区域，速度向量终点落在第q个区域，则速度向量的角度按邻域方法的分类结果为q。
所述步骤2中，对速度向量的幅值按照大小分类是指：将速度向量的幅值分成m个区间，速度向量幅值落在第j个区间，则速度向量的幅值按照大小的分类结果为j。
步骤3：处理各个速度向量的角度和幅值类别得到手部运动向量，将手部运 动向量中各分量做循环移位，获得手部运动描述子；通过循环移位使最大分量位于向量首位，使手部运动描述子具有旋转不变性。
步骤4：手部投影平面数据中在手部周围建立手部运动区域，将手部运动区域网格化，按采样拍摄的时序遍历所有帧的手部三维位置，将每帧的手部三维位置所在的网格赋值，再计算所有被赋值网格的质心位置；
步骤5：对从质心位置出发指向每个被赋值网格的位置作向量作为质心指向向量，然后按照步骤2中的相同方式作角度和幅值分类，分类方法与步骤2中对速度向量分类的方法相同，即对质心指向向量的角度按邻域方法分为多类，对质心指向向量的幅值按照大小分为多类；接着，处理各个质心指向向量的角度和幅值类别得到手部形状向量，将手部形状向量中各分量做循环移位得到手部轨迹形状描述子；使最大分量位于向量的首位。
步骤6：计算手部运动描述子和标准手部运动描述子的夹角余弦值，计算手部轨迹形状描述子和标准手部轨迹形状描述子的夹角余弦值，综合两个夹角余弦值处理获得运动描述子、轨迹形状描述子与标准描述子的共同相似度，取共同相似度最大的标准手势作为手势识别的结果。
所述标准手部运动描述子和标准手部轨迹形状描述子均为标准描述子，标准描述子获得的方法如下：通过kinect采集数据获得标准的模板手势，而后通过上述步骤处理获得标准手部运动描述子和标准手部轨迹形状描述子。
所述的步骤2中，对投影后的手部三维位置的运动数据进行滑动平均平滑处理，具体采用以下公式进行处理为：
Figure PCTCN2019128938-appb-000001
Figure PCTCN2019128938-appb-000002
P i'＝P i，i＝0,N
其中，P i表示第i个手部三维位置，P i＝(x i,y i)，x i,y i分别表示第i个手部三维位置在投影平面上的横纵坐标；P i'表示滑动平均平滑处理后的第i个手部三维位置，P i'＝(x i',y i')，x i',y i'分别表示滑动平均平滑处理后的第i个手部三维位置在投影平面上的横纵坐标，i≤N，N表示采样拍摄的手部三维位置的总数。
所述步骤3中，处理各个速度向量的角度和幅值类别得到手部运动向量，是指：将角度分类结果相同的速度向量的幅值分类结果相加，加和结果作为手 部运动向量的第i个序数分量，i与角度分类结果相同。
所述步骤4的网格赋值具体是针对每一帧的手部三维位置作为当前帧进行处理：
4.1、对当前帧的手部三维位置与进行检查，如果两帧的手部三维位置不在同一网格内，且前一帧的手部三维位置不在当前帧的手部三维位置所在网格的八邻域网格内，作一条连线段连接两帧的手部三维位置，将连线段经过的网格赋值，具体为：
4.1.1、计算两帧的手部三维位置之间连线段在投影平面上的斜率，获得连线段的坐标公式y＝kx+b，并且将前一帧手部三维位置所在网格和当前帧手部三维位置所在网格均进行赋值；
4.1.2、首先从前一帧手部三维位置所在网格列出发，取向靠近当前帧手部三维位置所在网格方向相邻的一列网格作为第一网格列，从前一帧手部三维位置所在网格行开始向靠近当前帧手部三维位置所在网格方向将第一网格列中的H个网格进行赋值，H取为连线段斜率绝对值四舍五入后的整数值；
4.1.3、然后再从第一网格列出发，取向靠近当前帧手部三维位置所在网格方向相邻的一列网格作为下一网格列，从上述步骤赋值后最靠近当前帧手部三维位置所在网格的网格所在网格行开始向靠近当前帧手部三维位置所在网格方向将下一网格列中的H’个网格进行赋值，H’的取值为：将上述步骤赋值后最靠近当前帧手部三维位置所在网格的网格列坐标带入连线段的坐标公式获得纵坐标，若行坐标相比纵坐标更接近当前帧手部三维位置所在网格，则H’取为小于连线段斜率绝对值的最大整数值；若纵坐标相比行坐标更接近当前帧手部三维位置所在网格，则H’取为大于连线段斜率绝对值的最小整数值。
4.1.4、不断重复上述步骤4.1.3处理直到赋值网格到达当前帧手部三维位置所在网格；
4.2、由所有被赋值网格构成了手势中心的运动轨迹，具体实施中赋值是作为标记添加，再计算所有被赋值网格的质心位置，质心位置可以为小数。
所述步骤5中的处理各个质心指向向量的角度和幅值类别得到手部形状向量的方式与步骤3中的处理各个速度向量的角度和幅值类别得到手部运动向量的方式相同，即将角度分类结果相同的质心指向向量的幅值分类结果相加，加和结果作为手部形状向量的第i个序数分量，i与角度分类结果相同。
所述步骤6中，综合运动描述子和轨迹形状描述子与标准描述子的相似度的计算方法是：
S 0＝ω 1S 1+ω 2S 2
式中，S 1表示运动描述子与标准描述子的相似度，S 2表示轨迹形状描述子与标准描述子的相似度，S 0表示运动描述子、轨迹形状描述子与标准描述子的共同相似度，ω 1表示运动描述子与标准描述子的相似度在运动描述子、轨迹形状描述子与标准描述子的共同相似度中所占的权重，ω 2表示轨迹形状描述子与标准描述子的相似度在运动描述子、轨迹形状描述子与标准描述子的共同相似度中所占的权重。
本发明的有益效果是：
本发明对kinect相机采集获得的数据进行处理，能从中准确识别获得人手势的类型，对手势轨迹的平移、缩放、旋转、轨迹的时序不敏感，灵活性高。
本发明不需要大量样本的训练，节省时间和精力。
本发明识别速度快，资源消耗少。
附图说明
图1是本发明所述手势轨迹识别的流程图；
图2是截取的开始与结束标志之间的手部运动位置图；
图3是对手部运动位置滑动平滑滤波后的效果图；
图4是速度向量角度和幅值在极坐标图下进行分类的效果图；
图5是手部运动描述子示意图；
图6是手部三维位置所在的网格赋值示意图；
图7是手部轨迹形状描述子示意图。
具体实施方式
下面结合图和实例对本发明进行进一步描述。
本发明实施例及其实施过程如下：
步骤1：kinect相机朝向操作者采样拍摄获得骨骼关节的三维位置信息，根据骨骼关节的三维位置信息以及操作者和kinect相机的相对位置建立投影平面，kinect相机朝向操作者拍摄获得人手部的手部三维位置，从操作者进行开始标志手势到进行结束标志手势之间的过程中，记录手部三维位置的运动数据，将手部三维位置的运动数据投影到投影平面，获得手部投影平面数据。将步骤一获得的手部投影平面数据用‘*’标记，然后按时序依次用直线相连，如图2。
步骤2：对手部投影平面数据进行滑动平均平滑处理，处理后的效果如图3所示。相邻的两个手部三维位置用一个从上一帧手部三维位置出发指向下一帧 手部三维位置的向量连接，该向量作为速度向量，对速度向量的角度按邻域方法分为n类，对速度向量的幅值按照大小分为m类，将每个速度向量表示如下：
(q,j),q≤n,j≤m      (1)
其中，q表示速度向量的角度按邻域的方法分类的类型结果，j表示速度向量的幅值按大小分类的类型结果。
本实施例将速度向量从-22.5度起每45度作为一类，在0到360度之间共分8类；对于幅值每0.01米分为一类，共分m 0类。m 0取决于最大速度向量的幅值。对角度分类的判别条件如表一所示；对幅值分类的判别条件如表二所示。把所有速度向量的起点移动到极坐标图原点，如图4所示。
表1
x的条件 y的条件 y/x的条件 类别
x>0   -0.414≤y/x<0.414 0
x>0 y>0 0.414≤y/x<2.414 1
  y>0 2.414≤y/x||y/x<-2.414 2
x<0 y>0 -2.414≤y/x<-0.414 3
x<0   -0.414≤y/x<0.414 4
x<0 y<0 0.414≤y/x<2.414 5
  y<0 2.414≤y/x||y/x<-2.414 6
x>0 y<0 -2.414≤y/x<-0.414 7
表2
Figure PCTCN2019128938-appb-000003
步骤3：处理各个速度向量的角度和幅值类别得到手部运动向量，将手部运动向量中各分量做循环移位，获得手部运动描述子。经过以上步骤获得的手部运动向量为：[116；74；73；108；71；79；102；59]；手部运动描述子为：[116；74；73；108；71；79；102；59]；手部运动描述子对应的直方图如图5所示。
步骤4：手部投影平面数据中在手部周围建立手部运动区域，将手部运动区 域网格化，按采样拍摄的时序遍历所有帧的手部三维位置，将每帧的手部三维位置所在的网格赋值，再计算所有被赋值网格的质心位置；以轨迹上两点(21，4)，(25，10)为例说明赋值过程，如表3和图6所示。
计算斜率
Figure PCTCN2019128938-appb-000004
大于斜率绝对值的最小整数为2，小于斜率绝对值的最大整数值为1。直线方程y＝1.667x-31。
将赋值后最靠近当前帧手部三维位置所在网格的网格列称为上一网格列；将从上一网格列出发，靠近当前帧手部三维位置所在网格方向相邻的一列网格称为当前网格列。
表3：
Figure PCTCN2019128938-appb-000005
所有点的质心计算结果为：(15.35，25.75)。
步骤5：对从质心位置出发指向每个被赋值网格的位置作向量作为质心指向向量，然后按照步骤2中的相同方式作角度和幅值分类，处理各个质心指向向量的角度和幅值类别得到手部形状向量，将手部形状向量中各分量做循环移位；经过以上步骤获得的手部形状向量为：[45；137；162；50；168；136；90；136]；手部轨迹形状描述子为：[168；136；90；136；45；137；162；50]；手部轨迹形状描述子对应的直方图如图6所示。
步骤6：计算手部运动描述子和标准手部运动描述子的夹角余弦值，计算手部轨迹形状描述子和标准手部轨迹形状描述子的夹角余弦值，综合两个夹角余弦值处理获得运动描述子、轨迹形状描述子与标准描述子的共同相似度，取共同相似度最大的标准手势作为手势识别的结果。
本实施例中标准手部运动描述子获得的方法如下：给定轨迹形状的表达式，按一定密度在解析式上取点，对取得的点按照步骤1-5所述方法得到标准手部运 动描述子和标准手部轨迹形状描述子。本次通过表达式x 2+y 2＝1采集得到的对于圆形的标准手部运动描述子为：[1；1；1；1；1；1；1；1]；标准手部轨迹形状描述子为：[1；1；1；1；1；1；1；1]。手部运动描述子和标准手部运动描述子的夹角余弦值为：0.9755；手部轨迹形状描述子和标准手部轨迹形状描述子的夹角余弦值为：0.9320。
具体实施中，ω 1＝0.5,ω 2＝0.5。运动描述子、轨迹形状描述子与标准描述子的共同相似度：0.9538。
将步骤1-5得到的运动描述子、轨迹形状描述子分别与其他标准手势描述子对比相似度，选取相似度最大的作为手势轨迹识别的结果。表4是将手势描述子与对于直线和S形曲线的标准手势描述子对比结果。
表4
手势类型 圆形 直线 S形曲线
相似度 0.9538 0.1812 0.4733
得到轨迹识别结果是圆形。
由上述实施可见，本发明对kinect相机采集获得的数据进行处理，在不需要大量样本的训练情况下能从中准确识别获得人手势的类型，对手势轨迹的平移、缩放、旋转、轨迹的时序不敏感，灵活性高，非常节省时间和精力，识别速度快，资源消耗少。

Claims (6)

  1. 一种基于手部速度与轨迹分布的跟随机器人手势轨迹识别方法，其特征在于，包括以下步骤：
    步骤1：kinect相机朝向操作者采样拍摄获得骨骼关节的三维位置信息，根据骨骼关节的三维位置信息以及操作者和kinect相机的相对位置建立投影平面，kinect相机朝向操作者拍摄获得人手部的手部三维位置，从操作者进行开始标志手势到进行结束标志手势之间的过程中，记录手部三维位置的运动数据，将手部三维位置的运动数据投影到投影平面，获得手部投影平面数据；
    步骤2：对手部投影平面数据进行滑动平均平滑处理，相邻的两个手部三维位置用一个从上一帧手部三维位置出发指向下一帧手部三维位置的向量连接，该向量作为速度向量，对速度向量的角度按邻域方法分为n类，对速度向量的幅值按照大小分为m类，将每个速度向量表示如下：
    (q,j),q≤n,j≤m    (1)
    其中，q表示速度向量的角度按邻域的方法分类的类型结果，j表示速度向量的幅值按大小分类的类型结果；
    步骤3：处理各个速度向量的角度和幅值类别得到手部运动向量，将手部运动向量中各分量做循环移位，获得手部运动描述子；
    步骤4：手部投影平面数据中在手部周围建立手部运动区域，将手部运动区域网格化，按采样拍摄的时序遍历所有帧的手部三维位置，将每帧的手部三维位置所在的网格赋值，再计算所有被赋值网格的质心位置；
    步骤5：对从质心位置出发指向每个被赋值网格的位置作向量作为质心指向向量，然后按照步骤2中的相同方式作角度和幅值分类；接着，处理各个质心指向向量的角度和幅值类别得到手部形状向量，将手部形状向量中各分量做循环移位得到手部轨迹形状描述子；
    步骤6：计算手部运动描述子和标准手部运动描述子的夹角余弦值，计算手部轨迹形状描述子和标准手部轨迹形状描述子的夹角余弦值，综合两个夹角余弦值处理获得运动描述子、轨迹形状描述子与标准描述子的共同相似度，取共同相似度最大的标准手势作为手势识别的结果。
  2. 根据权利要求1所述的一种基于手部速度与轨迹分布的跟随机器人手势轨迹识别方法，其特征在于：所述的步骤2中，对投影后的手部三维位置的运动数据进行滑动平均平滑处理，具体采用以下公式进行处理为：
    Figure PCTCN2019128938-appb-100001
    Figure PCTCN2019128938-appb-100002
    P i'＝P i，i＝0,N
    其中，P i表示第i个手部三维位置，P i＝(x i,y i)，x i,y i分别表示第i个手部三维位置在投影平面上的横纵坐标；P i'表示滑动平均平滑处理后的第i个手部三维位置，P i'＝(x i',y i')，x i',y i'分别表示滑动平均平滑处理后的第i个手部三维位置在投影平面上的横纵坐标，i≤N，N表示采样拍摄的手部三维位置的总数。
  3. 根据权利要求1所述的一种基于手部速度与轨迹分布的跟随机器人手势轨迹识别方法，其特征在于：所述步骤3中，处理各个速度向量的角度和幅值类别得到手部运动向量，是指：将角度分类结果相同的速度向量的幅值分类结果相加，加和结果作为手部运动向量的第i个序数分量。
  4. 根据权利要求1所述的一种基于手部速度与轨迹分布的跟随机器人手势轨迹识别方法，其特征在于：所述步骤4的网格赋值具体是针对每一帧的手部三维位置作为当前帧进行处理：
    4.1、对当前帧的手部三维位置与进行检查，如果两帧的手部三维位置不在同一网格内，且前一帧的手部三维位置不在当前帧的手部三维位置所在网格的八邻域网格内，作一条连线段连接两帧的手部三维位置，将连线段经过的网格赋值，具体为：
    4.1.1、计算两帧的手部三维位置之间连线段在投影平面上的斜率，获得连线段的坐标公式，并且将前一帧手部三维位置所在网格和当前帧手部三维位置所在网格均进行赋值；
    4.1.2、首先从前一帧手部三维位置所在网格列出发，取向靠近当前帧手部三维位置所在网格方向相邻的一列网格作为第一网格列，从前一帧手部三维位置所在网格行开始向靠近当前帧手部三维位置所在网格方向将第一网格列中的H个网格进行赋值，H取为连线段斜率绝对值四舍五入后的整数值；
    4.1.3、然后再从第一网格列出发，取向靠近当前帧手部三维位置所在网格方向相邻的一列网格作为下一网格列，从上述步骤赋值后最靠近当前帧手部三维位置所在网格的网格所在网格行开始向靠近当前帧手部三维位置所在网格方向将下一网格列中的H’个网格进行赋值，H’的取值为：将上述步骤赋值后最靠近当前帧手部三维位置所在网格的网格列坐标带入连线段的坐标公式获得纵坐标，若行坐标相比纵坐标更接近当前帧手部三维位置所在网格，则H’取为小于连线段斜率绝对值的最大整数值；若纵坐标相比行坐标更接近当前帧手部三维 位置所在网格，则H’取为大于连线段斜率绝对值的最小整数值。
    4.1.4、不断重复上述步骤处理直到赋值网格到达当前帧手部三维位置所在网格；
    4.2、由所有被赋值网格构成了手势中心的运动轨迹，再计算所有被赋值网格的质心位置。
  5. 根据权利要求1所述的一种基于手部速度与轨迹分布的跟随机器人手势轨迹识别方法，其特征在于：所述步骤5中的处理各个质心指向向量的角度和幅值类别得到手部形状向量的方式与步骤3中的处理各个速度向量的角度和幅值类别得到手部运动向量的方式相同。
  6. 根据权利要求1所述的一种基于手部速度与轨迹分布的跟随机器人手势轨迹识别方法，其特征在于：所述步骤6中，综合运动描述子和轨迹形状描述子与标准描述子的相似度的计算方法是：
    S 0＝ω 1S 1+ω 2S 2
    式中，S 1表示运动描述子与标准描述子的相似度，S 2表示轨迹形状描述子与标准描述子的相似度，S 0表示运动描述子、轨迹形状描述子与标准描述子的共同相似度，ω 1表示运动描述子与标准描述子的相似度在运动描述子、轨迹形状描述子与标准描述子的共同相似度中所占的权重，ω 2表示轨迹形状描述子与标准描述子的相似度在运动描述子、轨迹形状描述子与标准描述子的共同相似度中所占的权重。
PCT/CN2019/128938 2019-04-17 2019-12-27 基于手部速度与轨迹分布的跟随机器人手势轨迹识别方法 WO2020211447A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
EP19924763.6A EP3951643B1 (en) 2019-04-17 2019-12-27 Gesture trajectory recognition method for a following robot, based on a histogram corresponding to a hand trajectory shape descriptor
US17/279,062 US11847803B2 (en) 2019-04-17 2019-12-27 Hand trajectory recognition method for following robot based on hand velocity and trajectory distribution

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201910309591.3 2019-04-17
CN201910309591.3A CN110232308B (zh) 2019-04-17 2019-04-17 基于手部速度与轨迹分布的跟随机器人手势轨迹识别方法

Publications (1)

Publication Number Publication Date
WO2020211447A1 true WO2020211447A1 (zh) 2020-10-22

Family

ID=67860186

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2019/128938 WO2020211447A1 (zh) 2019-04-17 2019-12-27 基于手部速度与轨迹分布的跟随机器人手势轨迹识别方法

Country Status (4)

Country Link
US (1) US11847803B2 (zh)
EP (1) EP3951643B1 (zh)
CN (1) CN110232308B (zh)
WO (1) WO2020211447A1 (zh)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112735030A (zh) * 2020-12-28 2021-04-30 深兰人工智能(深圳)有限公司 售货柜的视觉识别方法、装置、电子设备和可读存储介质
CN114863549A (zh) * 2022-03-25 2022-08-05 哈尔滨工程大学 一种抗噪的手部运动轨迹快速识别方法
CN117944057A (zh) * 2024-03-26 2024-04-30 北京云力境安科技有限公司 一种机械臂轨迹规划方法、装置、设备及介质

Families Citing this family (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110232308B (zh) * 2019-04-17 2021-02-09 浙江大学 基于手部速度与轨迹分布的跟随机器人手势轨迹识别方法
CN110956124A (zh) * 2019-11-27 2020-04-03 云南电网有限责任公司电力科学研究院 一种基于手势的显示设备控制方法和显示设备
CN111241233B (zh) * 2019-12-24 2022-04-29 浙江大学 基于关键动词特征全密度传递的服务机器人指令解析方法
CN111260725B (zh) * 2020-01-15 2022-04-19 浙江大学 一种面向动态环境的轮速计辅助的视觉里程计方法
CN111694428B (zh) * 2020-05-25 2021-09-24 电子科技大学 基于Kinect的手势与轨迹远程控制机器人系统
CN112115853A (zh) * 2020-09-17 2020-12-22 西安羚控电子科技有限公司 一种手势识别方法、装置、计算机存储介质及电子设备

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6552729B1 (en) * 1999-01-08 2003-04-22 California Institute Of Technology Automatic generation of animation of synthetic characters
CN105320937A (zh) * 2015-09-25 2016-02-10 北京理工大学 基于Kinect的交警手势识别方法
CN105807926A (zh) * 2016-03-08 2016-07-27 中山大学 一种基于三维连续动态手势识别的无人机人机交互方法
CN110232308A (zh) * 2019-04-17 2019-09-13 浙江大学 基于手部速度与轨迹分布的跟随机器人手势轨迹识别方法

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9275274B2 (en) * 2013-03-12 2016-03-01 Robert Bosch Gmbh System and method for identifying handwriting gestures in an in-vehicle information system
KR101682268B1 (ko) * 2013-05-14 2016-12-05 중앙대학교 산학협력단 다중 클래스 svm과 트리 분류를 이용한 제스처 인식 장치 및 방법

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6552729B1 (en) * 1999-01-08 2003-04-22 California Institute Of Technology Automatic generation of animation of synthetic characters
CN105320937A (zh) * 2015-09-25 2016-02-10 北京理工大学 基于Kinect的交警手势识别方法
CN105807926A (zh) * 2016-03-08 2016-07-27 中山大学 一种基于三维连续动态手势识别的无人机人机交互方法
CN110232308A (zh) * 2019-04-17 2019-09-13 浙江大学 基于手部速度与轨迹分布的跟随机器人手势轨迹识别方法

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP3951643A4 *

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112735030A (zh) * 2020-12-28 2021-04-30 深兰人工智能(深圳)有限公司 售货柜的视觉识别方法、装置、电子设备和可读存储介质
CN114863549A (zh) * 2022-03-25 2022-08-05 哈尔滨工程大学 一种抗噪的手部运动轨迹快速识别方法
CN117944057A (zh) * 2024-03-26 2024-04-30 北京云力境安科技有限公司 一种机械臂轨迹规划方法、装置、设备及介质

Also Published As

Publication number Publication date
EP3951643A1 (en) 2022-02-09
US20220083770A1 (en) 2022-03-17
CN110232308A (zh) 2019-09-13
EP3951643B1 (en) 2024-10-16
EP3951643A4 (en) 2023-05-03
CN110232308B (zh) 2021-02-09
US11847803B2 (en) 2023-12-19

Similar Documents

Publication Publication Date Title
WO2020211447A1 (zh) 基于手部速度与轨迹分布的跟随机器人手势轨迹识别方法
CN106055091B (zh) 一种基于深度信息和校正方式的手部姿态估计方法
CN104680559B (zh) 基于运动行为模式的多视角室内行人跟踪方法
CN108256421A (zh) 一种动态手势序列实时识别方法、系统及装置
CN109325454A (zh) 一种基于YOLOv3的静态手势实时识别方法
Oprisescu et al. Automatic static hand gesture recognition using tof cameras
Tang et al. Online human action recognition based on incremental learning of weighted covariance descriptors
CN104036287A (zh) 一种基于人类运动显著轨迹的视频分类方法
Bhuyan et al. Hand pose recognition using geometric features
CN103593680A (zh) 一种基于隐马尔科夫模型自增量学习的动态手势识别方法
CN110929713B (zh) 一种基于bp神经网络的钢印字符识别方法
CN103745218B (zh) 一种深度图像中的姿势识别方法及装置
Liu et al. Static hand gesture recognition and its application based on support vector machines
CN103886325A (zh) 一种分块的循环矩阵视频跟踪方法
CN111652017A (zh) 一种动态手势识别方法及系统
CN108898623A (zh) 目标跟踪方法及设备
WO2019085060A1 (zh) 一种机器人的挥手检测方法、系统及一种机器人
CN103426000B (zh) 一种静态手势指尖检测方法
CN114792443A (zh) 一种基于图像识别的智能设备手势识别控制方法
Yi et al. Long-range hand gesture recognition with joint ssd network
CN107220634B (zh) 基于改进d-p算法与多模板匹配的手势识别方法
Wang et al. A new hand gesture recognition algorithm based on joint color-depth superpixel earth mover's distance
KR20120089948A (ko) Ｍｈｉ의 형태 정보를 이용한 실시간 동작 인식시스템 및 실시간 동작 인식 방법
Ikram et al. Real time hand gesture recognition using leap motion controller based on CNN-SVM architechture
WO2018135326A1 (ja) 画像処理装置、画像処理システム、画像処理プログラム、及び画像処理方法

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19924763

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

ENP Entry into the national phase

Ref document number: 2019924763

Country of ref document: EP

Effective date: 20211102