WO2021051575A1 - Procédé et système de reconnaissance de geste, ainsi que terminal de gestion et support de stockage lisible par ordinateur - Google Patents

Procédé et système de reconnaissance de geste, ainsi que terminal de gestion et support de stockage lisible par ordinateur Download PDF

Info

Publication number
WO2021051575A1
WO2021051575A1 PCT/CN2019/117813 CN2019117813W WO2021051575A1 WO 2021051575 A1 WO2021051575 A1 WO 2021051575A1 CN 2019117813 W CN2019117813 W CN 2019117813W WO 2021051575 A1 WO2021051575 A1 WO 2021051575A1
Authority
WO
WIPO (PCT)
Prior art keywords
gesture
image data
management terminal
finger
preprocessing
Prior art date
Application number
PCT/CN2019/117813
Other languages
English (en)
Chinese (zh)
Inventor
赵莫言
王红伟
Original Assignee
平安科技（深圳）有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 平安科技（深圳）有限公司 filed Critical 平安科技（深圳）有限公司
Publication of WO2021051575A1 publication Critical patent/WO2021051575A1/fr

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/107—Static hand or arm
    • G06V40/113—Recognition of static hand signs
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
    • G06V10/267—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion by performing operations on regions, e.g. growing, shrinking or watersheds
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features

Definitions

  • This application relates to the field of computer technology. Specifically, this application relates to a gesture recognition method, system, management terminal, and computer-readable storage medium.
  • high-definition camera devices are usually used to capture standard gestures and form images of standard gestures, which can improve the image clarity of the marked gestures to a certain extent, thereby improving the operator’s recognition of standard gestures Accuracy, however, because this method relies on high-definition camera devices that are expensive and require high installation requirements, this method will result in high costs and is not conducive to large-scale promotion.
  • this application proposes a gesture recognition method, system, management terminal, and computer-readable storage medium.
  • the first aspect of the present application provides a gesture recognition method, which is applied to a hand shadow recognition system, and the method includes:
  • the management terminal receives the gesture image data sent by the hand shadow former, wherein the gesture image data corresponds to the target gesture placed in the hand shadow former taken by the camera in the hand shadow former Generated by shadows formed at at least two angles;
  • the management terminal completes the preprocessing of the gesture image data according to a preset preprocessing rule
  • the management terminal uses an image segmentation algorithm to perform image segmentation on the preprocessed gesture image data
  • the management terminal extracts the feature of the target gesture from the gesture image data for which the image segmentation is completed according to a preset feature extraction rule
  • the management terminal uses a gesture recognition model to recognize the features of the target gesture, and then completes the recognition of the target gesture, wherein the gesture recognition model includes a hierarchical decision classifier of a sparse autoencoder.
  • the second aspect of the present application provides a hand shadow recognition system.
  • the system includes a hand shadow former and a management terminal, wherein:
  • the hand shadow former is used to use a camera to shoot shadows formed under at least two angles of the target gesture
  • the hand shadow former is further configured to send the gesture image data generated by the camera to the management terminal after using the camera to shoot the shadow formed at least two angles of the target gesture;
  • the management terminal is configured to complete the preprocessing of the gesture image data according to preset preprocessing rules, and use an image segmentation algorithm to perform image segmentation on the preprocessed gesture image data;
  • the features of the target gesture are extracted from the segmented gesture image data;
  • the management terminal uses a gesture recognition model to recognize the features of the target gesture, and then completes the recognition of the target gesture, wherein the gesture recognition model includes a sparse autoencoder Hierarchical decision classifier.
  • the third aspect of the present application also provides a management terminal, and the management terminal includes:
  • a processor and a memory configured to store machine-readable instructions that, when executed by the processor, cause the processor to execute the gesture recognition method as described in the first aspect of the present application.
  • the fourth aspect of the present application provides a computer non-volatile readable storage medium, the computer non-volatile readable storage medium stores a computer program, characterized in that the computer program program is executed by a processor as in the present application The gesture recognition method described in the first aspect.
  • the present application can restore a certain gesture based on the hand shadow data, and the present application does not need to use a high-standard camera device to capture instantaneous gestures, and thus the present application has the advantages of high recognition accuracy and low cost.
  • FIG. 1 is a schematic flowchart of a gesture recognition method provided by Embodiment 1 of the present application;
  • FIG. 2 is a schematic diagram of the structure of the hand shadow recognition system provided by the second embodiment of the present application.
  • FIG. 3 is a schematic structural diagram of a management terminal provided in Embodiment 3 of the present application.
  • FIG. 1 is a schematic flowchart of a gesture recognition method provided in Embodiment 1 of the present application, where the method includes the steps:
  • the management terminal receives gesture image data sent by the hand shadow former, where the gesture image data is based on a target gesture placed in the hand shadow former taken by a camera in the hand shadow former Corresponding to at least two angles formed by the shadow generated;
  • the management terminal completes the preprocessing of the gesture image data according to a preset preprocessing rule
  • the management terminal uses an image segmentation algorithm to perform image segmentation on the preprocessed gesture image data
  • the management terminal extracts the feature of the target gesture from the gesture image data of the completed image segmentation according to a preset feature extraction rule
  • the management terminal uses a gesture recognition model to recognize features of the target gesture, and then completes the recognition of the target gesture, wherein the gesture recognition model includes a hierarchical decision classifier of a sparse autoencoder.
  • the management terminal completes the preprocessing of the gesture image data according to a preset preprocessing rule, including:
  • the management terminal uses a median filter algorithm to filter the gesture image data, so as to remove the noise of the gesture image data;
  • the management terminal uses an image binarization algorithm to split the filtered gesture image data into a first image part and a second image part, where the first image part is a gray background part, and the second image part is Black background part.
  • the management terminal completes the processing of the gesture image according to a preset preprocessing rule.
  • the method further includes:
  • the management terminal performs histogram statistics on the gesture image data and generates statistics results.
  • the management terminal uses an image segmentation algorithm to perform image segmentation on the preprocessed gesture image data, including:
  • the management terminal uses the Kinect algorithm to complete the segmentation of the gesture image data.
  • the management terminal extracts the features of the target gesture from the gesture image data for which the image segmentation is completed according to a preset feature extraction rule, including:
  • the management terminal draws a single-pixel line of the center line of the finger in the gesture image data
  • the management terminal calculates the directional characteristics of the finger according to the pixel coordinates of the single-pixel line;
  • the management terminal determines the position feature of each finger according to the position distribution of all the pixels of each finger segmented in the circumferential surface of the gesture;
  • the management terminal extracts the shape feature of the target gesture by using the segmented image matrix feature of the invariable finger scale.
  • the management terminal using a gesture recognition model to recognize the characteristics of the target gesture includes:
  • the management terminal constructs a hierarchical decision classifier embedded with a deep sparse autoencoder to classify and recognize gestures step by step;
  • the hierarchical decision classifier embedded in the deep sparse autoencoder includes a finger direction classifier, a finger position classifier, and a finger shape classifier.
  • the management terminal completes the preprocessing of the gesture image data according to a preset preprocessing rule, including:
  • the management terminal removes noise points in the gesture image data by using a morphological processing algorithm
  • the management terminal determines the gesture center point, and uses the gesture center point as a reference to remove the image data of the wrist part from the gesture image data;
  • the management terminal uses the Hough transform algorithm to detect the direction of the linear feature on the contour line of the gesture based on the image data of the removed part of the wrist, and then compares the gesture image data with the average value of the direction of the linear feature.
  • the gesture image is rotated and corrected to the vertical direction.
  • the existing gesture recognition technology is based on the high-definition camera shooting the target gesture to form a high-definition image of the target gesture, and then completes the recognition and restoration of the gesture based on the high-definition image.
  • This method has high requirements on the camera, and because of how many high-definition images are formed by this method This pixel color leads to insufficient recognition accuracy of gestures. For example, the color of the skin color of the hand reduces the accuracy of gesture recognition or increases the difficulty of gesture recognition.
  • this application puts the gesture into the hand shadow former and combines The hand shadow of the gesture is captured through the camera, and the recognition and restoration of the gesture is completed through the management terminal. This method does not require a high-definition camera or high-definition images of the gesture.
  • the gesture recognition method based on the image in this application can avoid the influence of other colors on the recognition accuracy.
  • the gesture recognition method of the embodiment of the present application can restore the gesture based on the hand shadow data of a certain gesture, and the present application does not need to use a high-standard camera device to capture instantaneous gestures, and the present application has high recognition accuracy and low cost. advantage.
  • Figure 2 is a schematic structural diagram of a hand shadow recognition system provided in the second embodiment of the present application.
  • the system includes a hand shadow former and a management terminal, wherein:
  • the hand shadow former is used to use a camera to shoot shadows formed under at least two angles of the target gesture
  • the hand shadow former is further configured to send the gesture image data generated by the camera to the management terminal after using the camera to shoot the shadow formed at least two angles of the target gesture;
  • the management terminal is configured to complete the preprocessing of the gesture image data according to preset preprocessing rules, and use an image segmentation algorithm to perform image segmentation on the preprocessed gesture image data;
  • the features of the target gesture are extracted from the segmented gesture image data;
  • the management terminal uses a gesture recognition model to recognize the features of the target gesture, and then completes the recognition of the target gesture, wherein the gesture recognition model includes a sparse autoencoder Hierarchical decision classifier.
  • the management terminal completes the preprocessing of the gesture image data according to a preset preprocessing rule, including:
  • An image binarization algorithm is used to split the filtered gesture image data into a first image part and a second image part, wherein the first image part is a gray background part and the second image part is a black background part.
  • the management terminal completes the processing of the gesture image according to a preset preprocessing rule.
  • the method further includes:
  • the management terminal performs histogram statistics on the gesture image data and generates statistics results.
  • the management terminal uses an image segmentation algorithm to perform image segmentation on the preprocessed gesture image data, including:
  • the Kinect algorithm is used to complete the segmentation of the gesture image data.
  • the management terminal extracts the features of the target gesture from the gesture image data for which the image segmentation is completed according to a preset feature extraction rule, including:
  • the management terminal draws a single-pixel line of the center line of the finger in the gesture image data
  • the management terminal calculates the directional characteristics of the finger according to the pixel coordinates of the single-pixel line;
  • the management terminal determines the position feature of each finger according to the position distribution of all the pixels of each finger segmented in the circumferential surface of the gesture;
  • the management terminal extracts the shape feature of the target gesture by using the segmented image matrix feature of the invariable finger scale.
  • the management terminal using a gesture recognition model to recognize the characteristics of the target gesture includes:
  • the management terminal constructs a hierarchical decision classifier embedded with a deep sparse autoencoder to classify and recognize gestures step by step;
  • the hierarchical decision classifier embedded in the deep sparse autoencoder includes a finger direction classifier, a finger position classifier, and a finger shape classifier.
  • the management terminal completes the preprocessing of the gesture image data according to a preset preprocessing rule, including:
  • the management terminal removes noise points in the gesture image data by using a morphological processing algorithm
  • the management terminal determines the gesture center point, and uses the gesture center point as a reference to remove the image data of the wrist part from the gesture image data;
  • the management terminal uses the Hough transform algorithm to detect the direction of the linear feature on the contour line of the gesture based on the image data of the removed part of the wrist, and then compares the gesture image data with the average value of the direction of the linear feature.
  • the gesture image is rotated and corrected to the vertical direction.
  • the system of the embodiment of the present application can restore the gesture according to the hand shadow data of a certain gesture, and the present application does not need to use a high-standard camera device to capture instantaneous gestures, and the present application has high recognition accuracy, The advantage of low cost.
  • FIG. 3 is a schematic structural diagram of a management terminal provided in Embodiment 3 of the present application, where the management terminal includes:
  • the memory 301 is configured to store machine-readable instructions. When the instructions are executed by the processor 302, the processor 302 executes the gesture recognition method as described in the first embodiment of the present application.
  • the management terminal of the embodiment of this application can restore the gesture based on the hand shadow data of a certain gesture by executing the gesture recognition method, and this application does not need to use a high-standard camera device to capture instantaneous gestures, and this application has high recognition accuracy. , The advantages of low cost.
  • the instruction may also be divided into one or more modules, and the one or more modules are stored in the memory 301 and executed by the processor 302 to complete the application.
  • the module referred to in this application refers to a series of computer program instruction segments that can complete specific functions.
  • the instructions can be divided into: a data receiving module, a preprocessing module, a segmentation module, a feature extraction module, and a gesture recognition module.
  • the functions or operation steps implemented by the above modules are all similar to the above, and will not be omitted here. To elaborate, exemplarily, for example where:
  • the data receiving module receives the gesture image data sent by the hand shadow former, wherein the gesture image data is based on at least two corresponding target gestures taken by the camera in the hand shadow former and placed in the hand shadow former. The shadows formed at different angles are generated;
  • the preprocessing module completes the preprocessing of the gesture image data according to preset preprocessing rules
  • the segmentation module uses an image segmentation algorithm to segment the preprocessed gesture image data
  • a feature extraction module which extracts the feature of the target gesture from the gesture image data after the image segmentation is completed according to a preset feature extraction rule
  • the gesture recognition module uses a gesture recognition model to recognize the features of the target gesture, and then completes the recognition of the target gesture, wherein the gesture recognition model includes a hierarchical decision classifier of a sparse autoencoder.
  • the fourth embodiment of the present application provides a computer non-volatile readable storage medium, the computer non-volatile readable storage medium stores a computer program, and the computer program is executed by a processor as described in the first embodiment of the present application.
  • the computer non-volatile readable storage medium of the embodiment of the present application can restore a certain gesture based on the hand shadow data of the gesture by executing the gesture recognition method, and the present application does not need to use a high-standard camera device to capture instantaneous gestures. Furthermore, this application has the advantages of high recognition accuracy and low cost.
  • first and second are only used for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of indicated technical features. Therefore, the features defined with “first” and “second” may explicitly or implicitly include one or more of these features. In the description of this application, unless otherwise specified, “plurality” means two or more.
  • connection should be understood in a broad sense, unless otherwise clearly specified and limited.
  • it can be a fixed connection or a detachable connection.
  • Connected, or integrally connected it can be directly connected, or indirectly connected through an intermediate medium, and it can be the internal communication between two components.
  • connection should be understood in a broad sense, unless otherwise clearly specified and limited.
  • it can be a fixed connection or a detachable connection.
  • Connected, or integrally connected it can be directly connected, or indirectly connected through an intermediate medium, and it can be the internal communication between two components.
  • the specific meanings of the above-mentioned terms in this application can be understood under specific circumstances.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Data Mining & Analysis (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Evolutionary Biology (AREA)
  • Evolutionary Computation (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • General Engineering & Computer Science (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Artificial Intelligence (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Human Computer Interaction (AREA)
  • Image Analysis (AREA)
  • User Interface Of Digital Computer (AREA)

Abstract

La présente demande concerne un procédé et un système de reconnaissance de geste, ainsi qu'un terminal de gestion et un support de stockage lisible par ordinateur. Le procédé de reconnaissance de geste est appliqué à un système de reconnaissance d'ombre de main. Le procédé de reconnaissance de geste comprend les étapes suivantes : la mise en place d'un geste cible dans un dispositif de formation d'ombre de main, l'utilisation d'un appareil photo dans le dispositif de formation d'ombre de main pour photographier des ombres formées sous au moins deux angles du geste cible, l'envoi par le dispositif de formation d'ombre de main au terminal de gestion des données d'image de geste générées par l'appareil photo, l'achèvement par le terminal de gestion, selon une règle de prétraitement prédéfinie, du prétraitement des données d'image de geste, l'utilisation par le terminal de gestion d'un algorithme de segmentation d'image pour effectuer une segmentation d'image sur les données d'image de geste prétraitées, et d'autres étapes. La présente demande peut restaurer un certain geste en fonction de données d'ombre de main du geste, et la présente demande ne nécessite pas l'utilisation d'un appareil photographique de spécification élevée pour capturer des gestes instantanés, de telle sorte que la présente demande présente les avantages d'une grande précision de reconnaissance et d'un faible coût.
PCT/CN2019/117813 2019-09-16 2019-11-13 Procédé et système de reconnaissance de geste, ainsi que terminal de gestion et support de stockage lisible par ordinateur WO2021051575A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201910870947.0 2019-09-16
CN201910870947.0A CN110738118B (zh) 2019-09-16 2019-09-16 手势识别方法、系统及管理终端、计算机可读存储介质

Publications (1)

Publication Number Publication Date
WO2021051575A1 true WO2021051575A1 (fr) 2021-03-25

Family

ID=69268043

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2019/117813 WO2021051575A1 (fr) 2019-09-16 2019-11-13 Procédé et système de reconnaissance de geste, ainsi que terminal de gestion et support de stockage lisible par ordinateur

Country Status (2)

Country Link
CN (1) CN110738118B (fr)
WO (1) WO2021051575A1 (fr)

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2012129649A1 (fr) * 2011-03-31 2012-10-04 Smart Technologies Ulc Reconnaissance de gestes par traitement d'ombres
CN106022319A (zh) * 2016-06-30 2016-10-12 联想(北京)有限公司 一种手势识别方法及手势识别系统
CN106462227A (zh) * 2014-08-07 2017-02-22 日立麦克赛尔株式会社 投影型影像显示装置及其控制方法
CN109359566A (zh) * 2018-09-29 2019-02-19 河南科技大学 利用手指特征进行层级分类的手势识别方法

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR101483741B1 (ko) * 2014-02-10 2015-01-27 전남대학교산학협력단 슬라이스 패턴 영상을 이용한 터치 인식 방법 및 그 시스템
EP2980728A1 (fr) * 2014-08-01 2016-02-03 Imersivo, S.L. Méthode d'identification d'un geste manuel
CN107844759A (zh) * 2017-10-24 2018-03-27 努比亚技术有限公司 一种手势识别方法、终端及存储介质
CN107846555A (zh) * 2017-11-06 2018-03-27 深圳慧源创新科技有限公司 基于手势识别的自动拍摄方法、装置、用户终端及计算机存储介质
CN108133206B (zh) * 2018-02-11 2020-03-06 辽东学院 静态手势识别方法、装置及可读存储介质
CN108647597B (zh) * 2018-04-27 2021-02-02 京东方科技集团股份有限公司 一种手腕识别方法、手势识别方法、装置和电子设备

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2012129649A1 (fr) * 2011-03-31 2012-10-04 Smart Technologies Ulc Reconnaissance de gestes par traitement d'ombres
CN106462227A (zh) * 2014-08-07 2017-02-22 日立麦克赛尔株式会社 投影型影像显示装置及其控制方法
CN106022319A (zh) * 2016-06-30 2016-10-12 联想(北京)有限公司 一种手势识别方法及手势识别系统
CN109359566A (zh) * 2018-09-29 2019-02-19 河南科技大学 利用手指特征进行层级分类的手势识别方法

Also Published As

Publication number Publication date
CN110738118A (zh) 2020-01-31
CN110738118B (zh) 2023-07-07

Similar Documents

Publication Publication Date Title
WO2019100608A1 (fr) Dispositif de capture vidéo, procédé de reconnaissance de visage, système, et support d'informations lisible par ordinateur
CN112287866B (zh) 一种基于人体关键点的人体动作识别方法及装置
CN110008806B (zh) 信息处理装置、学习处理方法、学习装置及物体识别装置
Soltany et al. Fast and accurate pupil positioning algorithm using circular Hough transform and gray projection
WO2021008252A1 (fr) Appareil et procédé de reconnaissance de position d'une personne dans une image, dispositif informatique et support d'enregistrement
US11682231B2 (en) Living body detection method and device
WO2016034059A1 (fr) Procédé de suivi d'objet cible utilisant des caractéristiques de couleur-structure
CN110852310B (zh) 三维人脸识别方法、装置、终端设备及计算机可读介质
CN111079545A (zh) 一种基于图像修复的三维目标检测方法和系统
CN107330397A (zh) 一种基于大间隔相对距离度量学习的行人重识别方法
CN108335331A (zh) 一种钢卷双目视觉定位方法及设备
WO2021098163A1 (fr) Procédé de détection de cible aérienne basée sur des coins
CN112287867B (zh) 一种多摄像头的人体动作识别方法及装置
WO2023045183A1 (fr) Traitement d'images
WO2020248848A1 (fr) Procédé et dispositif de détermination intelligente de cellule anormale, et support d'informations lisible par ordinateur
WO2020173024A1 (fr) Procédé de segmentation précise multi-geste pour scénario de maison intelligente
US11494922B2 (en) Object tracking device, object tracking method, and object tracking program
CN110570422A (zh) 一种基于矩阵分析的胶囊缺陷视觉检测方法
CN113688817A (zh) 一种自动巡检的仪表识别方法及识别系统
CN110929695A (zh) 一种人脸识别和行人重识别关联方法
CN113034544A (zh) 一种基于深度相机的人流分析方法及装置
WO2021051575A1 (fr) Procédé et système de reconnaissance de geste, ainsi que terminal de gestion et support de stockage lisible par ordinateur
CN109766860B (zh) 基于改进的Adaboost算法的人脸检测方法
TWI536280B (zh) 街景影像之文字區域偵測系統及裝置
CN106603888A (zh) 图像颜色提取处理结构

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19945701

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 25.07.2022)

122 Ep: pct application non-entry in european phase

Ref document number: 19945701

Country of ref document: EP

Kind code of ref document: A1