WO2020173024A1 - Procédé de segmentation précise multi-geste pour scénario de maison intelligente - Google Patents

Procédé de segmentation précise multi-geste pour scénario de maison intelligente Download PDF

Info

Publication number
WO2020173024A1
WO2020173024A1 PCT/CN2019/092970 CN2019092970W WO2020173024A1 WO 2020173024 A1 WO2020173024 A1 WO 2020173024A1 CN 2019092970 W CN2019092970 W CN 2019092970W WO 2020173024 A1 WO2020173024 A1 WO 2020173024A1
Authority
WO
WIPO (PCT)
Prior art keywords
gesture
image
smart home
area
segmentation method
Prior art date
Application number
PCT/CN2019/092970
Other languages
English (en)
Chinese (zh)
Inventor
张晖
张迪
赵海涛
孙雁飞
朱洪波
Original Assignee
南京邮电大学
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 南京邮电大学 filed Critical 南京邮电大学
Priority to JP2020515730A priority Critical patent/JP6932402B2/ja
Publication of WO2020173024A1 publication Critical patent/WO2020173024A1/fr

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/017—Gesture based interaction, e.g. based on a set of recognized hand gestures
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/50—Extraction of image or video features by performing operations within image blocks; by using histograms, e.g. histogram of oriented gradients [HoG]; by summing image-intensity values; Projection analysis
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/56—Extraction of image or video features relating to colour
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/20—Movements or behaviour, e.g. gesture recognition
    • G06V40/28—Recognition of hand or arm movements, e.g. recognition of deaf sign language

Definitions

  • the invention relates to an intelligent recognition method, in particular to a multi-gesture accurate segmentation method oriented to a smart home scene, and belongs to the field of smart home.
  • Gesture segmentation refers to the technology of segmenting gesture information from the complex image background.
  • the quality of gesture segmentation is good or bad for the recognition and detection accuracy of gesture-based human-computer interaction systems. important influence.
  • Real-time gesture segmentation in home-oriented scenarios is more complex.
  • User gestures are not only more complex and changeable, but also vulnerable to factors such as background, lighting, and shooting angle.
  • In the current computer vision field there is no adaptive gesture segmentation algorithm.
  • Some of the current representative gesture segmentation methods mainly rely on external devices or require special processing of the user’s hands. However, they limit the range of people's activities and require supporting hardware equipment, cost and expensive. The technology is also difficult to be widely promoted in practical applications.
  • gesture segmentation products only focus on segmentation of the skin, and cannot completely and accurately segment the gestures.
  • the segmentation effect is not ideal.
  • most of these devices rely on cloud servers and rely too much on the network, and will not be able to work without a network.
  • the purpose of the present invention is to propose a multi-gesture accurate segmentation method for smart home scenes, including the following steps:
  • the image Image5 is processed by the arm redundancy removal algorithm based on the hand shape feature to complete the removal of the arm redundancy.
  • the preprocessing in S1 at least includes: gesture image denoising, gesture image binarization, and morphological processing.
  • S2 specifically includes the following steps:
  • S3 specifically includes the following steps: store the contour information of the binarized gesture image obtained in S2 into the list contours, and obtain the coordinates of the four vertices of the circumscribed rectangle according to the coordinate information, which are top_left, top_right, and bottom_left. With bottom_right.
  • the non-gesture area exclusion criterion in S4 specifically includes:
  • the arm redundancy removal algorithm based on hand shape features described in S5 specifically includes: counting the hand width distribution histogram and gradient distribution histogram of the image Image6, wherein the width of the gesture width distribution histogram is the largest The value and its corresponding coordinate are the thumb carpal joint, and the coordinates of the wrist dividing line are determined by finding the value in the histogram of the gradient distribution after the thumb carpal joint point.
  • the coordinates of the wrist dividing line in step S5 are determined by finding the value in the histogram of the gradient distribution after the thumb carpal joint point, and the determination criterion is: the gradient of the current point is 0, and the gradient of the next point is greater than or equal to 0.
  • the multi-gesture precise segmentation method for smart home scenes proposed by the present invention can segment the gestures intelligently locally, overcomes the drawbacks of the prior art that is too dependent on the network, and makes the device applying this method not connected to the network. It can still work normally.
  • the present invention completes the segmentation of the skin color by converting the gesture picture from the RGB color space to the YCbCr color space, and then by the method of global fixed threshold binarization. Subsequently, the non-gesture area is excluded, the MBR and MABR of the gesture contour are constructed, the gesture image is rotated to count the hand width, the width distribution histogram and the width-based gradient distribution histogram are constructed, and the wrist division line is determined. Finally, the removal of arm redundancy is completed, and a complete gesture image is obtained.
  • the invention can quickly and accurately segment the gestures in the home environment image, significantly improves the use comfort of the gesture-based human-computer interaction system, and improves user satisfaction.
  • the present invention also provides a reference for other related issues in the same field, which can be used as a basis for expansion and application in other technical solutions related to gesture segmentation, and has very broad application prospects.
  • FIG. 1 is a schematic diagram of the steps of performing skin color segmentation on gesture images provided by the present invention
  • FIG. 2 is a schematic flow chart of steps for removing arm redundancy from gesture images provided by the present invention
  • FIG. 3 is a schematic flow diagram of the overall steps of the multi-gesture precise segmentation method for smart home scenes provided by the present invention.
  • the present invention discloses a multi-gesture precise segmentation method for smart home scenes.
  • the method is based on a skin color segmentation algorithm of YCbCr color space, non-gesture region exclusion criteria, and an arm redundancy removal algorithm based on hand shape features.
  • the method of the present invention includes the following steps:
  • the image Image5 is processed by the arm redundancy removal algorithm based on the hand shape feature to complete the removal of the arm redundancy.
  • the method of the present invention mainly includes two major aspects, namely skin color segmentation and arm redundancy removal.
  • FIG. 1 shows a method for segmenting a gesture image according to an embodiment of the present invention.
  • the steps of the method mainly include:
  • the preprocessing at least includes: gesture image denoising, gesture image binarization and morphological processing.
  • the gesture image denoising mainly uses a Gaussian filter, which is a linear filter.
  • the pixel value of the filter window obeys the Gaussian distribution, and decreases with the increase of the distance from the center of the template. Its two-dimensional Gaussian function for:
  • h(x,y) represents the value on the (x,y) coordinate in the Gaussian filter
  • ⁇ represents the standard deviation
  • YCbCr color space is a commonly used color space in video images and digital images. Contains three components: Y (luma, brightness), which represents the brightness and darkness of the image, ranging from 0 to 255; Cb component represents the blue component in the RGB color space and the brightness value in the RGB color space The value range of the difference is 0-255; the Cr component represents the difference between the value of the red component in the RGB color space and the brightness in the RGB color space, and the value range is 0-255.
  • the Cb component and the Cr component are independent of each other and can be effectively separated from the Y component.
  • each pixel is compared with the threshold.
  • the specific operation is that the Y, Cb, and Cr values of human skin are approximately [0:256,130:174,77:128], if the YCbCr value of the pixel in the image If it belongs to this interval, the pixel value is set to 255, otherwise it is set to 0, and the binary image Image3 can be obtained.
  • the resulting image will have gaps and defects.
  • the role of morphology is to remove isolated dots, burrs, fill small holes, bridge small gaps, etc. There are four main types of morphological operations:
  • the corrosion calculation process in the morphological operation is to eliminate all boundary points of the object. As a result, the area of the target object becomes smaller; its significance is to eliminate some small meaningless isolated points in the target area.
  • Open operation The opening operation process in the morphological operation first performs the erosion operation on the binary image, and then performs the expansion operation on it. Its significance is to eliminate the isolated small dots, burrs and other meaningless points in the target area (corrosion operation), and fill cavities and gaps (expansion operation).
  • the closing operation process in the morphological operation first performs dilation operation on the binary image, and then performs erosion operation on it. Its significance lies in filling the voids and gaps in the target area (expansion operation), and eliminating isolated small dots, burrs and other meaningless points (corrosion operation).
  • the median filter is a non-linear filter, which mainly counts and sorts the surrounding pixels of the current point, and selects the median value as the pixel value of the current point, thereby eliminating isolated noise points.
  • the median filter is mainly used to smooth the burrs on the edges of the gesture binarization image, so that the edges are smoothed, and the influence of the search of the wrist division line is reduced.
  • the steps of the method mainly include:
  • the MABR of the image is constructed on the basis of the MBR.
  • the convex hull of the gesture contour can be obtained according to the Graham scanning method.
  • the center of the MBR of the graphic is the origin, and the ⁇ is the scale within the 90 degree range, etc. Rotate at intervals.
  • the MBR area of the graphics under the corresponding rotation angle is recorded, and the MBR corresponding to the smallest MBR area in the record is the required MABR.
  • the non-gesture area exclusion criteria specifically include:
  • the image Image5 is processed by the arm redundancy removal algorithm based on the hand shape feature to complete the removal of the arm redundancy.
  • the arm redundancy removal algorithm based on hand shape features described in S5 specifically includes: Counting image Image6's hand width distribution histogram and gradient distribution histogram, where the maximum width of the gesture width distribution histogram and its The corresponding coordinate is the thumb carpal joint, and the coordinates of the wrist dividing line are determined by finding the value in the histogram of the gradient distribution after the thumb carpal joint point.
  • the coordinates of the wrist segmentation line can be found by looking for the value in the gradient distribution histogram after the thumb carpal joint point To determine, the method for determining is: the gradient of the current point is 0, and the gradient of the next point is greater than or equal to 0.
  • Step S301 image acquisition
  • the home image is collected mainly through a 2D camera.
  • Step S302 preprocessing the collected image
  • Step S303 perform skin tone segmentation on the image
  • Step S304 filtering the non-gesture area
  • the MBR of the gesture image is first constructed, regions that do not meet the conditions are filtered, and those that meet the conditions are subjected to gesture segmentation processing.
  • Step S305 perform gesture segmentation on the image
  • the MABR is constructed on the basis of the gesture image MBR, and the deflection angle of the gesture image is obtained.
  • the histogram of the hand width distribution and the hand gradient distribution histogram the division line of the wrist of the gesture is obtained, and the arm area is filtered.
  • Step S306 obtaining a complete gesture image
  • gesture segmentation After the gesture segmentation, 0 to multiple gestures will be generated, and all the gestures in the image can be extracted for subsequent needs. It is mainly used for gesture-based human-computer interaction systems to realize people's control of home equipment through gestures.
  • the multi-gesture precise segmentation method for smart home scenes proposed by the present invention can segment the gestures intelligently locally, overcomes the drawbacks of the prior art that is too dependent on the network, and makes the device applying this method not connected to the network. It can still work normally.
  • the present invention completes the segmentation of the skin color by converting the gesture picture from the RGB color space to the YCbCr color space, and then by the method of global fixed threshold binarization. Subsequently, the non-gesture area is excluded, the MBR and MABR of the gesture contour are constructed, the gesture image is rotated to count the hand width, the width distribution histogram and the width-based gradient distribution histogram are constructed, and the wrist division line is determined. Finally, the removal of arm redundancy is completed, and a complete gesture image is obtained.
  • the invention can quickly and accurately segment the gestures in the home environment image, significantly improves the use comfort of the gesture-based human-computer interaction system, and improves user satisfaction.
  • the present invention also provides a reference for other related issues in the same field, which can be used as a basis for expansion and application in other technical solutions related to gesture segmentation, and has very broad application prospects.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Human Computer Interaction (AREA)
  • General Engineering & Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Psychiatry (AREA)
  • Social Psychology (AREA)
  • General Health & Medical Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Image Analysis (AREA)
  • User Interface Of Digital Computer (AREA)

Abstract

La présente invention concerne un procédé de segmentation précise multi-geste pour un scénario de maison intelligente, comprenant les étapes suivantes: S1: le prétraitement d'une image de geste d'image0 pour obtenir une image1 d'image; S2: la réalisation d'une segmentation de ton de peau sur l'image1 de l'image prétraitée pour obtenir une image4 d'image traitée; S3: la construction d'un rectangle de délimitation minimale d'image (MBR) dans l'image4 d'image; S4: l'exclusion des zones sans geste dans l'image4 de l'image au moyen d'un critère d'exclusion de zone sans geste pour acquérir une image5 d'image de geste; et S5: le traitement de l'image5 de l'image au moyen d'un algorithme d'élimination de redondance de bras sur la base de caractéristiques de forme de main pour mettre en oeuvre l'élimination de redondance de bras. La présente invention peut segmenter localement et de manière intelligente des gestes, et l'ensemble du processus est rapide et précis, augmentant significativement le confort d'utilisation de systèmes d'interaction homme-machine basés sur des gestes.
PCT/CN2019/092970 2019-02-26 2019-06-26 Procédé de segmentation précise multi-geste pour scénario de maison intelligente WO2020173024A1 (fr)

Priority Applications (1)

Application Number Priority Date Filing Date Title
JP2020515730A JP6932402B2 (ja) 2019-02-26 2019-06-26 スマートホームシーン向けのマルチジェスチャー精分割方法

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201910140430.6A CN109961016B (zh) 2019-02-26 2019-02-26 面向智能家居场景的多手势精准分割方法
CN201910140430.6 2019-02-26

Publications (1)

Publication Number Publication Date
WO2020173024A1 true WO2020173024A1 (fr) 2020-09-03

Family

ID=67023818

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2019/092970 WO2020173024A1 (fr) 2019-02-26 2019-06-26 Procédé de segmentation précise multi-geste pour scénario de maison intelligente

Country Status (3)

Country Link
JP (1) JP6932402B2 (fr)
CN (1) CN109961016B (fr)
WO (1) WO2020173024A1 (fr)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112613355A (zh) * 2020-12-07 2021-04-06 北京理工大学 一种基于寻岛算法的手势分割方法

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112070859B (zh) * 2020-09-16 2021-05-04 山东晨熙智能科技有限公司 一种照片书的照片图像自动填充方法及系统
CN112949542A (zh) * 2021-03-17 2021-06-11 哈尔滨理工大学 一种基于凸包检测的手腕分割线的确立方法
CN113204991B (zh) * 2021-03-25 2022-07-15 南京邮电大学 一种基于多层预处理的快速人脸检测方法

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20110282140A1 (en) * 2010-05-14 2011-11-17 Intuitive Surgical Operations, Inc. Method and system of hand segmentation and overlay using depth data
CN103426000A (zh) * 2013-08-28 2013-12-04 天津大学 一种静态手势指尖检测方法
CN106557173A (zh) * 2016-11-29 2017-04-05 重庆重智机器人研究院有限公司 动态手势识别方法及装置
CN109190496A (zh) * 2018-08-09 2019-01-11 华南理工大学 一种基于多特征融合的单目静态手势识别方法

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH11167455A (ja) * 1997-12-05 1999-06-22 Fujitsu Ltd 手形状認識装置及び単色物体形状認識装置
JP4332649B2 (ja) * 1999-06-08 2009-09-16 独立行政法人情報通信研究機構 手の形状と姿勢の認識装置および手の形状と姿勢の認識方法並びに当該方法を実施するプログラムを記録した記録媒体
CN106325485B (zh) * 2015-06-30 2019-09-10 芋头科技（杭州）有限公司 一种手势检测识别方法及系统
CN108345867A (zh) * 2018-03-09 2018-07-31 南京邮电大学 面向智能家居场景的手势识别方法
CN109214297A (zh) * 2018-08-09 2019-01-15 华南理工大学 一种结合深度信息与肤色信息的静态手势识别方法

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20110282140A1 (en) * 2010-05-14 2011-11-17 Intuitive Surgical Operations, Inc. Method and system of hand segmentation and overlay using depth data
CN103426000A (zh) * 2013-08-28 2013-12-04 天津大学 一种静态手势指尖检测方法
CN106557173A (zh) * 2016-11-29 2017-04-05 重庆重智机器人研究院有限公司 动态手势识别方法及装置
CN109190496A (zh) * 2018-08-09 2019-01-11 华南理工大学 一种基于多特征融合的单目静态手势识别方法

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
CAO, XINYAN : "Monocular Vision Gesture Segmentation Based on Skin Color and Motion Detection", JOURNAL OF HUNAN UNIVERSITY (NATURAL SCIENCE), vol. 38, no. 01, 31 January 2011 (2011-01-31), pages 1 - 83, XP009522952 *
GONG, TAOBO: "Computer Vision-Based Static Hand Gesture Recognition System", ELECTRONIC TECHNOLOGY & INFORMATION SCIENCE, CHINA MASTER’S THESES FULL-TEXT DATABASE, 15 September 2008 (2008-09-15), pages 1 - 56, XP009522953, ISSN: 1674-0246 *

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112613355A (zh) * 2020-12-07 2021-04-06 北京理工大学 一种基于寻岛算法的手势分割方法
CN112613355B (zh) * 2020-12-07 2022-07-26 北京理工大学 一种基于寻岛算法的手势分割方法

Also Published As

Publication number Publication date
JP2021517281A (ja) 2021-07-15
CN109961016A (zh) 2019-07-02
CN109961016B (zh) 2022-10-14
JP6932402B2 (ja) 2021-09-08

Similar Documents

Publication Publication Date Title
WO2020173024A1 (fr) Procédé de segmentation précise multi-geste pour scénario de maison intelligente
EP1969559B1 (fr) Detection des contours lors de la segmentation de sequences video
EP1969560B1 (fr) Fermeture morphologique guidee par les contours lors de la segmentation de sequences video
JP5538909B2 (ja) 検出装置およびその方法
EP1969562B1 (fr) Fermeture morphologique guidee par les contours lors de la segmentation de sequences video
WO2007076890A1 (fr) Segmentation de sequences video
US10885321B2 (en) Hand detection method and system, image detection method and system, hand segmentation method, storage medium, and device
US20150319373A1 (en) Method and device to compose an image by eliminating one or more moving objects
EP2863362B1 (fr) Procédé et appareil pour une segmentation de scène à partir d'images de pile focale
JPH0877334A (ja) 顔画像の特徴点自動抽出方法
CN105894464A (zh) 一种中值滤波图像处理方法和装置
JP2010057105A (ja) オブジェクトの３次元追跡方法およびシステム
CN111539980A (zh) 一种基于可见光的多目标追踪方法
Ansari et al. An approach for human machine interaction using dynamic hand gesture recognition
CN109934152B (zh) 一种针对手语图像的改进小弯臂图像分割方法
CN111831123B (zh) 一种适用于桌面混合现实环境的手势交互方法及系统
Huang et al. M2-Net: multi-stages specular highlight detection and removal in multi-scenes
Avinash et al. Color hand gesture segmentation for images with complex background
CN108717699B (zh) 一种基于连续最小割的超声波图像分割方法
CN108491820B (zh) 图像中肢体表示信息的识别方法、装置及设备、存储介质
CN107481253B (zh) 一种基于边缘的斑点检测方法
Chuang et al. Moving object segmentation and tracking using active contour and color classification models
CN114283157A (zh) 一种基于椭圆拟合的椭圆形物体分割方法
CN111932470A (zh) 基于视觉选择融合的图像复原方法、装置、设备和介质
Malavika et al. Moving object detection and velocity estimation using MATLAB

Legal Events

Date Code Title Description
ENP Entry into the national phase

Ref document number: 2020515730

Country of ref document: JP

Kind code of ref document: A

121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19805113

Country of ref document: EP

Kind code of ref document: A1

122 Ep: pct application non-entry in european phase

Ref document number: 19805113

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE