WO2018032701A1 - 跟踪指部轮廓的方法及其装置 - Google Patents

跟踪指部轮廓的方法及其装置 Download PDF

Info

Publication number
WO2018032701A1
WO2018032701A1 PCT/CN2016/113498 CN2016113498W WO2018032701A1 WO 2018032701 A1 WO2018032701 A1 WO 2018032701A1 CN 2016113498 W CN2016113498 W CN 2016113498W WO 2018032701 A1 WO2018032701 A1 WO 2018032701A1
Authority
WO
WIPO (PCT)
Prior art keywords
finger
outer contour
contour
hand
depth
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2016/113498
Other languages
English (en)
French (fr)
Inventor
杨铭
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Guangzhou Shiyuan Electronics Thecnology Co Ltd
Original Assignee
Guangzhou Shiyuan Electronics Thecnology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Guangzhou Shiyuan Electronics Thecnology Co Ltd filed Critical Guangzhou Shiyuan Electronics Thecnology Co Ltd
Publication of WO2018032701A1 publication Critical patent/WO2018032701A1/zh
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/12—Edge-based segmentation

Definitions

  • the present invention relates to the field of computer technologies, and in particular, to a method and device for tracking a finger profile.
  • Embodiments of the present invention provide a method and apparatus for tracking a finger contour, which can accurately track the outer contour of the finger.
  • an embodiment of the present invention provides a method for tracking a finger profile, including:
  • the outer contour of the finger is constrained to conform to the outer contour of the finger to fit the edge of the corresponding finger in the color image to obtain the tracking of the finger outline.
  • the initial outer contour of the finger is V 0 (s), and the tracking outer contour of the finger is among them
  • V(s) is a function describing the outer contour of the finger
  • E img (V(s)) is a target that constrains the outer contour of the finger to fit the edge of the color image corresponding to the edge of the finger
  • the function, I is a discrete function describing the image value of each pixel in the color image.
  • the outer contour of each finger is bound to the initial contour of the finger to constrain the While the outer contour of the finger fits the edge of the finger in the color image, the method further includes:
  • the obtained tracking outline of the finger is then revised to:
  • E con (V(s))
  • E int (V(s)) is the target that constrains the total length of the outer contour of the finger and the curvature of curvature
  • the function, E con (V(s)), is an objective function that constrains the outer contour of the finger toward the user-provided hand contour curve.
  • the outer contour of the hand is extracted from the depth image, specifically:
  • the centroid of the contour is selected from the outer contours closest to the average distance of the joint points, and the outer contour having the longest total length of the contour curve serves as the outer contour of the hand.
  • the method for tracking a finger contour further includes extracting a position of a finger of the outer contour of the hand, specifically:
  • a structural deviation degree of the candidate region from the local region is d(P, Q),
  • P is a set of pixel values including each pixel of the local area
  • Q is a set of pixel values including each pixel of the candidate area
  • ⁇ P is an average of all pixel values in the set P
  • ⁇ Q is the mean of all pixel values in the set Q
  • ⁇ PQ is the covariance of the set P and the set Q
  • ⁇ P is the variance of the set P
  • ⁇ Q is the square of the set Q
  • c 1 and c 2 are preset constants
  • the current location is used as the location of the fingerprint
  • the present invention also provides an apparatus for tracking a finger contour, comprising:
  • An image acquisition module configured to acquire a depth image and a color image of the same image recorded by the user's hand
  • An outer contour extraction module configured to extract an outer contour of the hand from the depth image or the color image
  • a finger contour extraction module configured to divide the outer contour of the hand with the finger of the outer contour of the hand as a dividing point of the dividing finger, to obtain an initial outer contour of each finger
  • a finger contour constraint module for constraining an outer contour of the finger with an outer contour of the finger to constrain an outer contour of the finger to fit a finger edge in the color image to obtain the finger
  • the tracking outline of the department for constraining an outer contour of the finger with an outer contour of the finger to constrain an outer contour of the finger to fit a finger edge in the color image to obtain the finger
  • the initial outer contour of the finger is V 0 (s), and the tracking outer contour of the finger is among them
  • V(s) is a function describing the outer contour of the finger
  • E img (V(s)) is a target that constrains the outer contour of the finger to fit the edge of the color image corresponding to the edge of the finger
  • the function, I is a discrete function describing the image value of each pixel in the color image.
  • the finger contour constraint module is further configured to: bind the initial outer contour of the finger treatment to constrain the Describe the total length and varying curvature of the outer contour of the finger, and constraining the outer contour of the finger toward the hand contour curve provided by the user;
  • the obtained tracking outline of the finger is then revised to:
  • E con (V(s))
  • E int (V(s)) is the target that constrains the total length of the outer contour of the finger and the curvature of curvature
  • the function, E con (V(s)), is an objective function that constrains the outer contour of the finger toward the user-provided hand contour curve.
  • the outer contour extraction module includes:
  • a node depth calculation unit configured to calculate a depth of each joint point of the hand from the depth image according to a preset hand joint point model
  • a reference depth determining unit configured to take a median of the depth of the related node as a reference depth d ref ;
  • a first contour extracting unit configured to extract, from the depth image, an outer contour of a region having a depth within a hand depth range [d ref - ⁇ , d ref + ⁇ ]; wherein ⁇ is a measure of the back of the hand The parameter value of the thickness between the palm and the palm;
  • a contour selecting unit configured to select, from the outer contour, a centroid whose contour is closest to an average distance of the joint point, and an outer contour having a longest total length of the contour curve as an outer contour of the hand;
  • a second contour extracting unit configured to extract, from the color image, an outer contour of a region of the pixel value in the hand pixel interval.
  • the apparatus for tracking a finger contour further includes a fingerprint extraction module for extracting a position of a finger of the outer contour of the hand,
  • the fingerprint extraction module includes:
  • a contour inflection point selecting unit configured to select, from a contour point between adjacent fingertips of the outer contour of the hand, a contour point farthest from a line connecting the adjacent fingertips as a contour inflection point;
  • a local area determining unit configured to extract, from the color image, a local area centered on the current position by using the contour inflection point as a current position of the fingerprint
  • a candidate region determining unit configured to offset the current location to obtain a plurality of offset locations, and for each offset location, extract, from the color image, the offset location as a center point and the local portion a region of the same shape as a candidate region; wherein the current position is (x, y), the offset position is (x + ⁇ x , y + ⁇ y ); ⁇ x ⁇ ⁇ -1, 0, 1 ⁇ , ⁇ y ⁇ -1,0,1 ⁇ , and ⁇ x and ⁇ y are not 0 at the same time;
  • a deviation degree calculation unit configured to calculate a degree of structural deviation of each of the candidate regions and the local region; for each candidate region, a structural deviation degree of the candidate region from the local region is d(P, Q) ,
  • P is a set of pixel values including each pixel of the local area
  • Q is a set of pixel values including each pixel of the candidate area
  • ⁇ P is an average of all pixel values in the set P
  • ⁇ Q is the mean of all pixel values in the set Q
  • ⁇ PQ is the covariance of the set P and the set Q
  • ⁇ P is the variance of the set P
  • ⁇ Q is the square of the set Q
  • c 1 and c 2 are preset constants
  • a fingerprint position determining unit configured to use the current position as the position of the fingerprint when the degree of structural deviation of each of the candidate regions and the local region is greater than a preset threshold
  • a current location updating unit configured to: when there is a structural deviation degree of the candidate region and the local region is not greater than the preset threshold, select a candidate region corresponding to a minimum degree of structural deviation of the local region The offset position is used to update the current location and update the local area and the candidate area.
  • the method and device for tracking the contour of a finger can acquire a depth image and a color image of the same image of the user's hand; extract the outer contour of the hand from the depth image or the color image;
  • the finger of the outer contour of the hand serves as a dividing point of the dividing finger, divides the outer contour of the hand, obtains the outer contour of each finger, and constrains the outer contour of each finger to fit the color Corresponding finger edges in the image to accurately track the finger contour of the user's hand.
  • FIG. 1 is a schematic flow chart of one embodiment of a method for tracking a finger profile provided by the present invention
  • FIG. 2 is a schematic structural view of an embodiment of an apparatus for tracking a finger profile provided by the present invention
  • FIG. 3 is a schematic structural diagram of an embodiment of an outer contour extraction module of a device for tracking a finger contour provided by the present invention
  • FIG. 4 is a schematic structural view of an embodiment of a fingerprint extraction module of a device for tracking a finger profile provided by the present invention.
  • FIG. 1 is a schematic flowchart of an embodiment of a method for tracking a finger profile provided by the present invention.
  • the method includes steps S1 to S4, specifically:
  • the depth image is an image captured by the depth camera, and the pixel value of each pixel included therein reflects the position of the object corresponding to the pixel from the camera.
  • Distance information a color image is an image captured by an ordinary camera device, and the pixels of each pixel included therein The value reflects the appearance color information of the position of the object corresponding to the pixel.
  • the centroid of the contour is selected from the outer contours closest to the average distance of the joint points, and the outer contour having the longest total length of the contour curve serves as the outer contour of the hand.
  • the outer contour of the acquired hand should be a set of coordinate points composed of a group of coordinate points.
  • the hand joint point model is a model that is trained in advance using a large number of training sets that record the depth image of the hand, including: a kinect-based hand joint point model, a multi-random forest model, etc., which is based on the hand.
  • the information generated by the depth image is trained to train the hand joint point model using a random forest algorithm.
  • the joint point of the hand provides the approximate position of each joint point of the hand, and the depth range of the entire hand can be estimated by the depth of each joint point.
  • the depth of the entire hand is within the hand depth range [d ref - ⁇ , d ref + ⁇ ], which is a measure of the back of the hand and the palm of the hand.
  • the parameter value between the thicknesses is extracted from the outer contour of the edge of the region within the range, that is, the initial outer contour of the hand.
  • the outer contour of the contour whose center of mass is closest to the joint point and the longest total length of the contour curve is selected. Just fine.
  • the centroid of the contour is selected from the outer contours closest to the average distance of the joint points, and the outer contour having the longest total length of the contour curve serves as the outer contour of the hand.
  • the skin color of the hand has a range in the RGB space, and the range can be used as the hand pixel interval in the embodiment of the present invention, and the color image is threshold-based according to the hand pixel interval, that is, The hand area is available.
  • step S3 Before performing this step S3, further comprising determining the position of each finger on the outer contour of the hand, the acquiring process of the position of each finger is:
  • a structural deviation degree of the candidate region from the local region is d(P, Q),
  • P is a set of pixel values including each pixel of the local area
  • Q is a set of pixel values including each pixel of the candidate area
  • ⁇ P is an average of all pixel values in the set P
  • ⁇ Q is the mean of all pixel values in the set Q
  • ⁇ PQ is the covariance of the set P and the set Q
  • ⁇ P is the variance of the set P
  • ⁇ Q is the square of the set Q
  • c 1 and c 2 are preset constants
  • the current location is used as the location of the fingerprint
  • the color distribution (ie, the pixel value distribution) of the local area and the adjacent candidate area are largely different; when the local area is in the finger seam, the local area and the edge finger The difference in color distribution of adjacent candidate regions in the seam direction is relatively small, and the color distribution of the local region and the other candidate regions is relatively large.
  • the degree of structural deviation of each of the candidate regions and the local region is greater than a preset threshold, it can be determined that the local region falls on the fingerprint, and the central location of the local region (the above current Position) as the position of the finger, thereby completing the correction of the position of the finger; otherwise, it can be determined that the local area falls on the finger joint, and it is necessary to continue to correct the current position of the finger and select the local area
  • the central position corresponding to the candidate region with the smallest degree of structural deviation (the above-mentioned offset position) is updated to the current position of the fingerprint, and it can be ensured that the current position of the subsequently updated fingerprint is still on the finger joint, and is not offset to the non-offset. Fingers in other positions.
  • the first embodiment is that the constraint optimization for the outer contour of each finger in step S4 may only constrain the outer contour of the finger to fit the finger edge of the color image corresponding to the finger, for example
  • V(s) refers to a point on the outer contour of the finger
  • V(s) refers to the coordinate corresponding to the point
  • the tracking outer contour of the finger is described as among them, E img (V(s)) is an objective function for constraining an outer contour of the finger to fit an edge of the color image corresponding to the finger, and I is an image describing each pixel in the color image A discrete function of the value.
  • the outer contour of each finger in step S4 is combined with the initial outer contour of the finger to constrain the outer contour of the finger to fit the finger edge in the color image,
  • the method includes: combining an initial outer contour of the finger to constrain an overall length and a varying curvature of the outer contour of the finger, and constraining an outer contour of the finger toward an initial outer contour of the finger;
  • an E int (V(s)) objective function is proposed to constrain the total length and the curvature of the outer contour of the finger, ie based on The initial outer contour V 0 (s) integrates the first term of E int (V(s)) and selects the outer contour curve corresponding to the integral minimum, thereby constraining the total length of the outer contour of the finger; and, based on The initial outer contour V 0 (s) integrates the second term of E int (V(s)) and selects the outer contour curve corresponding to the integral minimum, which can constrain the curvature of the outer contour of the finger, thereby making the final The outer contour of the obtained finger is smoother.
  • the contour point of the initial outer contour may be affected by noise and cannot meet the accuracy requirement, since it is still extracted from the depth image or the color image, its positioning is relatively robust, so the following constraints are introduced, and constrained optimization is introduced.
  • the rear contour point should not be far from the initial contour point, ie, the outer contour constraining the finger tends to the initial outer contour of the finger, for example, the outer contour of the constraining finger is minimally offset from the initial outer contour, that is, for
  • 2 is integrated on the outer contour of the initial V 0 (s) and select the minimum value of the integral outer contour corresponding to V (s) to satisfy the constraint Claim.
  • the process of the present invention adopts an iterative optimization method to obtain the tracking outer contour V * (s) of the finger, and the iterative process takes the initial outer contour as the initial iteration value; as a preferred embodiment, the gradient descent method may be selected.
  • This calculation process is an algorithm of the prior art and will not be described here.
  • the method for tracking the position of the fingertip includes, but is not limited to, tracking the human hand, and can also track the foot or other hand or foot constructed by the mold, or even the hand or foot of the animal.
  • the method for tracking the position of the fingertip can be applied to VR (Virtual Reality), for example, virtual ring trial, that is, the user raises the hand in front of the camera of the client, and the camera captures the hand of the user. And transmitting the captured depth image and color image to another receiving end, and the receiving end traces the exact position of the finger of the user's hand from the depth image and the color image according to the tracking finger position provided above.
  • VR Virtual Reality
  • the receiving end determines the wearing position of the ring on the finger according to the position of the adjacent two fingers, and optimizes the outer contour of the photographed hand, and wears the ring on the wearing position of the user's finger and the image
  • the image is returned to the client for the contour of the optimized hand, and then displayed on the client's display so that the user can know the effect of wearing the ring from the image in the display.
  • FIG. 2 is a structure of an embodiment of the apparatus for tracking the contour of a finger provided by the present invention. Schematic, the device specifically includes:
  • the image obtaining module 10 is configured to acquire a depth image and a color image of the same image recorded by the user's hand;
  • An outer contour extraction module 20 configured to extract an outer contour of the hand from the depth image or the color image
  • a finger contour extraction module 30 configured to divide the outer contour of the hand with the finger of the outer contour of the hand as a dividing point of the dividing finger, to obtain an initial outer contour of each finger;
  • a finger contour constraint module 40 for constraining an outer contour of the finger with an outer contour of the finger to constrain an outer contour of the finger to fit a finger edge in the color image, to obtain the The tracking outline of the finger.
  • the initial outer contour of the finger is V 0 (s), and the tracking outer contour of the finger is among them, V(s) is a function describing the outer contour of the finger, and E img (V(s)) is a target that constrains the outer contour of the finger to fit the edge of the color image corresponding to the edge of the finger.
  • the function, I is a discrete function describing the image value of each pixel in the color image.
  • the finger contour constraint module further configured to: in conjunction with the initial outer contour of the finger treatment, constrain the total length of the outer contour of the finger and Varying the curvature, and constraining the outer contour of the finger toward the hand contour curve provided by the user;
  • the obtained tracking outline of the finger is then revised to:
  • E con (V(s))
  • E int (V(s)) is the target that constrains the total length of the outer contour of the finger and the curvature of curvature
  • the function, E con (V(s)), is an objective function that constrains the outer contour of the finger toward the user-provided hand contour curve.
  • FIG. 3 is a schematic structural diagram of an embodiment of an outer contour extraction module of a device for tracking a finger profile provided by the present invention
  • the outer contour extraction module 20 includes:
  • a node depth calculation unit 21 configured to calculate a depth of each joint point of the hand from the depth image according to a preset hand joint point model
  • a reference depth determining unit 22 configured to take a median of the depth of the related node as a reference depth d ref ;
  • a first contour extracting unit 23 configured to extract, from the depth image, an outer contour of a region having a depth within a hand depth range [d ref - ⁇ , d ref + ⁇ ]; wherein ⁇ is a measure of the hand The parameter value of the thickness between the back of the hand and the palm of the hand;
  • a contour selecting unit 24 configured to select, from the outer contour, a centroid whose contour is closest to an average distance of the joint point, and an outer contour having a longest total length of the contour curve as an outer contour of the hand;
  • the second contour extracting unit 25 is configured to extract an outer contour of a region of the pixel pixel in the hand pixel interval from the color image.
  • the apparatus for tracking the contour of the finger further includes a fingerprint extraction module 50 for extracting the position of the fingerprint of the outer contour of the hand, as specifically shown in FIG. 4 and FIG. It is a schematic structural diagram of an embodiment of a fingerprint extraction module of a device for tracking a finger profile provided by the present invention; the fingerprint extraction module 50 includes:
  • a contour inflection point selecting unit 51 configured to select, from a contour point between adjacent fingertips of the outer contour of the hand, a contour point farthest from a line connecting the adjacent fingertips as a contour inflection point;
  • a local area determining unit 52 configured to extract, from the color image, a local area centered on the current position by using the contour inflection point as a current position of the fingerprint
  • a candidate region determining unit 53 configured to offset the current location to obtain a plurality of offset locations, and for each offset location, extract, from the color image, the offset location as a center point and the a region of the same shape of the local region as a candidate region; wherein the current position is (x, y), the offset position is (x + ⁇ x , y + ⁇ y ); ⁇ x ⁇ ⁇ -1, 0, 1 ⁇ , ⁇ y ⁇ -1,0,1 ⁇ , and ⁇ x and ⁇ y are not 0 at the same time;
  • the deviation degree calculation unit 54 is configured to calculate a structural deviation degree of each of the candidate regions and the local region; for each candidate region, a structural deviation degree of the candidate region and the local region is d (P, Q) ), Wherein P is a set of pixel values including each pixel of the local area, Q is a set of pixel values including each pixel of the candidate area, and ⁇ P is an average of all pixel values in the set P, ⁇ Q is the mean of all pixel values in the set Q, ⁇ PQ is the covariance of the set P and the set Q, ⁇ P is the variance of the set P, ⁇ Q is the square of the set Q, and c 1 and c 2 are preset constants;
  • the fingerprint position determining unit 55 is configured to use the current position as the position of the fingerprint when the degree of structural deviation of each of the candidate regions and the local region is greater than a preset threshold;
  • the current location updating unit 56 is configured to: when there is a structural deviation degree of the candidate region and the local region is not greater than the preset threshold, select a candidate region that has the smallest degree of structural deviation from the local region. The offset position to update the current location and update the local area and the candidate area.
  • the storage medium may be a magnetic disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Image Analysis (AREA)
  • Collating Specific Patterns (AREA)

Abstract

一种跟踪指部轮廓的方法和相应的装置，该方法包括：获取记录用户手部的同一影像的深度图像和彩色图像(S1)；从所述深度图像或彩色图像中提取所述手部的外轮廓(S2)；以所述手部的外轮廓的指蹼作为分割指部的分割点，分割所述手部的外轮廓，获得每个指部的初始外轮廓(S3)；对于每个指部的外轮廓，结合所述指部的初始外轮廓来约束所述指部的外轮廓贴合所述彩色图像中的对应所述指部的边缘，获得所述指部的跟踪外轮廓(S4)。该方法能准确跟踪指部的外轮廓。

Description

跟踪指部轮廓的方法及其装置 技术领域
本发明涉及计算机技术领域，尤其涉及一种跟踪指部轮廓的方法及其装置。
背景技术
在许多手部交互应用中，不仅仅需要跟踪手部的骨架坐标与朝向，还需要跟踪手指宽度等信息。为了获得精确的手指宽度，较为合理的方案是通过跟踪手指轮廓估算。而现有的手指级别的轮廓跟踪算法往往把重点放在指尖跟踪或大致手指位姿关系，较少关注手指侧边缘的精度，跟踪效果难以满足实际应用需求
发明内容
本发明实施例提出一种跟踪指部轮廓的方法及装置，能准确跟踪指部的外轮廓。
在第一方面，本发明实施例提供一种跟踪指部轮廓的方法，包括：
获取记录用户手部的同一影像的深度图像和彩色图像；
从所述深度图像或彩色图像中提取所述手部的外轮廓；
以所述手部的外轮廓的指蹼作为分割指部的分割点，分割所述手部的外轮廓，获得每个指部的初始外轮廓；
对于每个指部的外轮廓，结合所述指部的初始外轮廓来约束所述指部的外轮廓贴合所述彩色图像中的对应所述指部的边缘，获得所述指部的跟踪外轮廓。
结合第一方面，在第一方面的第一种可能的实现方式中，所述指部的初始外轮廓为V0(s)，所述指部的跟踪外轮廓为
Figure PCTCN2016113498-appb-000001
其中，
Figure PCTCN2016113498-appb-000002
V(s)为描述所述指部的外轮廓的函数，Eimg(V(s))为约束所述指部的外轮廓贴合所述彩色图像中的对应所述指部的边缘的目标函数，I为描述所述彩色图像中的每一个像素点的像数值的离散函数。
结合第一方面的第一种可能的实现方式，在第一方面的第二种可能的实现方式中，在对于每个指部的外轮廓，结合所述指部的初始外轮廓来约束所述指部的外轮廓贴合所述彩色图像中的手指边缘的同时，还包括：
结合所述指部的初始外轮廓来约束所述指部的外轮廓的总长度和变化曲率，以及约束所述指部的外轮廓趋向于所述用户提供的手部轮廓曲线；
则获得的所述指部的跟踪外轮廓修订为：
Figure PCTCN2016113498-appb-000003
其中，
Figure PCTCN2016113498-appb-000004
Econ(V(s))＝||V(s)-V0(s)||2，Eint(V(s))为约束所述指部的外轮廓的总长度和变化曲率的目标函数，Econ(V(s))为约束所述指部的外轮廓趋向于所述用户提供的手疗轮廓曲线的目标函数。
结合第一方面，在第一方面的第三种可能的实现方式中，从所述深度图像中提取所述手部的外轮廓，具体为：
根据预设的手部关节点模型，从所述深度图像中计算出所述手部的每一个关节点的深度；
取所述有关节点的深度的中值作为参考深度dref；
从所述深度图像中提取深度在手部深度范围[dref-δ，dref+δ]内的区域的外轮廓；其中，δ为衡量所述手部的手背与手掌之间厚度的参数值；
从所述外轮廓中选取轮廓的质心距离所述关节点的平均距离最近，且轮廓曲线总长度最长的外轮廓作为所述手部的外轮廓；
以及，从所述彩色图像中提取所述手部的外轮廓，具体为：
从所述彩色图像中提取像素值在手部像素区间的区域的外轮廓；
从所述外轮廓中选取轮廓的质心距离所述关节点的平均距离最近，且轮廓曲线总长度最长的外轮廓作为所述手部的外轮廓。
结合第一方面，在第一方面的第四种可能的实现方式中，所述跟踪指部轮廓的方法还包括提取所述手部的外轮廓的指蹼的位置，具体为：
从所述手部的外轮廓的相邻指尖之间的轮廓点中，选取距离连接所述相邻指尖的直线最远的轮廓点作为轮廓拐点；
以所述轮廓拐点作为指蹼的当前位置，从所述彩色图像中提取以所述当前位置为中心点的局部区域；
将所述当前位置进行偏移获得多个偏移位置，并对于每一个偏移位置，从所述彩色图像中提取以该偏移位置为中心点且与所述局部区域相同形状的区域作为候选区域；其中，所述当前位置为(x，y)，所述偏移位置为(x+δx，y+δy)；δx∈{-1，0，1}，δy∈{-1，0，1}，且δx和δy不同时为0；
计算每一个所述候选区域与所述局部区域的结构偏差程度；对于每一个候选区域，所述候选区域与所述局部区域的结构偏差程度为d(P，Q)，
Figure PCTCN2016113498-appb-000005
其中，P为包含所述局部区域的每一个像素点的像素值的集合，Q为包含所述候选区域的每一个像素点的像素值的集合，μP为集合P中所有像素值的均值，μQ为集合Q中所有像素值的均值，σPQ为集合P和集合Q的协方差，σP为集合P的方差，σQ为集合Q的方，c1和c2为预设常数；
若每一个所述候选区域与所述局部区域的结构偏差程度均大于预设阈值，则将所述当前位置作为所述指蹼的位置；
若存在一个所述候选区域与所述局部区域的结构偏差程度不大于所述预设阈值，则选取与所述局部区域的结构偏差程度最小的候选区域所对应的偏移位置来更新所述当前位置，并更新所述局部区域和所述候选区域。
相应地，在第二方面，本发明还提供一种跟踪指部轮廓的装置，包括：
图像获取模块，用于获取记录用户手部的同一影像的深度图像和彩色图像；
外轮廓提取模块，用于从所述深度图像或彩色图像中提取所述手部的外轮廓；
指部轮廓提取模块，用于以所述手部的外轮廓的指蹼作为分割指部的分割点，分割所述手部的外轮廓，获得每个指部的初始外轮廓；
指部轮廓约束模块，用于对于每个指部的外轮廓，结合所述指部的初始外轮廓来约束所述指部的外轮廓贴合所述彩色图像中的手指边缘，获得所述指部的跟踪外轮廓。
结合第二方面，在第二方面的第一种可能的实现方式中，所述指部的初始外轮廓为V0(s)，所述指部的跟踪外轮廓为
Figure PCTCN2016113498-appb-000006
其中，
Figure PCTCN2016113498-appb-000007
V(s)为描述所述指部的外轮廓的函数，Eimg(V(s))为约束所述指部的外轮廓贴合所述彩色图像中的对应所述指部的边缘的目标函数，I为描述所述彩色图像中的每一个像素点的像数值的离散函数。
结合第二方面的第一种可能的实现方式，在第二方面的第二种可能的实现方式中，所述指部轮廓约束模块还用于：结合所述指疗的初始外轮廓来约束所述指部的外轮廓的总长度和变化曲率，以及约束所述指部的外轮廓趋向于所述用户提供的手部轮廓曲线；
则获得的所述指部的跟踪外轮廓修订为：
Figure PCTCN2016113498-appb-000008
其中，
Figure PCTCN2016113498-appb-000009
Econ(V(s))＝||V(s)-V0(s)||2，Eint(V(s))为约束所述指部的外轮廓的总长度和变化曲率的目标函数，Econ(V(s))为约束所述指部的外轮廓趋向于所述用户提供的手疗轮廓曲线的目标函数。
结合第二方面，在第二方面的第三种可能的实现方式中，所述外轮廓提取模块包括：
关节点深度计算单元，用于根据预设的手部关节点模型，从所述深度图像中计算出所述手部的每一个关节点的深度；
参考深度确定单元，用于取所述有关节点的深度的中值作为参考深度dref；
第一轮廓提取单元，用于从所述深度图像中提取深度在手部深度范围[dref-δ，dref+δ]内的区域的外轮廓；其中，δ为衡量所述手部的手背与手掌之间厚度的参数值；
轮廓选取单元，用于从所述外轮廓中选取轮廓的质心距离所述关节点的平均距离最近，且轮廓曲线总长度最长的外轮廓作为所述手部的外轮廓；
第二轮廓提取单元，用于从所述彩色图像中提取像素值在手部像素区间的区域的外轮廓。
结合第二方面，在第二方面的第四种可能的实现方式中，所述跟踪指部轮廓的装置还包括用于提取所述手部的外轮廓的指蹼的位置的指蹼提取模块，所述指蹼提取模块包括：
轮廓拐点选取单元，用于从所述手部的外轮廓的相邻指尖之间的轮廓点中，选取距离连接所述相邻指尖的直线最远的轮廓点作为轮廓拐点；
局部区域确定单元，用于以所述轮廓拐点作为指蹼的当前位置，从所述彩色图像中提取以所述当前位置为中心点的局部区域；
候选区域确定单元，用于将所述当前位置进行偏移获得多个偏移位置，并对于每一个偏移位置，从所述彩色图像中提取以该偏移位置为中心点且与所述局部区域相同形状的区域作为候选区域；其中，所述当前位置为(x，y)，所述偏移位置为(x+δx，y+δy)；δx∈{-1，0，1}，δy∈{-1，0，1}，且δx和δy不同时为0；
偏差程度计算单元，用于计算每一个所述候选区域与所述局部区域的结构偏差程度；对于每一个候选区域，所述候选区域与所述局部区域的结构偏差程度为d(P，Q)，
Figure PCTCN2016113498-appb-000010
其中，P为包含所述局部区域的每一个像素点的像素值的集合，Q为包含所述候选区域的每一个像素点的像素值的集合，μP为集合P中所有像素值的均值，μQ为集合Q中所有像素值的均值，σPQ为集合P和集合Q的协方差，σP为集合P的 方差，σQ为集合Q的方，c1和c2为预设常数；
指蹼位置确定单元，用于当每一个所述候选区域与所述局部区域的结构偏差程度均大于预设阈值时，将所述当前位置作为所述指蹼的位置；
当前位置更新单元，用于当存在一个所述候选区域与所述局部区域的结构偏差程度不大于所述预设阈值时，则选取与所述局部区域的结构偏差程度最小的候选区域所对应的偏移位置来更新所述当前位置，并更新所述局部区域和所述候选区域。
实施本发明实施例，具有如下有益效果：
本发明实施例提供的跟踪指部轮廓的方法及装置，能获取记录用户手部的同一影像的深度图像和彩色图像；从所述深度图像或彩色图像中提取所述手部的外轮廓；以所述手部的外轮廓的指蹼作为分割指部的分割点，分割所述手部的外轮廓，获得每个指部的外轮廓，并约束每个指部的外轮廓贴合所述彩色图像中的相应的手指边缘，从而准确地跟踪到用户手部的指部轮廓。
附图说明
图1是本发明提供的跟踪指部轮廓的方法的一个实施例的流程示意图；
图2是本发明提供的跟踪指部轮廓的装置的一个实施例的结构示意图；
图3是本发明提供的跟踪指部轮廓的装置的外轮廓提取模块的一个实施例的结构示意图；
图4是本发明提供的跟踪指部轮廓的装置的指蹼提取模块的一个实施例的结构示意图。
具体实施方式
下面将结合本发明实施例中的附图，对本发明实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例仅仅是本发明一部分实施例，而不是全部的实施例。基于本发明中的实施例，本领域普通技术人员在没有作出创造性劳动前提下所获得的所有其他实施例，都属于本发明保护的范围。
参见图1，是本发明提供的跟踪指部轮廓的方法的一个实施例的流程示意图，该方法包括步骤S1至S4，具体为：
S1，获取记录用户手部的同一影像的深度图像和彩色图像；
需要说明的是，深度图像是由深度摄像装置捕获到被拍摄物体的图像，其所包含的每一个像素点的像素值反映的是该被拍摄物体与该像素点对应的位置距离摄像头之间的距离信息；彩色图像是由普通摄像装置捕获到被拍摄物体的图像，其所包含的每一个像素点的像素 值反映的是该被拍摄物体与该像素点对应的位置的外观颜色信息。
S2，从所述深度图像或彩色图像中提取所述手部的外轮廓；
以下将描述本步骤S2中从所述深度图像中提取所述手部的外轮廓的具体过程：
根据预设的手部关节点模型，从所述深度图像中计算出所述手部的每一个关节点的深度；
取所述有关节点的深度的中值作为参考深度dref；
从所述深度图像中提取深度在手部深度范围[dref-δ，dref+δ]内的区域的外轮廓；其中，δ为衡量所述手部的手背与手掌之间厚度的参数值；
从所述外轮廓中选取轮廓的质心距离所述关节点的平均距离最近，且轮廓曲线总长度最长的外轮廓作为所述手部的外轮廓。
在本发明实施例中，所获取到手部的外轮廓应是由一群坐标点构成的坐标点集合。手部关节点模型是预先利用大量的记录有手部的深度图像的训练集训练出来的模型，该模型包括：基于kinect的手部关节点踪模型、多随机森林模型等，其是基于手部的深度图像的信息训练生成的，可优选利用随机森林算法训练手部关节点模型。手部关节点提供手部各关节点的大致位置，并且通过各关节点的深度可以估算出整个手部的深度范围。另外，在少数情况下，计算到的部分关节点也可能因精度不足超出该手部的区域，或是因深度图像噪声导致关节点的深度误差较大，因而，为了减少这些异常关节点的影响，取所述关节点的深度的中值作为参考深度，则整个手部的深度处于手部深度范围[dref-δ，dref+δ]内，δ为衡量所述手部的手背与手掌之间厚度的参数值，则提取该范围内的区域的边缘的外轮廓出来，即为该手部的初始外轮廓。但是由于受到噪声或其它干扰区域的影响，提取出来的外轮廓可能会有多个，此时，从中选取轮廓的质心距离所述关节点的平均距离最近，且轮廓曲线总长度最长的外轮廓即可。
以下将描述本步骤S2中从所述彩色图像中提取所述手部的外轮廓的具体过程：
从所述彩色图像中提取像素值在手部像素区间的区域的外轮廓；
从所述外轮廓中选取轮廓的质心距离所述关节点的平均距离最近，且轮廓曲线总长度最长的外轮廓作为所述手部的外轮廓。
需要说明的是，手部的皮肤颜色在RGB空间中有一个范围，可以将这个范围作为本发明实施例的手部像素区间，则根据该手部像素区间对上述彩色图像作区间阈值化，即可获得手部区域。
S3，以所述手部的外轮廓的指蹼作为分割指部的分割点，分割所述手部的外轮廓，获得每个指部的初始外轮廓；
在执行本步骤S3的之前，还包括确定所述手部的外轮廓上的每一个指蹼的位置，则每一个指蹼的位置的获取过程为：
从所述手部的外轮廓的相邻指尖之间的轮廓点中，选取距离连接所述相邻指尖的直线最远的轮廓点作为轮廓拐点；
以所述轮廓拐点作为指蹼的当前位置，从所述彩色图像中提取以所述当前位置为中心点的局部区域；
将所述当前位置进行偏移获得多个偏移位置，并对于每一个偏移位置，从所述彩色图像中提取以该偏移位置为中心点且与所述局部区域相同形状的区域作为候选区域；其中，所述当前位置为(x，y)，所述偏移位置为(x+δx，y+δy)；δx和δy包括但不限于为：δx∈{-1，0，1}、δy∈{-1，0，1}；δx和δy不同时为0；
计算每一个所述候选区域与所述局部区域的结构偏差程度；对于每一个候选区域，所述候选区域与所述局部区域的结构偏差程度为d(P，Q)，
Figure PCTCN2016113498-appb-000011
其中，P为包含所述局部区域的每一个像素点的像素值的集合，Q为包含所述候选区域的每一个像素点的像素值的集合，μP为集合P中所有像素值的均值，μQ为集合Q中所有像素值的均值，σPQ为集合P和集合Q的协方差，σP为集合P的方差，σQ为集合Q的方，c1和c2为预设常数；
若每一个所述候选区域与所述局部区域的结构偏差程度均大于预设阈值，则将所述当前位置作为所述指蹼的位置；
若存在一个所述候选区域与所述局部区域的结构偏差程度不大于所述预设阈值，则选取与所述局部区域的结构偏差程度最小的候选区域所对应的偏移位置来更新所述当前位置，并更新所述局部区域和所述候选区域。
需要说明的是，当局部区域处于指蹼时，该局部区域与邻近的候选区域的颜色分布(即像素值分布)均有较大差别；当局部区域处于指缝时，该局部区域与沿指缝方向上的邻近的候选区域的颜色分布差别相对较小，且该局部区域与其他的候选区域的颜色分布差较大。因而，当在比较出每一个所述候选区域与所述局部区域的结构偏差程度均大于预设阈值时，即可判断出局部区域落在指蹼上，将该局部区域的中心位置(上述当前位置)作为指蹼的位置，从而完成对指蹼的位置的修正；反之，可判断出该局部区域落在指缝上，需要继续对该指蹼的当前位置进行修正，且选取与该局部区域的结构偏差程度最小的候选区域所对应的中心位置(上述偏移位置)更新为该指蹼的当前位置，可以确保后续更新后的指蹼的当前位置仍处于指缝上，不偏移到非指缝的其他位置上。
S4，对于每个指部的外轮廓，结合所述指部的初始外轮廓来约束所述指部的外轮廓贴合 所述彩色图像中的对应所述指部的边缘，获得所述指部的跟踪外轮廓。
对于本步骤S4的具体实施方式有以下两种，以下将描述这两种方式：
第一种实施方式是，步骤S4中的对于每个指部的外轮廓的约束优化，可仅约束所述指部的外轮廓贴合所述彩色图像中对应所述指部的手指边缘，例如，将所述指部的外轮廓描述成函数表达方式V(s)时，s是指所述指部的外轮廓上的一点，则V(s)是指该点对应的坐标；进而，上述所述指部的跟踪外轮廓描述为
Figure PCTCN2016113498-appb-000012
其中，
Figure PCTCN2016113498-appb-000013
Eimg(V(s))为约束所述指部的外轮廓贴合所述彩色图像中的对应所述指部的边缘的目标函数，I为描述所述彩色图像中每一个像素点的像数值的离散函数。由于指部的外轮廓往往(但不总是)对应于上述记录有所述用户的手部的彩色图像梯度较大的位置，则计算所述彩色图像I(V(s))的梯度，进而基于所述指部的初始外轮廓V0(s)对该梯度的变型
Figure PCTCN2016113498-appb-000014
进行积分，并选取积分的最小值所对应的外轮廓V(s)作为V*(s)，从而获取的外轮廓能够沿所述彩色图像中梯度最大的位置走向变化，即约束指部的外轮廓贴合所述彩色图像中对应所述指部的手指边缘。另外，由于梯度对噪声敏感，在计算梯度之前，采用所述彩色图像包含的像素值的标准差σ为的对所述彩色图像I进行高斯核Gσ平滑。
第二种实施方式，步骤S4中的对于每个指部的外轮廓，除了结合所述指部的初始外轮廓来约束所述指部的外轮廓贴合所述彩色图像中的手指边缘，还包括：结合所述指部的初始外轮廓来约束所述指部的外轮廓的总长度和变化曲率，以及约束所述指部的外轮廓趋向于所述指部的初始外轮廓；
基于第一种实施方式对约束所述指部的外轮廓贴合所述彩色图像中对应的所述指部的手指的边缘的举例，以下结合该举例，继续对第二种实施方式中的另外两种约束进行举例，即所述指部的跟踪外轮廓修订为：
Figure PCTCN2016113498-appb-000015
其中，
Figure PCTCN2016113498-appb-000016
Econ(V(s))＝||V(s)-V0(s)||2；
由于从步骤S3所获取的指部的外轮廓的曲线形状通常不太合理，因而提出Eint(V(s))目标函数来约束所述指部的外轮廓的总长度和变化曲率，即基于初始外轮廓V0(s)对Eint(V(s))的第一项进行积分并选取积分最小值所对应的外轮廓曲线，即可约束指部的外轮廓的总长度；以及，基于初始外轮廓V0(s)对Eint(V(s))的第二项进行积分并选取积分最小值所对应的外轮廓曲线，即可约束指部的外轮廓的变化曲率，从而使得最终获取的指部的外轮廓较为平滑。
另外，即使初始外轮廓的轮廓点可能受到噪声影响而不能满足精度要求，但由于其仍然是从深度图像或彩色图像中提取的，其定位还是相对较鲁棒的，所以引入以下约束，约束优化后的轮廓点不应远离初始轮廓点，即，约束所述指部的外轮廓趋向于所述指部的初始外轮廓，例如，约束指部的外轮廓相对初始外轮廓的偏移最小，也就是说，基于初始外轮廓V0(s)对||V(s)-V0(s)||2进行积分并选取积分最小值所对应的外轮廓V(s)即可满足该约束的要求。
对于上述计算指部的跟踪外轮廓
Figure PCTCN2016113498-appb-000017
或
Figure PCTCN2016113498-appb-000018
的过程，本发明实施例采作迭代优化方法获得指部的跟踪外轮廓V*(s)，且迭代的过程以初始外轮廓为初始迭代值；作为一种优选实施例，可选用梯度下降法，此计算过程运用是现有技术的算法，在此不再赘述。
需要说明的是，上述提供的跟踪指蹼位置的方法包括但并不限于跟踪人体手部，还可以跟踪脚部或其他由模具构造成的手部或脚部，甚至动物的手部或脚部。另外，上述提供的跟踪指蹼位置的方法可以应用于VR(Virtual Reality，虚拟现实)，例如，虚拟戒指试戴，即：用户在客户端的摄像头前举起手部，该摄像头拍摄用户的手部，并将拍摄到的深度图像和彩色图像传送到另一接收端，接收端根据以上提供的跟踪指蹼位置的方法从深度图像和彩色图像中跟踪到用户的手部的指蹼的确切位置，然后接收端根据相邻两个指蹼的位置确定戒指在手指上的佩戴位置，以及优化所拍摄的手部的外轮廓，并将戒指佩戴在所述用户的手指的佩戴位置上的图像且该图像为已优化手部的外轮廓的图像返回到客户端，进而在客户端的显示器上显示，从而用户可以从显示器中的图像获知其佩带戒指的效果。
本发明还提供一种跟踪指部轮廓的装置能够实现上述跟踪指部轮廓的方法的全部流和，具体参见图2，图2是本发明提供的跟踪指部轮廓的装置的一个实施例的结构示意图，该装置具体包括：
图像获取模块10，用于获取记录用户手部的同一影像的深度图像和彩色图像；
外轮廓提取模块20，用于从所述深度图像或彩色图像中提取所述手部的外轮廓；
指部轮廓提取模块30，用于以所述手部的外轮廓的指蹼作为分割指部的分割点，分割所述手部的外轮廓，获得每个指部的初始外轮廓；
指部轮廓约束模块40，用于对于每个指部的外轮廓，结合所述指部的初始外轮廓来约束所述指部的外轮廓贴合所述彩色图像中的手指边缘，获得所述指部的跟踪外轮廓。
作为本发明提供的一种优选实施例，所述指部的初始外轮廓为V0(s)，所述指部的跟踪外 轮廓为
Figure PCTCN2016113498-appb-000019
其中，
Figure PCTCN2016113498-appb-000020
V(s)为描述所述指部的外轮廓的函数，Eimg(V(s))为约束所述指部的外轮廓贴合所述彩色图像中的对应所述指部的边缘的目标函数，I为描述所述彩色图像中的每一个像素点的像数值的离散函数。
结合上述优选实施例，本发明提供另一种优选的实施例，所述指部轮廓约束模块还用于：结合所述指疗的初始外轮廓来约束所述指部的外轮廓的总长度和变化曲率，以及约束所述指部的外轮廓趋向于所述用户提供的手部轮廓曲线；
则获得的所述指部的跟踪外轮廓修订为：
Figure PCTCN2016113498-appb-000021
其中，
Figure PCTCN2016113498-appb-000022
Econ(V(s))＝||V(s)-V0(s)||2，Eint(V(s))为约束所述指部的外轮廓的总长度和变化曲率的目标函数，Econ(V(s))为约束所述指部的外轮廓趋向于所述用户提供的手疗轮廓曲线的目标函数。
作为本发明提供的一种优选实施例，具体参见图3，图3是本发明提供的跟踪指部轮廓的装置的外轮廓提取模块的一个实施例的结构示意图；该外轮廓提取模块20包括：
关节点深度计算单元21，用于根据预设的手部关节点模型，从所述深度图像中计算出所述手部的每一个关节点的深度；
参考深度确定单元22，用于取所述有关节点的深度的中值作为参考深度dref；
第一轮廓提取单元23，用于从所述深度图像中提取深度在手部深度范围[dref-δ，dref+δ]内的区域的外轮廓；其中，δ为衡量所述手部的手背与手掌之间厚度的参数值；
轮廓选取单元24，用于从所述外轮廓中选取轮廓的质心距离所述关节点的平均距离最近，且轮廓曲线总长度最长的外轮廓作为所述手部的外轮廓；
第二轮廓提取单元25，用于从所述彩色图像中提取像素值在手部像素区间的区域的外轮廓。
作为本发明提供的一种优选实施例，所述跟踪指部轮廓的装置还包括用于提取所述手部的外轮廓的指蹼的位置的指蹼提取模块50，具体参见图4，图4是本发明提供的跟踪指部轮廓的装置的指蹼提取模块的一个实施例的结构示意图；所述指蹼提取模块50包括：
轮廓拐点选取单元51，用于从所述手部的外轮廓的相邻指尖之间的轮廓点中，选取距离连接所述相邻指尖的直线最远的轮廓点作为轮廓拐点；
局部区域确定单元52，用于以所述轮廓拐点作为指蹼的当前位置，从所述彩色图像中提取以所述当前位置为中心点的局部区域；
候选区域确定单元53，用于将所述当前位置进行偏移获得多个偏移位置，并对于每一个偏移位置，从所述彩色图像中提取以该偏移位置为中心点且与所述局部区域相同形状的区域作为候选区域；其中，所述当前位置为(x，y)，所述偏移位置为(x+δx，y+δy)；δx∈{-1，0，1}，δy∈{-1，0，1}，且δx和δy不同时为0；
偏差程度计算单元54，用于计算每一个所述候选区域与所述局部区域的结构偏差程度；对于每一个候选区域，所述候选区域与所述局部区域的结构偏差程度为d(P，Q)，
Figure PCTCN2016113498-appb-000023
其中，P为包含所述局部区域的每一个像素点的像素值的集合，Q为包含所述候选区域的每一个像素点的像素值的集合，μP为集合P中所有像素值的均值，μQ为集合Q中所有像素值的均值，σPQ为集合P和集合Q的协方差，σP为集合P的方差，σQ为集合Q的方，c1和c2为预设常数；
指蹼位置确定单元55，用于当每一个所述候选区域与所述局部区域的结构偏差程度均大于预设阈值时，将所述当前位置作为所述指蹼的位置；
当前位置更新单元56，用于当存在一个所述候选区域与所述局部区域的结构偏差程度不大于所述预设阈值时，则选取与所述局部区域的结构偏差程度最小的候选区域所对应的偏移位置来更新所述当前位置，并更新所述局部区域和所述候选区域。
本领域普通技术人员可以理解实现上述实施例方法中的全部或部分流程，是可以通过计算机程序来指令相关的硬件来完成，所述的程序可存储于一计算机可读取存储介质中，该程序在执行时，可包括如上述各方法的实施例的流程。其中，所述的存储介质可为磁碟、光盘、只读存储记忆体(Read-Only Memory，ROM)或随机存储记忆体(RandomAccess Memory，RAM)等。
以上所述是本发明的优选实施方式，应当指出，对于本技术领域的普通技术人员来说，在不脱离本发明原理的前提下，还可以做出若干改进和润饰，这些改进和润饰也视为本发明的保护范围。

Claims (10)

  1. 一种跟踪指部轮廓的方法，其特征在于，包括：
    获取记录用户手部的同一影像的深度图像和彩色图像；
    从所述深度图像或彩色图像中提取所述手部的外轮廓；
    以所述手部的外轮廓的指蹼作为分割指部的分割点，分割所述手部的外轮廓，获得每个指部的初始外轮廓；
    对于每个指部的外轮廓，结合所述指部的初始外轮廓来约束所述指部的外轮廓贴合所述彩色图像中的对应所述指部的边缘，获得所述指部的跟踪外轮廓。
  2. 如权利要求1所述的跟踪指部轮廓的方法，其特征在于，所述指部的初始外轮廓为V0(s)，所述指部的跟踪外轮廓为
    Figure PCTCN2016113498-appb-100001
    其中，
    Figure PCTCN2016113498-appb-100002
    V(s)为描述所述指部的外轮廓的函数，Eimg(V(s))为约束所述指部的外轮廓贴合所述彩色图像中的对应所述指部的边缘的目标函数，I为描述所述彩色图像中的每一个像素点的像数值的离散函数。
  3. 如权利要求2所述的跟踪指部轮廓的方法，其特征在于，在对于每个指部的外轮廓，结合所述指部的初始外轮廓来约束所述指部的外轮廓贴合所述彩色图像中的手指边缘的同时，还包括：
    结合所述指部的初始外轮廓来约束所述指部的外轮廓的总长度和变化曲率，以及约束所述指部的外轮廓趋向于所述用户提供的手部轮廓曲线；
    则获得的所述指部的跟踪外轮廓修订为：
    Figure PCTCN2016113498-appb-100003
    其中，
    Figure PCTCN2016113498-appb-100004
    Econ(V(s))＝||V(s)-V0(s)||2，Eint(V(s))为约束所述指部的外轮廓的总长度和变化曲率的目标函数，Econ(V(s))为约束所述指部的外轮廓趋向于所述用户提供的手疗轮廓曲线的目标函数。
  4. 如权利要求1所述的跟踪指部轮廓的方法，其特征在于，从所述深度图像中提取所述手部的外轮廓，具体为：
    根据预设的手部关节点模型，从所述深度图像中计算出所述手部的每一个关节点的深度；
    取所述有关节点的深度的中值作为参考深度dref；
    从所述深度图像中提取深度在手部深度范围[dref-δ，dref+δ]内的区域的外轮廓；其中，δ为衡量所述手部的手背与手掌之间厚度的参数值；
    从所述外轮廓中选取轮廓的质心距离所述关节点的平均距离最近，且轮廓曲线总长度最长的外轮廓作为所述手部的外轮廓；
    以及，从所述彩色图像中提取所述手部的外轮廓，具体为：
    从所述彩色图像中提取像素值在手部像素区间的区域的外轮廓；
    从所述外轮廓中选取轮廓的质心距离所述关节点的平均距离最近，且轮廓曲线总长度最长的外轮廓作为所述手部的外轮廓。
  5. 如权利要求1所述的跟踪指部轮廓的方法，其特征在于，所述跟踪指部轮廓的方法还包括提取所述手部的外轮廓的指蹼的位置，具体为：
    从所述手部的外轮廓的相邻指尖之间的轮廓点中，选取距离连接所述相邻指尖的直线最远的轮廓点作为轮廓拐点；
    以所述轮廓拐点作为指蹼的当前位置，从所述彩色图像中提取以所述当前位置为中心点的局部区域；
    将所述当前位置进行偏移获得多个偏移位置，并对于每一个偏移位置，从所述彩色图像中提取以该偏移位置为中心点且与所述局部区域相同形状的区域作为候选区域；其中，所述当前位置为(x，y)，所述偏移位置为(x+δx，y+δy)；δx∈{-1，0，1}，δy∈{-1，0，1}，且δx和δy不同时为0；
    计算每一个所述候选区域与所述局部区域的结构偏差程度；对于每一个候选区域，所述候选区域与所述局部区域的结构偏差程度为d(P，Q)，
    Figure PCTCN2016113498-appb-100005
    其中，P为包含所述局部区域的每一个像素点的像素值的集合，Q为包含所述候选区域的每一个像素点的像素值的集合，μP为集合P中所有像素值的均值，μQ为集合Q中所有像素值的均值，σPQ为集合P和集合Q的协方差，σP为集合P的方差，σQ为集合Q的方，c1和c2为预设常数；
    若每一个所述候选区域与所述局部区域的结构偏差程度均大于预设阈值，则将所述当前位置作为所述指蹼的位置；
    若存在一个所述候选区域与所述局部区域的结构偏差程度不大于所述预设阈值，则选取 与所述局部区域的结构偏差程度最小的候选区域所对应的偏移位置来更新所述当前位置，并更新所述局部区域和所述候选区域。
  6. 一种跟踪指部轮廓的装置，其特征在于，包括：
    图像获取模块，用于获取记录用户手部的同一影像的深度图像和彩色图像；
    外轮廓提取模块，用于从所述深度图像或彩色图像中提取所述手部的外轮廓；
    指部轮廓提取模块，用于以所述手部的外轮廓的指蹼作为分割指部的分割点，分割所述手部的外轮廓，获得每个指部的初始外轮廓；
    指部轮廓约束模块，用于对于每个指部的外轮廓，结合所述指部的初始外轮廓来约束所述指部的外轮廓贴合所述彩色图像中的对应所述指部的边缘，获得所述指部的跟踪外轮廓。
  7. 如权利要求6所述的跟踪指部轮廓的装置，其特征在于，所述指部的初始外轮廓为V0(s)，所述指部的跟踪外轮廓为
    Figure PCTCN2016113498-appb-100006
    其中，
    Figure PCTCN2016113498-appb-100007
    V(s)为描述所述指部的外轮廓的函数，Eimg(V(s))为约束所述指部的外轮廓贴合所述彩色图像中的对应所述指部的边缘的目标函数，I为描述所述彩色图像中的每一个像素点的像数值的离散函数。
  8. 如权利要求7所述的跟踪指部轮廓的装置，其特征在于，所述指部轮廓约束模块还用于：结合所述指部的初始外轮廓来约束所述指部的外轮廓的总长度和变化曲率，以及约束所述指部的外轮廓趋向于所述用户提供的手部轮廓曲线；
    则获得的所述指部的跟踪外轮廓修订为：
    Figure PCTCN2016113498-appb-100008
    其中，
    Figure PCTCN2016113498-appb-100009
    Econ(V(s))＝||V(s)-V0(s)||2，Eint(V(s))为约束所述指部的外轮廓的总长度和变化曲率的目标函数，Econ(V(s))为约束所述指部的外轮廓趋向于所述用户提供的手疗轮廓曲线的目标函数。
  9. 如权利要求6所述的跟踪指部轮廓的装置，其特征在于，所述外轮廓提取模块包括：
    关节点深度计算单元，用于根据预设的手部关节点模型，从所述深度图像中计算出所述手部的每一个关节点的深度；
    参考深度确定单元，用于取所述有关节点的深度的中值作为参考深度dref；
    第一轮廓提取单元，用于从所述深度图像中提取深度在手部深度范围[dref-δ，dref+δ]内的区域的外轮廓；其中，δ为衡量所述手部的手背与手掌之间厚度的参数值；
    轮廓选取单元，用于从所述外轮廓中选取轮廓的质心距离所述关节点的平均距离最近，且轮廓曲线总长度最长的外轮廓作为所述手部的外轮廓；
    第二轮廓提取单元，用于从所述彩色图像中提取像素值在手部像素区间的区域的外轮廓。
  10. 如权利要求6所述的跟踪指部轮廓的装置，其特征在于，所述跟踪指部轮廓的装置还包括用于提取所述手部的外轮廓的指蹼的位置的指蹼提取模块，所述指蹼提取模块包括：
    轮廓拐点选取单元，用于从所述手部的外轮廓的相邻指尖之间的轮廓点中，选取距离连接所述相邻指尖的直线最远的轮廓点作为轮廓拐点；
    局部区域确定单元，用于以所述轮廓拐点作为指蹼的当前位置，从所述彩色图像中提取以所述当前位置为中心点的局部区域；
    候选区域确定单元，用于将所述当前位置进行偏移获得多个偏移位置，并对于每一个偏移位置，从所述彩色图像中提取以该偏移位置为中心点且与所述局部区域相同形状的区域作为候选区域；其中，所述当前位置为(x，y)，所述偏移位置为(x+δx，y+δy)；δx∈{-1，0，1}，δy∈{-1，0，1}，且δx和δy不同时为0；
    偏差程度计算单元，用于计算每一个所述候选区域与所述局部区域的结构偏差程度；对于每一个候选区域，所述候选区域与所述局部区域的结构偏差程度为d(P，Q)，
    Figure PCTCN2016113498-appb-100010
    其中，P为包含所述局部区域的每一个像素点的像素值的集合，Q为包含所述候选区域的每一个像素点的像素值的集合，μP为集合P中所有像素值的均值，μQ为集合Q中所有像素值的均值，σPQ为集合P和集合Q的协方差，σP为集合P的方差，σO为集合Q的方，c1和c2为预设常数；
    指蹼位置确定单元，用于当每一个所述候选区域与所述局部区域的结构偏差程度均大于预设阈值时，将所述当前位置作为所述指蹼的位置；
    当前位置更新单元，用于当存在一个所述候选区域与所述局部区域的结构偏差程度不大于所述预设阈值时，则选取与所述局部区域的结构偏差程度最小的候选区域所对应的偏移位置来更新所述当前位置，并更新所述局部区域和所述候选区域。
PCT/CN2016/113498 2016-08-16 2016-12-30 跟踪指部轮廓的方法及其装置 Ceased WO2018032701A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201610675185.5 2016-08-16
CN201610675185.5A CN106355591B (zh) 2016-08-16 2016-08-16 跟踪指部轮廓的方法及其装置

Publications (1)

Publication Number Publication Date
WO2018032701A1 true WO2018032701A1 (zh) 2018-02-22

Family

ID=57845062

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2016/113498 Ceased WO2018032701A1 (zh) 2016-08-16 2016-12-30 跟踪指部轮廓的方法及其装置

Country Status (2)

Country Link
CN (1) CN106355591B (zh)
WO (1) WO2018032701A1 (zh)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112819878A (zh) * 2021-01-28 2021-05-18 北京市商汤科技开发有限公司 一种深度检测方法、装置、计算机设备和存储介质
CN115423778A (zh) * 2022-09-05 2022-12-02 重庆大学 一种用于小儿骨龄评估的手部骨干特征图的生成方法
CN116309349A (zh) * 2023-02-15 2023-06-23 珞石（北京）科技有限公司 一种基于视觉的布料缝纫轨迹校准方法

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116309528B (zh) * 2023-04-11 2025-11-11 湖南大学 胎儿心脏超声图像处理方法、装置和计算机设备

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6256411B1 (en) * 1997-05-28 2001-07-03 Minolta Co., Ltd. Image processing device and method for detecting objects in image data
CN101470800A (zh) * 2007-12-30 2009-07-01 沈阳工业大学 手形识别方法

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101807114B (zh) * 2010-04-02 2011-12-07 浙江大学 一种基于三维手势的自然交互方法
CN103729875B (zh) * 2013-12-09 2016-09-07 深圳先进技术研究院 心脏磁共振图像的左心室三维轮廓重建方法和系统

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6256411B1 (en) * 1997-05-28 2001-07-03 Minolta Co., Ltd. Image processing device and method for detecting objects in image data
CN101470800A (zh) * 2007-12-30 2009-07-01 沈阳工业大学 手形识别方法

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
GONG, YULAN: "Hand-wrist Extraction of Edge based on Active Contour Model in Skeleton Automatic Assessment System", CHINA MASTERS' THESES FULL-TEXT DATABASE INFORMATION TECHNOLOGY, 15 December 2005 (2005-12-15) *
WANG, YANKAI: "Research of Dynamic Gesture Recognition And Application", CHINA MASTERS' THESES FULL-TEXT DATABASE INFORMATION TECHNOLOGY, 15 March 2016 (2016-03-15) *

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112819878A (zh) * 2021-01-28 2021-05-18 北京市商汤科技开发有限公司 一种深度检测方法、装置、计算机设备和存储介质
CN115423778A (zh) * 2022-09-05 2022-12-02 重庆大学 一种用于小儿骨龄评估的手部骨干特征图的生成方法
CN116309349A (zh) * 2023-02-15 2023-06-23 珞石（北京）科技有限公司 一种基于视觉的布料缝纫轨迹校准方法
CN116309349B (zh) * 2023-02-15 2025-05-06 珞石(山东)机器人集团股份有限公司 一种基于视觉的布料缝纫轨迹校准方法

Also Published As

Publication number Publication date
CN106355591A (zh) 2017-01-25
CN106355591B (zh) 2019-06-21

Similar Documents

Publication Publication Date Title
AU2019223705B2 (en) A method and device for the characterization of living specimens from a distance
US8086027B2 (en) Image processing apparatus and method
US9864430B2 (en) Gaze tracking via eye gaze model
US9747493B2 (en) Face pose rectification method and apparatus
US10048749B2 (en) Gaze detection offset for gaze tracking models
US9684928B2 (en) Foot tracking
US20160162673A1 (en) Technologies for learning body part geometry for use in biometric authentication
EP3241151B1 (en) An image face processing method and apparatus
CN113164098A (zh) 人类步态分析系统和方法
Gibelli et al. The identification of living persons on images: a literature review
US9727776B2 (en) Object orientation estimation
US9020251B2 (en) Image processing apparatus and method
CN108985210A (zh) 一种基于人眼几何特征的视线追踪方法及系统
WO2009091029A1 (ja) 顔姿勢推定装置、顔姿勢推定方法、及び、顔姿勢推定プログラム
US9922244B2 (en) Fast and robust identification of extremities of an object within a scene
WO2018032701A1 (zh) 跟踪指部轮廓的方法及其装置
KR20170092533A (ko) 얼굴 포즈 교정 방법 및 장치
JP2020513629A (ja) 被験者の頭部の追跡
WO2018032700A1 (zh) 跟踪指蹼位置的方法及其装置
JPWO2016067573A1 (ja) 姿勢推定方法および姿勢推定装置
WO2018032703A1 (zh) 跟踪指部轮廓的方法及其装置
CN108197549A (zh) 基于3d成像的人脸识别方法及终端
KR101726505B1 (ko) 설 촬영 장치 및 설 영상의 프로세싱 방법
Haines et al. Integrating Stereo with Shape-from-Shading derived Orientation Information.
WO2018032704A1 (zh) 跟踪指部轮廓的方法及其装置

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 16913453

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 25/04/2019)

122 Ep: pct application non-entry in european phase

Ref document number: 16913453

Country of ref document: EP

Kind code of ref document: A1