CN111144212B - Depth image target segmentation method and device - Google Patents
Depth image target segmentation method and device Download PDFInfo
- Publication number
- CN111144212B CN111144212B CN201911173140.8A CN201911173140A CN111144212B CN 111144212 B CN111144212 B CN 111144212B CN 201911173140 A CN201911173140 A CN 201911173140A CN 111144212 B CN111144212 B CN 111144212B
- Authority
- CN
- China
- Prior art keywords
- palm
- upper limb
- depth image
- area
- arm
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/107—Static hand or arm
- G06V40/113—Recognition of static hand signs
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/187—Segmentation; Edge detection involving region growing; involving region merging; involving connected component labelling
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/60—Analysis of geometric attributes
- G06T7/66—Analysis of geometric attributes of image moments or centre of gravity
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
- G06V10/267—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion by performing operations on regions, e.g. growing, shrinking or watersheds
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Multimedia (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Human Computer Interaction (AREA)
- Geometry (AREA)
- Image Analysis (AREA)
- Measurement Of The Respiration, Hearing Ability, Form, And Blood Characteristics Of Living Organisms (AREA)
Abstract
The application provides a depth image target segmentation method and device, wherein the method comprises the following steps: determining the centroid position of a palm area and the centroid position of an arm area in the upper limb depth image according to the palm position in the upper limb depth image; determining a palm bottom edge according to the palm region centroid position and the arm region centroid position; according to the bottom edge of the palm, determining the front end edge of the arm and determining the midpoint position of the front end edge of the arm; determining the boundary position between the palm area and the arm area according to the midpoint position of the bottom edge of the palm and the midpoint position of the front edge of the arm; and dividing a palm image area from the upper limb depth image based on the boundary position between the palm area and the arm area. According to the technical scheme, the palm image area can be segmented from the upper limb depth image, namely, accurate segmentation of the palm image area is realized.
Description
Technical Field
The present disclosure relates to the field of depth image processing technologies, and in particular, to a method and an apparatus for segmenting a depth image target.
Background
With the development of machine vision technology, a series of applications and markets involving 3D camera technology are vigorously developed, such as 3D gesture interaction technology, which relies on depth cameras to enable users to experience virtual three-dimensional object interactions, and the gesture interaction technology necessarily involves technologies such as hand detection and hand tracking based on depth images.
In the conventional depth image processing technique, palm coordinates can be accurately detected, but a hand region cannot be accurately segmented from a depth image yet.
Disclosure of Invention
Based on the above requirements, the application provides a depth image target segmentation method and device, which can segment a hand region from a depth image.
A depth image object segmentation method, comprising:
determining the centroid position of a palm area and the centroid position of an arm area in the upper limb depth image according to the palm position in the upper limb depth image; the upper limb depth image comprises an image obtained by performing depth imaging on the palm and the arm of the same upper limb;
determining a palm bottom edge according to the palm region centroid position and the arm region centroid position;
according to the bottom edge of the palm, determining the front end edge of the arm and determining the midpoint position of the front end edge of the arm;
Determining the boundary position between the palm area and the arm area according to the midpoint position of the bottom edge of the palm and the midpoint position of the front edge of the arm;
and dividing a palm image area from the upper limb depth image based on the boundary position between the palm area and the arm area.
Optionally, the determining the palm area centroid position and the arm area centroid position in the upper limb depth image according to the palm center position in the upper limb depth image includes:
and determining the palm area centroid position and the arm area centroid position by taking the palm center position in the upper limb depth image as a seed point and performing area growth processing in the upper limb depth image.
Optionally, the determining the palm area centroid position and the arm area centroid position by using the palm center position in the upper limb depth image as a seed point and performing area growth processing in the upper limb depth image includes:
the palm center position in the upper limb depth image is used as a seed point, and the palm area and the arm area in the upper limb depth image are determined through unconditional area growth processing in the upper limb depth image and area growth processing within a set distance range from the seed point; the set distance is determined according to a camera focal length of the upper limb depth image obtained through shooting and a depth value of a palm in the upper limb depth image;
And respectively calculating the barycenters of the palm area and the arm area, and determining the barycenter position of the palm area and the barycenter position of the arm area.
Optionally, the determining, by using the palm position in the upper limb depth image as a seed point, an unconditional region growing process in the upper limb depth image and a region growing process within a set distance range from the seed point, a palm region and an arm region in the upper limb depth image includes:
taking the palm center position in the upper limb depth image as a seed point, and performing unconditional region growth treatment in the upper limb depth image to obtain an upper limb mask image;
determining an effective upper limb area in the upper limb depth image according to the upper limb mask image and the upper limb depth image;
taking the palm center position in the upper limb depth image as a seed point, and performing region growth processing within a set distance range from the seed point in the upper limb depth image to obtain a palm region mask image;
and determining a palm region and an arm region in the upper limb depth image according to the palm region mask image and the effective upper limb region.
Optionally, the determining the palm bottom edge according to the palm area centroid position and the arm area centroid position includes:
Determining a wrist midpoint position according to the palm region centroid position and the arm region centroid position;
and determining the bottom edge of the palm according to the centroid position of the palm area and the midpoint position of the wrist.
Optionally, the determining the boundary position between the palm area and the arm area according to the midpoint position of the bottom edge of the palm and the midpoint position of the front edge of the arm includes:
determining a pixel row where the central axis of the arm is located according to the midpoint position of the bottom edge of the palm and the midpoint position of the front edge of the arm;
and determining the boundary position between the palm area and the arm area according to the midpoint position of the bottom edge of the palm and the pixel row where the central axis of the arm is located.
A depth image object segmentation apparatus, comprising:
the position determining unit is used for determining the palm area centroid position and the arm area centroid position in the upper limb depth image according to the palm position in the upper limb depth image; the upper limb depth image comprises an image obtained by performing depth imaging on the palm and the arm of the same upper limb;
the first computing unit is used for determining the bottom edge of the palm according to the centroid position of the palm area and the centroid position of the arm area;
The second computing unit is used for determining the front end edge of the arm according to the bottom edge of the palm and determining the midpoint position of the front end edge of the arm;
the third calculation unit is used for determining the boundary position between the palm area and the arm area according to the midpoint position of the bottom edge of the palm and the midpoint position of the front edge of the arm;
and the image segmentation unit is used for segmenting the palm image area from the upper limb depth image based on the boundary position between the palm area and the arm area.
Optionally, when the position determining unit determines the palm area centroid position and the arm area centroid position in the upper limb depth image according to the palm position in the upper limb depth image, the position determining unit is specifically configured to:
and determining the palm area centroid position and the arm area centroid position by taking the palm center position in the upper limb depth image as a seed point and performing area growth processing in the upper limb depth image.
Optionally, the position determining unit uses the palm position in the upper limb depth image as a seed point, and when determining the palm region centroid position and the arm region centroid position by performing region growing processing in the upper limb depth image, the position determining unit is specifically configured to:
The palm center position in the upper limb depth image is used as a seed point, and the palm area and the arm area in the upper limb depth image are determined through unconditional area growth processing in the upper limb depth image and area growth processing within a set distance range from the seed point; the set distance is determined according to a camera focal length of the upper limb depth image obtained through shooting and a depth value of a palm in the upper limb depth image;
and respectively calculating the barycenters of the palm area and the arm area, and determining the barycenter position of the palm area and the barycenter position of the arm area.
Optionally, the position determining unit uses the palm position in the upper limb depth image as a seed point, and performs unconditional region growing processing in the upper limb depth image and region growing processing within a set distance range from the seed point, so as to determine a palm region and an arm region in the upper limb depth image, where the position determining unit is specifically configured to:
taking the palm center position in the upper limb depth image as a seed point, and performing unconditional region growth treatment in the upper limb depth image to obtain an upper limb mask image;
determining an effective upper limb area in the upper limb depth image according to the upper limb mask image and the upper limb depth image;
Taking the palm center position in the upper limb depth image as a seed point, and performing region growth processing within a set distance range from the seed point in the upper limb depth image to obtain a palm region mask image;
and determining a palm region and an arm region in the upper limb depth image according to the palm region mask image and the effective upper limb region.
The depth image target segmentation method provided by the application can segment the palm area aiming at the upper limb depth image. Firstly, determining the centroid position of a palm area and the centroid position of an arm area in an upper limb depth image according to palm coordinates in the upper limb depth image; then, determining the bottom edge of the palm and the midpoint position of the front edge of the arm according to the centroid position of the palm area and the centroid position of the arm area; and secondly, determining the boundary position between the palm region and the arm region according to the midpoint position of the bottom edge of the palm and the midpoint position of the front edge of the arm, and further dividing the palm image region from the upper limb depth image according to the boundary position. The processing process is based on palm coordinates in the upper limb depth image, so that the boundary position between the palm region and the arm region in the upper limb depth image is determined, and further, the palm image region is segmented from the upper limb depth image, namely, accurate segmentation of the palm image region is realized.
Drawings
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings that are required to be used in the embodiments or the description of the prior art will be briefly described below, and it is obvious that the drawings in the following description are only embodiments of the present application, and that other drawings may be obtained according to the provided drawings without inventive effort to a person skilled in the art.
Fig. 1 is a schematic flow chart of a depth image object segmentation method according to an embodiment of the present application;
fig. 2 is a schematic diagram of an upper limb depth image provided in an embodiment of the present application;
fig. 3 is a flowchart of another depth image object segmentation method according to an embodiment of the present application;
FIG. 4 is a flowchart of another depth image object segmentation method according to an embodiment of the present disclosure;
fig. 5 is a schematic structural diagram of another depth image object segmentation apparatus according to an embodiment of the present application.
Detailed Description
The technical scheme of the embodiment of the application is suitable for an application scene of dividing the palm image area in the upper limb depth image. By adopting the technical scheme of the embodiment of the application, the palm image area can be segmented from the upper limb depth image.
The technical solution of the embodiment of the present application may be applied to a hardware device such as a hardware processor, or packaged as a software program to be executed, and when the hardware processor executes the processing procedure of the technical solution of the embodiment of the present application, or the software program is executed, the palm image area and the arm image area may be distinguished from the upper limb depth image, so that the palm image area may be distinguished from the upper limb depth image. The embodiment of the application only describes a specific processing procedure of the technical scheme by way of example, and does not limit a specific implementation form of the technical scheme, and any technical implementation form capable of executing the processing procedure of the technical scheme can be adopted by the embodiment of the application.
The following description of the embodiments of the present application will be made clearly and fully with reference to the accompanying drawings, in which it is evident that the embodiments described are only some, but not all, of the embodiments of the present application. All other embodiments, which can be made by one of ordinary skill in the art without undue burden from the present disclosure, are within the scope of the present disclosure.
The embodiment of the application provides a depth image object segmentation method, which is shown in fig. 1, and includes:
s101, determining the centroid position of a palm area and the centroid position of an arm area in an upper limb depth image according to the palm position in the upper limb depth image; the upper limb depth image comprises an image obtained by performing depth imaging on the palm and the arm of the same upper limb;
specifically, the upper limb depth image refers to a depth image obtained by performing depth imaging on an upper limb of a human body, and particularly refers to a depth image obtained by performing depth imaging on a forearm portion of the human body. The depth image includes a human arm and a palm. The upper limb depth image may be as shown in fig. 2.
The embodiment of the application predetermines the position coordinates of the palm center K in the upper limb depth image. For example, the position coordinates of the palm center K may be determined based on a preliminary hand detection result, by calculating a centroid for the hand detection result, or by calculating a demonst cluster center, or the like. Alternatively, palm coordinate data in the known upper limb depth image may be read.
Then, based on the palm coordinates, a palm area and an arm area in the upper limb depth image are determined, and then the centroid position of the palm area and the centroid position of the arm area are determined.
The palm area and the arm area determined in this step are not precisely determined, but are roughly determined, and in particular, the boundary between the two partial areas cannot be precisely determined. The embodiment of the application further determines the limit between the two through subsequent processing.
For example, in the embodiment of the present application, the palm area and the arm area in the upper limb depth image are determined by performing the area growth process in the upper limb depth image with the position of the palm center K as the seed point. Further, the centroids of the palm area and the arm area are calculated, and the centroid positions of the palm area and the arm area are respectively determined.
S102, determining the bottom edge of the palm according to the centroid position of the palm area and the centroid position of the arm area;
specifically, the bottom edge of the palm refers to the boundary between the palm and the arm.
The embodiment of the application determines the junction position of the palm and the arm, namely the position of the midpoint W of the wrist by calculating the midpoint position of the connecting line between the centroid position of the palm area and the centroid position of the arm area.
Then, a pixel row where the palm central axis Lcw is located is determined at the junction W of the palm and the arm, and a pixel row LTcw perpendicular to the pixel row where the palm central axis Lcw is located is determined, where the pixel row LTcw is located in the pixel set Sw within the palm region, that is, the bottom edge of the palm.
S103, determining the front end edge of the arm according to the bottom edge of the palm, and determining the midpoint position of the front end edge of the arm;
specifically, after determining the bottom edge Sw of the palm, the embodiment of the present application determines, from the above-mentioned upper limb depth image, a pixel row PLTcw parallel to the bottom edge Sw of the palm and having a distance b from the pixel row LTcw where the bottom edge Sw of the palm is located.
The pixel row PLTcw is the pixel row where the front edge of the arm is located. The distance b may be flexibly set according to practical situations or may be empirically set.
The pixel set Spw in the arm region is determined in the pixel row PLTcw, that is, the front edge of the arm, and then the midpoint position of the pixel set Spw is determined, that is, the position of the midpoint Mpw of the front edge of the arm is determined.
Illustratively, after determining the palm bottom edge Sw and the pixel row LTcw in which the palm bottom edge is located, the parallel pixel row PLTcw of the pixel row LTcw is calculated:
A(PLTcw)x+B(PLTcw)y+C(PLTcw)＝0
wherein A, B, C are the coefficients of the linear equations in which the pixel rows are located, respectively.
The pixel rows PLTcw and LTcw satisfy the following distance requirements:
|C(PLTcw)-C(LTcw)|/(A(LTcw)*A(LTcw)+B(LTcw)*B(LTcw))1/2＝b
and, pixel row PLTcw and palm area centroid are located on both sides of pixel row LTcw respectively; the value of b may be flexibly set according to actual situations or may be empirically set.
S104, determining the boundary position between the palm area and the arm area according to the midpoint position of the bottom edge of the palm and the midpoint position of the front edge of the arm;
specifically, the boundary position between the palm region and the arm region refers to a pixel row where the boundary between the palm region and the arm region is located.
Illustratively, in the embodiment of the present application, the pixel row where the line is located is determined according to the midpoint position Mw (xmw, ymw) of the bottom edge of the palm and the midpoint position Mpw (xmpw, ympw) of the front edge of the arm, and then the pixel row perpendicular to the pixel row is determined as the boundary position between the palm area and the arm area.
S105, dividing a palm image area from the upper limb depth image based on the boundary position between the palm area and the arm area.
Specifically, on the basis of determining the boundary position between the palm region and the arm region in the upper limb depth image described above and determining the palm region and the arm region in step S101, the palm region and the arm region may be accurately segmented in the upper limb depth image, so that the palm image region may be segmented from the upper limb depth image.
For example, an effective upper limb image region on the same side as the palm coordinates in the boundary position between the palm region and the arm region in the upper limb depth image is determined as a palm image region, and the palm image region is obtained by dividing the palm image region.
As can be seen from the above description, the method for segmenting a depth image target according to the embodiment of the present application may segment a palm region for an upper limb depth image. Firstly, determining the centroid position of a palm area and the centroid position of an arm area in an upper limb depth image according to palm coordinates in the upper limb depth image; then, determining the bottom edge of the palm and the midpoint position of the front edge of the arm according to the centroid position of the palm area and the centroid position of the arm area; and secondly, determining the boundary position between the palm region and the arm region according to the midpoint position of the bottom edge of the palm and the midpoint position of the front edge of the arm, and further dividing the palm image region from the upper limb depth image according to the boundary position. The processing process is based on palm coordinates in the upper limb depth image, so that the boundary position between the palm region and the arm region in the upper limb depth image is determined, and further, the palm image region is segmented from the upper limb depth image, namely, accurate segmentation of the palm image region is realized.
As an exemplary implementation manner, the embodiment of the application further discloses that determining the palm area centroid position and the arm area centroid position in the upper limb depth image according to the palm center position in the upper limb depth image includes:
and determining the centroid position of the palm region and the centroid position of the arm region by taking the palm center position in the upper limb depth image as a seed point and performing region growth processing in the upper limb depth image.
Referring to fig. 3, the processing procedure specifically includes:
s301, performing unconditional region growth treatment on the upper limb depth image by taking the palm position in the upper limb depth image as a seed point to obtain an upper limb mask image;
specifically, the palm center position (x 0, y 0) is taken as a seed point, whether the absolute value of the pixel value difference between the adjacent pixel value and the current reference seed point pixel value is within a set range or not is accessed according to the eight neighborhood or the four neighborhood (up, down, left and right and the like), if the absolute value is met, the point coordinate is added into a seed point queue, the point coordinate is marked as valid, otherwise, the skip is performed; the accessed coordinates are marked and then no longer accessed.
And then, sequentially reading the coordinates of the seed point queues until the dequeuing is completed, calculating whether the pixel value of the neighborhood coordinates meets the pixel value difference condition when the coordinates of each seed point queue are read, and finally returning all the coordinates meeting the condition to generate a mask image meeting the growth condition.
S302, determining an effective upper limb area in the upper limb depth image according to the upper limb mask image and the upper limb depth image;
specifically, the mask image and the upper limb depth image generated in step S301 are subjected to bitwise and (& gt) operation, so as to obtain an effective upper limb area in the upper limb depth image.
The above operation can keep the position marked valid in the mask image with the depth value of the point of the corresponding upper limb depth image, and the other invalid positions with the depth value of 0. After the calculation, the pixel position with the reserved depth value in the upper limb depth image is an effective upper limb pixel position, the pixel position with the depth value of 0 is an ineffective upper limb pixel position, and all the effective pixel positions form an effective upper limb area.
S303, taking the palm position in the upper limb depth image as a seed point, and performing region growing treatment within a set distance range from the seed point in the upper limb depth image to obtain a palm region mask image;
specifically, the palm position coordinates (x 0, y 0) are used as seed points (the points are also original seed points), the distance between the plane and the palm coordinates (x 0, y 0) is set as lambda, and the region growth is performed, namely, the region growth is performed within the range of the distance lambda from the seeds.
When the region of each step grows, judging whether the pixel value of the current seed point eight neighborhood or four neighborhood coordinates (x ', y ') and the pixel value error of the current seed point coordinates (x, y) meet the set condition, and the coordinates (x 0, y 0) from the original seed point (palm center position) meet (x ' -x 0)/(x ' -x 0) + (y ' -y 0) <=λ, if the two conditions are met, marking as valid, adding the valid into a seed point queue, and waiting to be sequentially read as a reference seed point; otherwise skip; the accessed point is marked as read and is not accessed.
After the region growth is stopped, coordinates in the seed point queue are sequentially read until the dequeuing is completed, when one coordinate is read, whether the pixel value of the neighborhood coordinate meets the pixel value difference condition is calculated, and finally, all coordinates meeting the condition are returned to generate a mask image meeting the growth condition, namely, a palm region mask image is obtained.
The lambda value can be determined with reference to lambda=250 (fx+fy)/2/d.
Where fx, fy is the focal length of the camera reference horizontal and vertical, and d is the depth value of the palm center (mm).
S304, determining a palm region and an arm region in the upper limb depth image according to the palm region mask image and the effective upper limb region;
Specifically, performing bit-wise and (& gt) operation on the palm region mask image and the effective upper limb region, and classifying coordinate points marked as effective positions by the mask as palm regions into a hand point set, namely the palm regions; other invalid regions are considered arm regions, classified as arm point sets, i.e., arm regions. Note that here the depth values of all points in the hand and arm point set are within the set range, excluding 0 values and values greater than the furthest point.
It will be understood that the above-described processing in steps S301 to S304 uses the palm position in the upper limb depth image as a seed point, and determines the palm region and the arm region in the upper limb depth image by performing unconditional region growing processing in the upper limb depth image and region growing processing within a set distance range from the seed point.
S305, calculating the mass centers of the palm area and the arm area respectively, and determining the mass center position of the palm area and the mass center position of the arm area.
Specifically, calculating the centroid of the set in the palm area, namely the hand point set, so as to obtain the centroid position of the palm area; and calculating the mass center of the set in the arm area, namely the arm point set, and obtaining the mass center position of the arm area.
Steps S306 to S309 in the present embodiment correspond to steps S102 to S105 in the method embodiment shown in fig. 1, respectively, and the specific content thereof is shown in the method embodiment shown in fig. 1, and will not be described herein.
Fig. 4 shows a process of another embodiment of the present application, in which the process of the depth image object segmentation method proposed in the present application is illustrated in detail from another point of view.
Referring to fig. 4, the depth image object segmentation method provided in the present application specifically includes:
s401, determining the centroid position of a palm area and the centroid position of an arm area in an upper limb depth image according to the palm position in the upper limb depth image;
in particular, the specific process of this step can be seen in the process shown in fig. 3, which is not repeated here.
The palm area centroid position and the arm area centroid position are shown as point C and point a in fig. 2, respectively.
S402, determining the midpoint position of the wrist according to the centroid position of the palm area and the centroid position of the arm area;
specifically, referring to fig. 2, assuming that the coordinates of the centroid position C of the palm area are (xc, yc) and the coordinates of the centroid position a of the arm area are (xa, ya), the midpoint position between the C point and the a point is the wrist midpoint position in the first-person view, and if the position is W (xw, yw), xw= (xc+xa)/2, yw= (yc+ya)/2.
According to the above positional relationship, the wrist midpoint position can be calculated and determined.
S403, determining the bottom edge of the palm according to the centroid position of the palm area and the midpoint position of the wrist;
specifically, as shown in fig. 2, first, a linear equation Lcw of the pixel row passing through the palm area centroid position C point and the wrist midpoint position W point is calculated: a (Lcw) x+b (Lcw) y+c (Lcw) =0; wherein A, B, C are equation coefficients.
Then, a linear equation LTcw for a pixel row perpendicular to Lcw passing through the point W is calculated: a (LTcw) x+b (LTcw) y+c (LTcw) =0, the pixel row LTcw can be regarded as the pixel row where the palm bottom edge is located.
Finally, the pixel set Sw of the effective upper limb area determined in step S401 in the pixel row LTcw is determined as the bottom edge of the palm.
S404, determining the front end edge of the arm according to the bottom edge of the palm, and determining the midpoint position of the front end edge of the arm;
specifically, a pixel line equation PLTcw parallel to LTcw is calculated according to the pixel line LTcw where the palm bottom edge Sw is located: a (PLTcw) x+b (PLTcw) y+c (PLTcw) =0, such that |c (PLTcw) -C (LTcw) |/(a (LTcw) ×a (LTcw) +b (LTcw) ×b (LTcw)) 1/2=b, i.e., such that the distance of the two linear equations is B, and the pixel row PLTcw and the palm area centroid position C point are located on both sides of the straight line LTcw, respectively. The pixel row where the above straight line equation PLTcw is located is the pixel row where the front edge of the arm is located.
Then, the pixel set Spw of the effective upper limb area determined in step S401 in the pixel row PLTcw is determined as the front edge of the arm.
Finally, the midpoint coordinate Mpw (xmpw, ympw) of the pixel set Spw is calculated, so as to obtain the midpoint position of the front edge of the arm.
S405, determining a pixel row where the central axis of the arm is located according to the midpoint position of the bottom edge of the palm and the midpoint position of the front edge of the arm;
specifically, a straight line equation LPcw for a pixel row passing through the midpoint position Mw (xmw, ymw) of the bottom edge of the palm and the midpoint position Mpw (xmpw) of the front edge of the arm is calculated, where a (LPcw) x+b (LPcw) y+c (LPcw) =0, i.e. the pixel row where the central axis of the arm is located.
S406, determining the boundary position between the palm area and the arm area according to the midpoint position of the bottom edge of the palm and the pixel row where the central axis of the arm is located;
specifically, a straight line equation LPTcw of a pixel row perpendicular to the pixel row LPcw where the central axis of the arm is located, i.e., a (LPTcw) x+b (LPTcw) +c (LPTcw) =0, is calculated to pass through the midpoint position Mw (xmw, ymw) of the bottom edge of the palm, and the position where the pixel row LPTcw is located can be used as the boundary position between the palm area and the arm area.
And S407, dividing a palm image area from the upper limb depth image based on the boundary position between the palm area and the arm area.
Specifically, after determining the boundary position LPTcw between the palm region and the arm region, the image region on the same side as the centroid position C of the palm region in the effective upper limb region determined in step S401 is the palm image region; the remaining image area in the effective upper limb area is the arm image area.
The specific processing contents of steps S401, S404, and S407 in this embodiment may also refer to steps S301 to S305 in the method embodiment shown in fig. 3, and steps S102 and S104 in the method embodiment shown in fig. 1, which are not described herein.
In the depth image object segmentation method according to the embodiment of the present application, in order to conveniently express the image positions where some pixel rows are located, the image positions are expressed in the form of a linear equation, and in the embodiment of the present application, each linear equation has its coordinates, length, and the like calculated based on the coordinates of the pixels of the image. It can be understood that each of the straight line equations in the above embodiments of the present application is a representation of a corresponding pixel row in the upper limb depth image.
Corresponding to the above-mentioned depth image object segmentation method, the embodiment of the present application further provides a depth image object segmentation apparatus, as shown in fig. 5, including:
a position determining unit 100, configured to determine a palm area centroid position and an arm area centroid position in an upper limb depth image according to a palm position in the upper limb depth image; the upper limb depth image comprises an image obtained by performing depth imaging on the palm and the arm of the same upper limb;
a first calculating unit 110, configured to determine a palm bottom edge according to the palm area centroid position and the arm area centroid position;
a second calculating unit 120, configured to determine an arm front edge according to the palm bottom edge, and determine a midpoint position of the arm front edge;
a third calculation unit 130, configured to determine a boundary position between the palm area and the arm area according to the midpoint position of the bottom edge of the palm and the midpoint position of the front edge of the arm;
an image segmentation unit 140, configured to segment a palm image region from the upper limb depth image based on a boundary position between the palm region and the arm region.
As an exemplary implementation manner, when the position determining unit 100 determines the palm area centroid position and the arm area centroid position in the upper limb depth image according to the palm position in the upper limb depth image, the position determining unit is specifically configured to:
and determining the palm area centroid position and the arm area centroid position by taking the palm center position in the upper limb depth image as a seed point and performing area growth processing in the upper limb depth image.
As an exemplary implementation manner, when the position determining unit 100 uses the palm position in the upper limb depth image as a seed point and performs the region growing process in the upper limb depth image, the position determining unit is specifically configured to:
the palm center position in the upper limb depth image is used as a seed point, and the palm area and the arm area in the upper limb depth image are determined through unconditional area growth processing in the upper limb depth image and area growth processing within a set distance range from the seed point; the set distance is determined according to a camera focal length of the upper limb depth image obtained through shooting and a depth value of a palm in the upper limb depth image;
And respectively calculating the barycenters of the palm area and the arm area, and determining the barycenter position of the palm area and the barycenter position of the arm area.
As an exemplary implementation manner, when the position determining unit 100 uses the palm position in the upper limb depth image as a seed point, and performs an unconditional region growing process in the upper limb depth image and a region growing process within a set distance range from the seed point, the position determining unit is specifically configured to:
taking the palm center position in the upper limb depth image as a seed point, and performing unconditional region growth treatment in the upper limb depth image to obtain an upper limb mask image;
determining an effective upper limb area in the upper limb depth image according to the upper limb mask image and the upper limb depth image;
taking the palm center position in the upper limb depth image as a seed point, and performing region growth processing within a set distance range from the seed point in the upper limb depth image to obtain a palm region mask image;
and determining a palm region and an arm region in the upper limb depth image according to the palm region mask image and the effective upper limb region.
As an exemplary implementation, the first computing unit 110 is specifically configured to, when determining the palm bottom edge according to the palm area centroid position and the arm area centroid position:
determining a wrist midpoint position according to the palm region centroid position and the arm region centroid position;
and determining the bottom edge of the palm according to the centroid position of the palm area and the midpoint position of the wrist.
As an exemplary implementation manner, the third computing unit 130 is specifically configured to, when determining the boundary position between the palm area and the arm area according to the midpoint position of the bottom edge of the palm and the midpoint position of the front edge of the arm:
determining a pixel row where the central axis of the arm is located according to the midpoint position of the bottom edge of the palm and the midpoint position of the front edge of the arm;
and determining the boundary position between the palm area and the arm area according to the midpoint position of the bottom edge of the palm and the pixel row where the central axis of the arm is located.
In detail, for specific working contents of each unit of the depth image target segmentation apparatus, please refer to the contents of the above method embodiment, and details are not repeated herein.
For the foregoing method embodiments, for simplicity of explanation, the methodologies are shown as a series of acts, but one of ordinary skill in the art will appreciate that the present application is not limited by the order of acts described, as some acts may, in accordance with the present application, occur in other orders or concurrently. Further, those skilled in the art will also appreciate that the embodiments described in the specification are all preferred embodiments, and that the acts and modules referred to are not necessarily required in the present application.
It should be noted that, in the present specification, each embodiment is described in a progressive manner, and each embodiment is mainly described as different from other embodiments, and identical and similar parts between the embodiments are all enough to be referred to each other. For the apparatus class embodiments, the description is relatively simple as it is substantially similar to the method embodiments, and reference is made to the description of the method embodiments for relevant points.
The steps in the methods of the embodiments of the present application may be sequentially adjusted, combined, and pruned according to actual needs.
The modules and sub-modules in the device and the terminal of the embodiments of the present application may be combined, divided, and deleted according to actual needs.
In the embodiments provided in the present application, it should be understood that the disclosed terminal, apparatus and method may be implemented in other manners. For example, the above-described terminal embodiments are merely illustrative, and for example, the division of modules or sub-modules is merely a logical function division, and there may be other manners of division in actual implementation, for example, multiple sub-modules or modules may be combined or integrated into another module, or some features may be omitted, or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed with each other may be an indirect coupling or communication connection via some interfaces, devices or modules, which may be in electrical, mechanical, or other forms.
The modules or sub-modules illustrated as separate components may or may not be physically separate, and components that are modules or sub-modules may or may not be physical modules or sub-modules, i.e., may be located in one place, or may be distributed over multiple network modules or sub-modules. Some or all of the modules or sub-modules may be selected according to actual needs to achieve the purpose of the embodiment.
In addition, each functional module or sub-module in each embodiment of the present application may be integrated in one processing module, or each module or sub-module may exist alone physically, or two or more modules or sub-modules may be integrated in one module. The integrated modules or sub-modules may be implemented in hardware or in software functional modules or sub-modules.
Those of skill would further appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or combinations of both, and that the various illustrative elements and steps are described above generally in terms of functionality in order to clearly illustrate the interchangeability of hardware and software. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.
The steps of a method or algorithm described in connection with the embodiments disclosed herein may be embodied directly in hardware, in a software unit executed by a processor, or in a combination of the two. The software elements may be disposed in Random Access Memory (RAM), memory, read Only Memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.
Finally, it is further noted that relational terms such as first and second, and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.
The previous description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the present application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the application. Thus, the present application is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims (10)
1. A depth image object segmentation method, comprising:
determining the centroid position of a palm area and the centroid position of an arm area in the upper limb depth image according to the palm position in the upper limb depth image; the upper limb depth image comprises an image obtained by performing depth imaging on the palm and the arm of the same upper limb;
determining a palm bottom edge according to the palm region centroid position and the arm region centroid position;
according to the bottom edge of the palm, determining the front end edge of the arm and determining the midpoint position of the front end edge of the arm;
determining the boundary position between the palm area and the arm area according to the midpoint position of the bottom edge of the palm and the midpoint position of the front edge of the arm;
and dividing a palm image area from the upper limb depth image based on the boundary position between the palm area and the arm area.
2. The method of claim 1, wherein determining palm region centroid positions and arm region centroid positions in the upper limb depth image from palm positions in the upper limb depth image comprises:
and determining the palm area centroid position and the arm area centroid position by taking the palm center position in the upper limb depth image as a seed point and performing area growth processing in the upper limb depth image.
3. The method according to claim 2, wherein determining the palm area centroid position and the arm area centroid position by performing an area growing process in the upper limb depth image with the palm center position in the upper limb depth image as a seed point comprises:
the palm center position in the upper limb depth image is used as a seed point, and the palm area and the arm area in the upper limb depth image are determined through unconditional area growth processing in the upper limb depth image and area growth processing within a set distance range from the seed point; the set distance is determined according to a camera focal length of the upper limb depth image obtained through shooting and a depth value of a palm in the upper limb depth image;
and respectively calculating the barycenters of the palm area and the arm area, and determining the barycenter position of the palm area and the barycenter position of the arm area.
4. A method according to claim 3, wherein determining palm and arm regions in the upper limb depth image by performing an unconditional region growing process in the upper limb depth image with the palm position in the upper limb depth image as a seed point, and a region growing process within a set distance range from the seed point, comprises:
Taking the palm center position in the upper limb depth image as a seed point, and performing unconditional region growth treatment in the upper limb depth image to obtain an upper limb mask image;
determining an effective upper limb area in the upper limb depth image according to the upper limb mask image and the upper limb depth image;
taking the palm center position in the upper limb depth image as a seed point, and performing region growth processing within a set distance range from the seed point in the upper limb depth image to obtain a palm region mask image;
and determining a palm region and an arm region in the upper limb depth image according to the palm region mask image and the effective upper limb region.
5. The method of claim 1, wherein the determining a palm bottom edge from the palm region centroid position and the arm region centroid position comprises:
determining a wrist midpoint position according to the palm region centroid position and the arm region centroid position;
and determining the bottom edge of the palm according to the centroid position of the palm area and the midpoint position of the wrist.
6. The method of claim 1, wherein determining the boundary position between the palm area and the arm area based on the midpoint position of the palm bottom edge and the midpoint position of the arm front edge comprises:
Determining a pixel row where the central axis of the arm is located according to the midpoint position of the bottom edge of the palm and the midpoint position of the front edge of the arm;
and determining the boundary position between the palm area and the arm area according to the midpoint position of the bottom edge of the palm and the pixel row where the central axis of the arm is located.
7. A depth image object segmentation apparatus, comprising:
the position determining unit is used for determining the palm area centroid position and the arm area centroid position in the upper limb depth image according to the palm position in the upper limb depth image; the upper limb depth image comprises an image obtained by performing depth imaging on the palm and the arm of the same upper limb;
the first computing unit is used for determining the bottom edge of the palm according to the centroid position of the palm area and the centroid position of the arm area;
the second computing unit is used for determining the front end edge of the arm according to the bottom edge of the palm and determining the midpoint position of the front end edge of the arm;
the third calculation unit is used for determining the boundary position between the palm area and the arm area according to the midpoint position of the bottom edge of the palm and the midpoint position of the front edge of the arm;
And the image segmentation unit is used for segmenting the palm image area from the upper limb depth image based on the boundary position between the palm area and the arm area.
8. The apparatus according to claim 7, wherein the position determining unit is configured to, when determining the palm area centroid position and the arm area centroid position in the upper limb depth image according to the palm position in the upper limb depth image:
and determining the palm area centroid position and the arm area centroid position by taking the palm center position in the upper limb depth image as a seed point and performing area growth processing in the upper limb depth image.
9. The apparatus according to claim 8, wherein the position determining unit is configured to, when determining the palm area centroid position and the arm area centroid position by performing the area growing process in the upper limb depth image with the palm center position in the upper limb depth image as a seed point:
the palm center position in the upper limb depth image is used as a seed point, and the palm area and the arm area in the upper limb depth image are determined through unconditional area growth processing in the upper limb depth image and area growth processing within a set distance range from the seed point; the set distance is determined according to a camera focal length of the upper limb depth image obtained through shooting and a depth value of a palm in the upper limb depth image;
And respectively calculating the barycenters of the palm area and the arm area, and determining the barycenter position of the palm area and the barycenter position of the arm area.
10. The apparatus according to claim 9, wherein the position determining unit is configured to determine, with a palm position in the upper limb depth image as a seed point, a palm region and an arm region in the upper limb depth image by performing an unconditional region growing process in the upper limb depth image and a region growing process within a set distance range from the seed point, specifically:
taking the palm center position in the upper limb depth image as a seed point, and performing unconditional region growth treatment in the upper limb depth image to obtain an upper limb mask image;
determining an effective upper limb area in the upper limb depth image according to the upper limb mask image and the upper limb depth image;
taking the palm center position in the upper limb depth image as a seed point, and performing region growth processing within a set distance range from the seed point in the upper limb depth image to obtain a palm region mask image;
and determining a palm region and an arm region in the upper limb depth image according to the palm region mask image and the effective upper limb region.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911173140.8A CN111144212B (en) | 2019-11-26 | 2019-11-26 | Depth image target segmentation method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911173140.8A CN111144212B (en) | 2019-11-26 | 2019-11-26 | Depth image target segmentation method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111144212A CN111144212A (en) | 2020-05-12 |
| CN111144212B true CN111144212B (en) | 2023-06-23 |
Family
ID=70516715
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911173140.8A Active CN111144212B (en) | 2019-11-26 | 2019-11-26 | Depth image target segmentation method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111144212B (en) |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103226387A (en) * | 2013-04-07 | 2013-07-31 | 华南理工大学 | Video fingertip positioning method based on Kinect |
| US9256780B1 (en) * | 2014-09-22 | 2016-02-09 | Intel Corporation | Facilitating dynamic computations for performing intelligent body segmentations for enhanced gesture recognition on computing devices |
| CN107341811A (en) * | 2017-06-20 | 2017-11-10 | 上海数迹智能科技有限公司 | The method that hand region segmentation is carried out using MeanShift algorithms based on depth image |
| CN108564063A (en) * | 2018-04-27 | 2018-09-21 | 北京华捷艾米科技有限公司 | Centre of the palm localization method based on depth information and system |
| CN109190516A (en) * | 2018-08-14 | 2019-01-11 | 东北大学 | A kind of static gesture identification method based on volar edge contour vectorization |
| CN109948461A (en) * | 2019-02-27 | 2019-06-28 | 浙江理工大学 | A kind of sign language image partition method based on center coordination and range conversion |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101657825B (en) * | 2006-05-11 | 2014-02-19 | 普莱姆传感有限公司 | Modeling of humanoid forms from depth maps |
| US8565476B2 (en) * | 2009-01-30 | 2013-10-22 | Microsoft Corporation | Visual target tracking |
| CN106250867B (en) * | 2016-08-12 | 2017-11-14 | 南京华捷艾米软件科技有限公司 | A kind of implementation method of the skeleton tracking system based on depth data |
-
2019
- 2019-11-26 CN CN201911173140.8A patent/CN111144212B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103226387A (en) * | 2013-04-07 | 2013-07-31 | 华南理工大学 | Video fingertip positioning method based on Kinect |
| US9256780B1 (en) * | 2014-09-22 | 2016-02-09 | Intel Corporation | Facilitating dynamic computations for performing intelligent body segmentations for enhanced gesture recognition on computing devices |
| CN107341811A (en) * | 2017-06-20 | 2017-11-10 | 上海数迹智能科技有限公司 | The method that hand region segmentation is carried out using MeanShift algorithms based on depth image |
| CN108564063A (en) * | 2018-04-27 | 2018-09-21 | 北京华捷艾米科技有限公司 | Centre of the palm localization method based on depth information and system |
| CN109190516A (en) * | 2018-08-14 | 2019-01-11 | 东北大学 | A kind of static gesture identification method based on volar edge contour vectorization |
| CN109948461A (en) * | 2019-02-27 | 2019-06-28 | 浙江理工大学 | A kind of sign language image partition method based on center coordination and range conversion |
Non-Patent Citations (2)
| Title |
|---|
| 崔家礼 ; 解威 ; 王一丁 ; 贾瑞明 ; .基于形状特征的静态手势数字识别.北方工业大学学报.2015,(03),第24、43-48页. * |
| 张黎.复杂背景下动态手势的识别研究.中国优秀硕士学位论文全文数据库（电子期刊）.2013,第I138-396页. * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111144212A (en) | 2020-05-12 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| KR101606628B1 (en) | Pointing-direction detecting device and its method, program and computer readable-medium | |
| WO2020119684A1 (en) | 3d navigation semantic map update method, apparatus and device | |
| CN109934065B (en) | Method and device for gesture recognition | |
| CN112506340B (en) | Equipment control method, device, electronic equipment and storage medium | |
| CN110363817B (en) | Target pose estimation method, electronic device, and medium | |
| JP6060632B2 (en) | Method and apparatus for determining projection area in image | |
| CN109961523B (en) | Method, device, system, equipment and storage medium for updating virtual target | |
| JP2013500536A5 (en) | ||
| WO2022156167A1 (en) | Image processing method and apparatus, and electronic device, computer-readable storage medium, computer program and program product | |
| JP2016099982A (en) | Behavior recognition device, behaviour learning device, method, and program | |
| JP2014202540A (en) | Position calculation system, position calculation program, and position calculation method | |
| CN108765317A (en) | A kind of combined optimization method that space-time consistency is stablized with eigencenter EMD adaptive videos | |
| CN113298870B (en) | Object posture tracking method and device, terminal equipment and storage medium | |
| CN111160088A (en) | VR (virtual reality) somatosensory data detection method and device, computer equipment and storage medium | |
| CN116188594B (en) | Calibration method, calibration system, calibration device and electronic equipment of camera | |
| CN110298314A (en) | The recognition methods of gesture area and device | |
| US9025022B2 (en) | Method and apparatus for gesture recognition using a two dimensional imaging device | |
| JP2010113530A (en) | Image recognition device and program | |
| CN111144212B (en) | Depth image target segmentation method and device | |
| CN108876704B (en) | Method and device for deforming human face image and computer storage medium | |
| CN113454684A (en) | Key point calibration method and device | |
| KR20220044156A (en) | Golf ball overhead detection method, system and storage medium | |
| EP3309660A1 (en) | Method for detecting input device, and detection device | |
| CN114564014A (en) | Object information determination method, mobile robot system, and electronic device | |
| JP2016162425A (en) | Body posture estimation device, method, and program |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |