CN112232332B - Non-contact palm detection method based on video sequence - Google Patents

Non-contact palm detection method based on video sequence Download PDF

Info

Publication number
CN112232332B
CN112232332B CN202011499311.9A CN202011499311A CN112232332B CN 112232332 B CN112232332 B CN 112232332B CN 202011499311 A CN202011499311 A CN 202011499311A CN 112232332 B CN112232332 B CN 112232332B
Authority
CN
China
Prior art keywords
image
palm
binary image
skin color
area
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202011499311.9A
Other languages
Chinese (zh)
Other versions
CN112232332A (en
Inventor
赵国栋
杨爽
李学双
张烜
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Jiangsu Shengdian Century Technology Co ltd
Original Assignee
Sichuan Shengdian Century Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Sichuan Shengdian Century Technology Co ltd filed Critical Sichuan Shengdian Century Technology Co ltd
Priority to CN202011499311.9A priority Critical patent/CN112232332B/en
Publication of CN112232332A publication Critical patent/CN112232332A/en
Application granted granted Critical
Publication of CN112232332B publication Critical patent/CN112232332B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/107—Static hand or arm
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/13—Edge detection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/168—Segmentation; Edge detection involving transform domain methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/187—Segmentation; Edge detection involving region growing; involving region merging; involving connected component labelling
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/28—Quantising the image, e.g. histogram thresholding for discrimination between background and foreground patterns
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/50—Extraction of image or video features by performing operations within image blocks; by using histograms, e.g. histogram of oriented gradients [HoG]; by summing image-intensity values; Projection analysis
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/56—Extraction of image or video features relating to colour
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10016—Video; Image sequence
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20048—Transform domain processing
    • G06T2207/20061—Hough transform
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/107—Static hand or arm
    • G06V40/117—Biometrics derived from hands

Abstract

The invention relates to a non-contact palm detection method based on a video sequence, which comprises the following steps: 1) acquiring a palm video image under non-contact imaging; 2) carrying out skin color detection on each frame of image to obtain a skin color area binary image; 3) detecting a moving target to obtain a moving area binary image; 4) counting the number of white pixel points of the binary image of the motion area at the time t, comparing the number with a threshold value, and continuing to perform the step 5) if the number is larger than the threshold value; 5) fusing the skin color area binary image and the motion area binary image; 6) removing the wrist area; 7) and extracting the HOG characteristic vector to finally judge the palm area. Aiming at the palm detection problem under non-contact imaging, the invention improves the elliptical skin color model to carry out skin color detection by adopting self-adaptive illumination compensation based on gamma conversion and a mode of carrying out segmentation judgment on ellipses with different sizes according to the brightness value, and the obtained region characteristics improve the accuracy of subsequent palm detection.

Description

Non-contact palm detection method based on video sequence
Technical Field
The invention belongs to the technical field of palm vein identification and information safety under non-contact imaging, and particularly relates to a non-contact palm detection method based on a video sequence.
Background
The non-contact palm vein recognition technology is a new biological characteristic recognition technology, and the palm vein recognition technology has the advantages of safety and difficulty in counterfeiting because the palm vein recognition technology acquires internal information of a human body. The contact palm vein is easy to cause the rejection psychology of a user in the aspect of sanitation due to the large contact area of the contact palm vein, the development of a palm vein identification technology is not facilitated, the non-contact palm vein identification technology is not limited by a fixing device, the user does not need to be in direct contact with equipment, hidden dangers in the aspect of sanitation can be eliminated, a novel biological feature identification experience is brought to the user, and the application and development prospect is wide.
Compared with a contact palm vein recognition technology, the non-contact palm vein recognition technology increases the steps of palm detection and palm interested region extraction, wherein the palm detection plays an important role as the first step of the whole recognition system.
Chinese patent CN104102347A discloses a fingertip positioning method, which can be used for extracting palm contours, and includes the following steps: acquiring a gesture image containing a gesture; extracting a palm outline from the gesture image; carrying out polygonal approximation on the palm outline to obtain an outline convex hull; and sequentially judging whether each salient point of the outline convex hull is a fingertip point. The fingertip positioning method obtains the outline convex hull by performing polygon approximation on the outline of the palm, and the fingertips in the gesture usually correspond to the positions of the salient points in the outline convex hull. The fingertip positioning method only needs to judge whether each salient point is a fingertip point, so that the number of the fingertip points needing to be judged is reduced, the calculated amount is reduced, the running speed is improved, and the real-time requirement is met; meanwhile, the curvature calculation of the edge points on the palm profile is not needed, so that certain errors are allowed in the extracted palm profile, and the requirement on the precision of the extracted palm profile is reduced.
Chinese patent CN109350018A discloses an image-based palm detection method for a palm herpes detection system, comprising the following steps: acquiring an RGB color image; converting the RGB color image into a YCrCb three-color image, obtaining a Cr image, and then carrying out binarization on the Cr image; skin color detection, obtaining skin color outline and convex outline characteristic detection; judging whether the convex profile characteristic number is larger than zero, and screening out a candidate palm area if the convex profile characteristic number is larger than zero; and judging whether the candidate palm region is a palm contour region. The patent performs skin color detection by extracting Cr components of YCrCb color space from the palm region to obtain a palm profile and performs palm detection according to its convex profile features.
The non-contact palm vein recognition technology has the advantages that the palm is placed at random due to the fact that the non-contact palm vein recognition technology is not guided by a fixed position in the palm collecting process, images are prone to deformation such as translation and rotation and are prone to being affected by the surrounding environment, and the palm area is detected from a complex background with certain difficulty.
Disclosure of Invention
The invention aims to solve the problem that a non-contact palm vein recognition technology is easily influenced by the surrounding environment in the image acquisition process, and provides a non-contact palm detection method based on a video sequence, which can effectively detect a palm area and prepare for the subsequent recognition process.
In order to achieve the purpose, the technical scheme provided by the invention is as follows:
the invention relates to a non-contact palm detection method based on a video sequence, which comprises the following steps:
1) acquiring a palm video image under non-contact imaging, wherein a first frame image is a background without a palm, and the palm is placed from a second frame image;
2) carrying out skin color detection on each frame of image, and obtaining a skin color area binary image I in a frame t1；
3) Detecting the moving object and obtaining a motion area binary image I in the t frame2；
4) Statistic t frame motion area binary image I2Comparing the number of the white pixel points with a threshold value, and continuing to perform the step 5) if the number is larger than the threshold value;
5) binary image I of skin color area in t frame1And exercise ofRegional binary image I2Performing fusion to obtain a fused binary image I3Performing morphological treatment on the obtained product, and reserving a maximum communication area;
6) for the fused binary image I3Removing the wrist area to obtain a palm area binary image I4；
7) Extracting a palm region binary image I4The final judgment of the palm region is performed by the HOG feature of (1).
Preferably, the step 2) of performing skin color detection on the current frame image according to the improved elliptical skin color model specifically includes:
2.1) carrying out adaptive illumination compensation based on gamma transformation on the current frame image, and converting from an RGB color space to a YCbCr color space:
Figure 940931DEST_PATH_IMAGE001
Figure 227556DEST_PATH_IMAGE002
wherein Y is a luminance component of a color in the YCbCr color space, Cb, Cr are density offset components of blue and red in the YCbCr color space, respectively, R, G, B are colors of three channels of red, green, and blue in the RGB color space, respectively, c is a constant,
Figure 289053DEST_PATH_IMAGE003
in order to be the gamma conversion coefficient(s),
Figure 635721DEST_PATH_IMAGE004
th0 is the pixel threshold value and is 0 ≦ th0 ≦ 255;
2.2) clustering the skin points according to the elliptical skin color model, wherein the elliptical skin color model is as follows:
Figure 48248DEST_PATH_IMAGE005
Figure 138563DEST_PATH_IMAGE006
wherein, x in the formula (3) is an abscissa in the ellipse, y is an ordinate in the ellipse, ecx and ecy are an abscissa and an ordinate of the center point of the ellipse, a is the major axis of the ellipse, and b is the minor axis of the ellipse; theta in the formula (4) is the rotation angle of the ellipse, Cb and Cr are the concentration offset components of blue and red colors in the YCbCr color space, respectively,
Figure 320146DEST_PATH_IMAGE007
is the horizontal coordinate of the center point of the skin color model in the CbCr coordinate system,
Figure 509819DEST_PATH_IMAGE008
the longitudinal coordinate of the skin color model at the central point of the CbCr coordinate system;
2.3) dividing elliptical models with different sizes according to the Y-channel brightness value, and judging whether the pixels are skin points according to whether the pixels are positioned in an ellipse with corresponding brightness:
Figure 973424DEST_PATH_IMAGE009
wherein, I1(i, j) is the skin color region binary image pixel value, Y (i, j) is the Y channel pixel value, i is the row number index of the image, j is the column number index of the image,
Figure 539534DEST_PATH_IMAGE010
the average brightness of the Y channel is shown, ecx and ecy are the abscissa and ordinate of the center point of the ellipse, a is the major axis of the ellipse, and b is the minor axis of the ellipse.
Preferably, the step 3) of detecting the moving target according to the ViBe algorithm includes the following specific steps:
3.1) initializing the background model with the first frame image to
Figure 903519DEST_PATH_IMAGE011
As a neighborhood point set around point x, a background model is constructed:
Figure 264094DEST_PATH_IMAGE012
wherein the content of the first and second substances,
Figure 713529DEST_PATH_IMAGE013
are the pixels in the initial background model,
Figure 83331DEST_PATH_IMAGE014
for the pixel values in the original image,
Figure 36243DEST_PATH_IMAGE011
is a set of neighborhood points around point x, y is
Figure 833298DEST_PATH_IMAGE011
Point (2).
3.2) updating the background model.
Preferably, the specific steps in step 4) include:
4.1) statistical motion region binary map I2The number of white pixels:
Figure 20564DEST_PATH_IMAGE015
wherein num is a motion region binary image I2M, N respectively represent the line height and the line width of the current frame image, i is the line index of the image, and j is the line index of the image;
4.2) according to I2The number of white areas (th 1) is a threshold value:
Figure 194057DEST_PATH_IMAGE016
wherein, I2Is a motion region binary image with num of I2Th1 is a quantity threshold value, th1 is an integer;
4.3) if the current frame image has a moving target, performing step 5), if no moving target exists, judging whether the t-1 frame image contains a rectangular frame mark, if so, continuing to mark the same position of the t frame image, and if not, returning to the step 1) to detect the t +1 frame image.
Preferably, the specific steps in the step 5) include:
5.1) mixing I1And I2Performing fusion with the operation:
Figure 204738DEST_PATH_IMAGE017
wherein, I3To fuse binary images, I1Is a binary map of the skin color region, I2The motion area binary image is shown as i, the row index of the image and j, the column index of the image;
5.2) pairs of I3Performing morphological opening operation and closing operation processing in sequence;
5.3) searching a connected region of the result binary image after morphological processing, and reserving the maximum connected region.
Preferably, the step 6) is performed according to Hough transform pair I3The method for removing the wrist area comprises the following specific steps:
6.1) Pair fusion binary image I3Performing edge detection to obtain I3Is (d) profile L3；
6.2) pairs of L3Carrying out Hough transform to obtain L3The upper straight line set L comprises a starting point and an end point of each line segment;
6.3) calculation of I3The center of gravity of (1):
Figure 235011DEST_PATH_IMAGE018
wherein the content of the first and second substances,
Figure 331143DEST_PATH_IMAGE019
is the abscissa of the point of gravity,
Figure 42747DEST_PATH_IMAGE020
is the ordinate of the center of gravity point, ((
Figure 970252DEST_PATH_IMAGE021
,
Figure 109109DEST_PATH_IMAGE022
) Is the coordinates of the pixel points of the image,
Figure 754854DEST_PATH_IMAGE023
is the pixel value of the point;
6.4) detecting whether a parallel straight line section below the gravity center exists in the straight line set L, and if so, detecting I3And setting the pixel points below the straight line starting point of the middle corresponding position as 0:
Figure 4570DEST_PATH_IMAGE024
wherein, I4For a palm region binary map, x0 represents the x coordinate of the start of a parallel straight line segment located below the center of gravity, i.e., the x coordinate
Figure 989843DEST_PATH_IMAGE025
I is the index of the number of rows of the image and j is the index of the number of columns of the image.
Preferably, the specific steps in step 7) include:
7.1) extracting a palm region binary image I4HOG feature vector H;
7.2) carrying out cosine similarity calculation on the H and the HOG characteristic vector U of the palm image of the preset template:
Figure 863383DEST_PATH_IMAGE026
wherein H isPalm area binary image I4U is the HOG feature vector of the preset template, HiIs a palm area binary image I4Of the ith HOG feature vector, UiIs the ith HOG feature vector of the template, n is the number of feature vectors, i is an integer and
Figure 934108DEST_PATH_IMAGE027
；
7.3) comparing the cosine similarity with the similarity threshold value, and judging the palm:
Figure 49831DEST_PATH_IMAGE028
wherein, I4Is a palm area binary diagram, th2 is a similarity threshold, th2 is a constant and-1 is greater than or equal to th2 is less than or equal to 1.
7.4) Using rectangular frame pairs I4And marking the current frame image corresponding to the palm area.
Compared with the prior art, the technical scheme provided by the invention has the following beneficial effects:
1. aiming at the palm detection problem under non-contact imaging, the invention improves the elliptical skin color model to carry out skin color detection by adopting self-adaptive illumination compensation based on gamma conversion and a mode of carrying out segmentation judgment on ellipses with different sizes according to the brightness value, and the obtained region characteristics improve the accuracy of subsequent palm detection;
2. the invention adopts a mode of combining skin color information and motion information, eliminates the interference of human face, judges whether a moving target exists according to a binary image obtained by a motion model, and solves the problem that palm staying cannot be detected;
3. according to the method, the possibly-appearing wrist area is removed according to the linear detection, the final palm area judgment is carried out according to the shape characteristics of the palm, and the average detection rate is as high as 99.5%;
4. the method is suitable for various conditions in the palm acquisition process, can more accurately detect the palm area, and lays a good foundation for subsequent feature extraction and identification.
Drawings
FIG. 1 is a flow chart of a non-contact palm detection method based on video sequences according to the present invention;
FIG. 2 is a frame of acquired palm image;
FIG. 3 is a binary image after skin color detection;
FIG. 4 is a binary image after moving object detection;
FIG. 5 is a fused binary image;
FIG. 6 is a binary image after morphological processing;
FIG. 7 is a binary image with the wrist area removed;
fig. 8 is a final palm detection result image.
Detailed Description
For further understanding of the present invention, the present invention will be described in detail with reference to examples, which are provided for illustration of the present invention but are not intended to limit the scope of the present invention.
Example 1
Referring to fig. 1, the present embodiment relates to a non-contact palm detection method based on a video sequence, which includes the following steps:
1) acquiring a palm video image under non-contact imaging, wherein the first frame image is a background without a palm, the palm is placed at the beginning of the second frame, and each frame of the image has a size of M × N, wherein M, N represents the number of rows and columns of the image respectively, and M × N is 640 × 480 in the embodiment, as shown in fig. 2.
2) Using an improved elliptical skin color model to detect skin color of each frame of image, and obtaining a skin color area binary image in a frame t
Figure 889611DEST_PATH_IMAGE029
As shown in fig. 3, the specific steps include:
2.1) carrying out adaptive illumination compensation based on gamma transformation on the current frame image, and converting from an RGB color space to a YCbCr color space:
Figure 370271DEST_PATH_IMAGE030
Figure 725029DEST_PATH_IMAGE002
wherein Y is a luminance component of a color in the YCbCr color space, Cb, Cr are density offset components of blue and red in the YCbCr color space, respectively, R, G, B are colors of three channels of red, green, and blue in the RGB color space, respectively, c is a constant,
Figure 582127DEST_PATH_IMAGE003
in order to be the gamma conversion coefficient(s),
Figure 276413DEST_PATH_IMAGE004
th0 is the pixel threshold and 0 ≦ th0 ≦ 255 for the average pixel value of each component of the image. Th0 in this example is 150;
2.2) clustering the skin points according to the elliptical skin color model, wherein the elliptical skin color model is as follows:
Figure 990291DEST_PATH_IMAGE005
Figure 35608DEST_PATH_IMAGE006
wherein, x in the formula (3) is an abscissa in the ellipse, y is an ordinate in the ellipse, ecx and ecy are an abscissa and an ordinate of the center point of the ellipse, a is the major axis of the ellipse, and b is the minor axis of the ellipse; theta in the formula (4) is the rotation angle of the ellipse, Cb and Cr are the concentration offset components of blue and red colors in the YCbCr color space, respectively,
Figure 991669DEST_PATH_IMAGE007
is the horizontal coordinate of the center point of the skin color model in the CbCr coordinate system,
Figure 540462DEST_PATH_IMAGE008
the longitudinal coordinate of the skin color model at the central point of the CbCr coordinate system;
in this example
Figure 628504DEST_PATH_IMAGE007
=113，
Figure 957854DEST_PATH_IMAGE008
=155.5，
Figure 156754DEST_PATH_IMAGE031
=2.51，ecx=1.63，ecy=2.44，a=23.20，b=15.2；
2.3) dividing elliptical models with different sizes according to the Y-channel brightness value, and judging whether the pixels are skin points according to whether the pixels are positioned in an ellipse with corresponding brightness:
Figure 622370DEST_PATH_IMAGE032
wherein, I1(i, j) is the skin color region binary image pixel value, Y (i, j) is the Y channel pixel value, i is the row number index of the image, j is the column number index of the image,
Figure 881313DEST_PATH_IMAGE010
the average brightness of the Y channel is shown, ecx and ecy are the abscissa and ordinate of the center point of the ellipse, a is the major axis of the ellipse, and b is the minor axis of the ellipse.
3) Detecting the moving target by using a ViBe algorithm, and obtaining a motion region binary image in a t frame
Figure 635643DEST_PATH_IMAGE033
As shown in fig. 4, the specific steps include:
3.1) initializing the background model with the first frame image to
Figure 434972DEST_PATH_IMAGE011
As a neighborhood point set around point x, a background model is constructed:
Figure 958357DEST_PATH_IMAGE034
wherein the content of the first and second substances,
Figure 388201DEST_PATH_IMAGE013
are the pixels in the initial background model,
Figure 928029DEST_PATH_IMAGE014
for the pixel values in the original image,
Figure 468732DEST_PATH_IMAGE011
is a set of neighborhood points around point x, y is
Figure 174520DEST_PATH_IMAGE011
Point (2).
3.2) updating the background model.
4) Statistic t frame motion area binary image I2The number of the white pixel points is compared with the threshold value, if the number is larger than the threshold value, the step 5) is continued, and the specific steps comprise:
4.1) statistical binary map
Figure 509686DEST_PATH_IMAGE033
The number of white pixels:
Figure 238608DEST_PATH_IMAGE015
wherein num is a motion region binary image I2The number of white pixels M, N represents the line height and line width of the current frame image, respectively, 640 and 480 in this embodiment; i is the row index of the image, and j is the column index of the image;
4.2) according to I2The area of the white region (th 1) is a threshold value:
Figure 379739DEST_PATH_IMAGE035
wherein, I2Is a motion region binary image with num of I2Th1 is a number threshold, th1 is an integer, and th1 is 1000 in this embodiment;
4.3) if the current frame image has a moving target, performing step 5), if the current frame image does not have the moving target, judging whether the t-1 moment image contains a rectangular frame mark, if so, continuing to mark the same position of the t frame image, and if not, returning to the step 1) to perform detection on the t +1 moment image.
5) At t frame will
Figure 877716DEST_PATH_IMAGE029
And
Figure 383784DEST_PATH_IMAGE033
fusing to obtain a binary image
Figure 662319DEST_PATH_IMAGE036
And performing morphological treatment on the obtained product, and reserving a maximum communication area, wherein the method specifically comprises the following steps:
5.1) mixing I1And I2The fusion with operation is performed as shown in fig. 5:
Figure 544824DEST_PATH_IMAGE037
wherein, I3To fuse binary images, I1Is a binary map of the skin color region, I2For the motion region binary image, i is the row index of the image, and j is the column index of the image.
5.2) pairs of I3Performing morphological opening operation and closing operation processing in sequence;
5.3) searching a connected region of the result binary image after the morphological processing, and reserving the maximum connected region, wherein the result is shown in FIG. 6.
6) According to Hough transform pair I3To carry outRemoving the wrist area to obtain a palm area binary image
Figure 464019DEST_PATH_IMAGE038
As shown in fig. 7, the specific steps include:
6.1) Pair fusion binary image I3Performing edge detection to obtain I3Is (d) profile L3；
6.2) pairs of L3Carrying out Hough transform to obtain L3The upper straight line set L comprises a starting point and an end point of each line segment;
6.3) calculation of I3The center of gravity of (1):
Figure 406568DEST_PATH_IMAGE018
wherein the content of the first and second substances,
Figure 844502DEST_PATH_IMAGE019
is the abscissa of the point of gravity,
Figure 327436DEST_PATH_IMAGE020
is the ordinate of the center of gravity point, ((
Figure 800006DEST_PATH_IMAGE021
,
Figure 710193DEST_PATH_IMAGE022
) Is the coordinates of the pixel points of the image,
Figure 635424DEST_PATH_IMAGE023
is the pixel value of the point;
6.4) detecting whether a parallel straight line section below the gravity center exists in the straight line set L, and if so, detecting I3And setting the pixel points below the straight line starting point of the middle corresponding position as 0:
Figure 859732DEST_PATH_IMAGE024
wherein, I4For a palm region binary map, x0 represents the x coordinate of the start of a parallel straight line segment located below the center of gravity, i.e., the x coordinate
Figure 249125DEST_PATH_IMAGE025
I is the index of the number of rows of the image and j is the index of the number of columns of the image.
7) Extracting a binary image
Figure 533476DEST_PATH_IMAGE038
The final judgment of the palm area is carried out by the HOG characteristic vector, and the specific steps comprise:
7.1) extracting a palm region binary image I4HOG feature vector H;
7.2) carrying out cosine similarity calculation on the H and the HOG characteristic vector U of the palm image of the preset template:
Figure 946002DEST_PATH_IMAGE039
wherein H is a palm area binary image I4U is the HOG feature vector of the preset template, HiIs a palm area binary image I4Of the ith HOG feature vector, UiIs the ith HOG feature vector of the template, n is the number of feature vectors, i is an integer and
Figure 272204DEST_PATH_IMAGE027
；
7.3) comparing the cosine similarity with the similarity threshold value, and judging the palm:
Figure 453786DEST_PATH_IMAGE028
wherein, I4Is a palm area binary diagram, th2 is a similarity threshold, th2 is a constant, th2 is equal to or more than-1 and equal to or less than 1, in this embodiment, th2 is 0.8;
7.4) Using rectangular frame pairs I4Marking the current frame image corresponding to the palm areaAs shown in fig. 8.
The following are experimental results and analyses of different video sequences using the detection method of the present invention.
In order to verify the non-contact palm detection method based on the video sequence, four groups of video sequences are collected, namely palm collection, palm resting collection, fist collection and palm not-off collection, wherein each group of collected 1000 palm videos respectively comprises 500 frames, and an experiment database is formed. Based on OpenCV4.1.0, Visual Studio2019 is used as compiling software, and an operating system of a computer is 64-bit Window10, a memory 8G and a main frequency of 2.30 GHz. The palm area of each group of video sequences is detected by the detection method and the fingertip positioning method disclosed in the chinese patent CN104102347A, and the detection results are shown in table 1.
TABLE 1
Figure 971355DEST_PATH_IMAGE040
As can be seen from the data in table 1, for the video images with the first group and the second group placed on the palm, the detection rate of the method of the present invention is up to 99.5% on average, while the detection rate of the fingertip positioning method is 87.4%, and for the video images without the palm placed on the third group and the fourth group, the detection rate of the method of the present invention is 0, and the fingertip positioning method has more false detections, so that the non-contact palm detection method based on the video sequence provided by the present invention can effectively detect the palm area acquired under the non-contact imaging, and provides a good guarantee for the subsequent identification process.
The above-described embodiments are illustrative examples of the present invention, which are provided for better understanding of the present invention and are not to be construed as limiting the scope of the present invention. All equivalent changes, modifications and the like made in accordance with the scope of the present invention shall fall within the protection scope of the present invention.

Claims (4)

1. A non-contact palm detection method based on a video sequence is characterized in that: which comprises the following steps:
1) acquiring a palm video image under non-contact imaging, wherein a first frame image is a background without a palm, and the palm is placed from a second frame image;
2) carrying out skin color detection on each frame of image, and obtaining a skin color area binary image I in a frame t1；
3) Detecting the moving object and obtaining a motion area binary image I in the t frame2；
4) Statistic t frame motion area binary image I2Comparing the number of the white pixel points with a threshold value, and continuing to perform the step 5) if the number is larger than the threshold value;
5) binary image I of skin color area in t frame1And a motion region binary image I2Performing fusion to obtain a fused binary image I3Performing morphological treatment on the obtained product, and reserving a maximum communication area;
6) for the fused binary image I3Removing the wrist area to obtain a palm area binary image I4；
7) Extracting a palm region binary image I4Carrying out final judgment on the palm area by using the HOG characteristic;
the step 2) of performing skin color detection on the current frame image according to the improved elliptical skin color model comprises the following specific steps:
2.1) carrying out adaptive illumination compensation based on gamma transformation on the current frame image, and converting from an RGB color space to a YCbCr color space:
Figure 435265DEST_PATH_IMAGE001
Figure 463001DEST_PATH_IMAGE002
wherein Y is a luminance component of a color in the YCbCr color space, Cb, Cr are density offset components of blue and red in the YCbCr color space, respectively, R, G, B are colors of three channels of red, green, and blue in the RGB color space, respectively, c is a constant,
Figure 626129DEST_PATH_IMAGE003
in order to be the gamma conversion coefficient(s),
Figure 12111DEST_PATH_IMAGE004
th0 is the pixel threshold value and is 0 ≦ th0 ≦ 255;
2.2) clustering the skin points according to the elliptical skin color model, wherein the elliptical skin color model is as follows:
Figure 526269DEST_PATH_IMAGE005
Figure 593582DEST_PATH_IMAGE006
wherein, x in the formula (3) is an abscissa in the ellipse, y is an ordinate in the ellipse, ecx and ecy are an abscissa and an ordinate of the center point of the ellipse, a is the major axis of the ellipse, and b is the minor axis of the ellipse; theta in the formula (4) is the rotation angle of the ellipse, Cb and Cr are the concentration offset components of blue and red colors in the YCbCr color space, respectively,
Figure 611217DEST_PATH_IMAGE007
is the horizontal coordinate of the center point of the skin color model in the CbCr coordinate system,
Figure 168100DEST_PATH_IMAGE008
the longitudinal coordinate of the skin color model at the central point of the CbCr coordinate system;
2.3) dividing elliptical models with different sizes according to the Y-channel brightness value, and judging whether the pixels are skin points according to whether the pixels are positioned in an ellipse with corresponding brightness:
Figure DEST_PATH_IMAGE009
wherein, I1(i, j) is the skin color region binary image pixel value, Y (i, j) is the Y channel pixel value, i is the row number index of the image, j is the column number index of the image,
Figure 331404DEST_PATH_IMAGE010
the average brightness of a Y channel is obtained, ecx and ecy are the abscissa and ordinate of the central point of the ellipse, a is the major axis of the ellipse, and b is the minor axis of the ellipse;
the step 6) is carried out according to Hough transform pair I3The method for removing the wrist area comprises the following specific steps:
6.1) Pair fusion binary image I3Performing edge detection to obtain I3Is (d) profile L3；
6.2) pairs of L3Carrying out Hough transform to obtain L3The upper straight line set L comprises a starting point and an end point of each line segment;
6.3) calculation of I3The center of gravity of (1):
Figure 999146DEST_PATH_IMAGE011
wherein the content of the first and second substances,
Figure 871287DEST_PATH_IMAGE012
is the abscissa of the point of gravity,
Figure 864651DEST_PATH_IMAGE013
is the ordinate of the center of gravity point, ((
Figure 524040DEST_PATH_IMAGE014
,
Figure 401997DEST_PATH_IMAGE015
) Is the coordinates of the pixel points of the image,
Figure 659803DEST_PATH_IMAGE016
is the pixel value of the point;
6.4) detecting whether a parallel straight line section below the gravity center exists in the straight line set L, and if so, detecting I3And setting the pixel points below the straight line starting point of the middle corresponding position as 0:
Figure 558489DEST_PATH_IMAGE017
wherein, I4For a palm region binary map, x0 represents the x coordinate of the start of a parallel straight line segment located below the center of gravity, i.e., the x coordinate
Figure DEST_PATH_IMAGE018
I is the index of the number of rows of the image, and j is the index of the number of columns of the image;
the step 7) comprises the following specific steps:
7.1) extracting a palm region binary image I4HOG feature vector H;
7.2) carrying out cosine similarity calculation on the H and the HOG characteristic vector U of the palm image of the preset template:
Figure 176946DEST_PATH_IMAGE019
wherein H is a palm area binary image I4U is the HOG feature vector of the preset template, HiIs a palm area binary image I4Of the ith HOG feature vector, UiIs the ith HOG feature vector of the template, n is the number of feature vectors, i is an integer and
Figure 452069DEST_PATH_IMAGE020
；
7.3) comparing the cosine similarity with the similarity threshold value, and judging the palm:
Figure DEST_PATH_IMAGE021
wherein, I4Is a palm area binary image, th2 is a similarity threshold, th2 is a constant, and-1 is more than or equal to th2 is less than or equal to 1;
7.4) Using rectangular frame pairs I4And marking the current frame image corresponding to the palm area.
2. The non-contact palm detection method based on video sequence according to claim 1, characterized in that: the step 3) of detecting the moving target according to the ViBe algorithm comprises the following specific steps:
3.1) initializing the background model with the first frame image to
Figure 33223DEST_PATH_IMAGE022
As a neighborhood point set around point x, a background model is constructed:
Figure 40494DEST_PATH_IMAGE023
wherein the content of the first and second substances,
Figure 238257DEST_PATH_IMAGE024
are the pixels in the initial background model,
Figure 51492DEST_PATH_IMAGE025
for the pixel values in the original image,
Figure 516846DEST_PATH_IMAGE022
is a set of neighborhood points around point x, y is
Figure 695018DEST_PATH_IMAGE022
A point of (1);
3.2) updating the background model.
3. The non-contact palm detection method based on video sequence according to claim 1, characterized in that: the specific steps in the step 4) comprise:
4.1) statistical motion region binary map I2The number of white pixels:
Figure 583339DEST_PATH_IMAGE026
wherein num is a motion region binary image I2M, N respectively represent the line height and the line width of the current frame image, i is the line index of the image, and j is the line index of the image;
4.2) according to I2The number of white areas (2) is used for judging whether a moving object exists or not:
Figure 373834DEST_PATH_IMAGE027
wherein, I2Is a motion region binary image with num of I2Th1 is a quantity threshold value, th1 is an integer;
4.3) if the current frame image has a moving target, performing step 5), if no moving target exists, judging whether the t-1 time image contains a rectangular frame mark, if so, continuing to mark the same position of the t frame image, and if not, returning to the step 1) to detect the t +1 time image.
4. The non-contact palm detection method based on video sequence according to claim 1, characterized in that: the step 5) comprises the following specific steps:
5.1) mixing I1And I2Performing fusion with the operation:
Figure 601684DEST_PATH_IMAGE028
wherein, I3To fuse binary images, I1Is a binary map of the skin color region, I2Is a binary map of the motion region, i is a mapRow index of the image, j is column index of the image;
5.2) pairs of I3Performing morphological opening operation and closing operation processing in sequence;
5.3) searching a connected region of the result binary image after morphological processing, and reserving the maximum connected region.
CN202011499311.9A 2020-12-17 2020-12-17 Non-contact palm detection method based on video sequence Active CN112232332B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202011499311.9A CN112232332B (en) 2020-12-17 2020-12-17 Non-contact palm detection method based on video sequence

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202011499311.9A CN112232332B (en) 2020-12-17 2020-12-17 Non-contact palm detection method based on video sequence

Publications (2)

Publication Number Publication Date
CN112232332A CN112232332A (en) 2021-01-15
CN112232332B true CN112232332B (en) 2021-04-13

Family

ID=74124909

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202011499311.9A Active CN112232332B (en) 2020-12-17 2020-12-17 Non-contact palm detection method based on video sequence

Country Status (1)

Country Link
CN (1) CN112232332B (en)

Families Citing this family (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112801062B (en) * 2021-04-07 2021-07-06 平安科技（深圳）有限公司 Live video identification method, device, equipment and medium
CN113128435B (en) * 2021-04-27 2022-11-22 南昌虚拟现实研究院股份有限公司 Hand region segmentation method, device, medium and computer equipment in image
CN113362390B (en) * 2021-06-21 2023-03-21 武汉理工大学 Rapid circular target positioning video processing method based on ellipse detection
CN114694233B (en) * 2022-06-01 2022-08-23 成都信息工程大学 Multi-feature-based method for positioning human face in examination room monitoring video image
CN115376167B (en) * 2022-10-26 2023-02-24 山东圣点世纪科技有限公司 Palm detection method and system under complex background

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102521567A (en) * 2011-11-29 2012-06-27 Tcl集团股份有限公司 Human-computer interaction fingertip detection method, device and television
CN103714328A (en) * 2013-12-31 2014-04-09 江苏图云智能科技发展有限公司 Adaboost algorithm for human face detection
CN104167006A (en) * 2014-07-10 2014-11-26 华南理工大学 Gesture tracking method of any hand shape
CN109034016A (en) * 2018-07-12 2018-12-18 辽宁工业大学 A kind of hand back vein image-recognizing method based on S-CNN model of universality
CN110008824A (en) * 2019-02-20 2019-07-12 平安科技（深圳）有限公司 Palm grain identification method, device, computer equipment and storage medium
CN111860424A (en) * 2020-07-30 2020-10-30 厦门熵基科技有限公司 Training method and device for visible light palm recognition model

Family Cites Families (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20110007806A (en) * 2009-07-17 2011-01-25 삼성전자주식회사 Apparatus and method for detecting hand motion using a camera
CN101916370B (en) * 2010-08-31 2012-04-25 上海交通大学 Method for processing non-feature regional images in face detection
KR101868903B1 (en) * 2012-01-11 2018-06-20 한국전자통신연구원 Apparatus and method for tracking human hand by using color features
CN103810490B (en) * 2014-02-14 2017-11-17 海信集团有限公司 A kind of method and apparatus for the attribute for determining facial image
KR101614798B1 (en) * 2014-08-22 2016-04-22 군산대학교 산학협력단 Non-contact multi touch recognition method and system using color image analysis
CN105975903A (en) * 2016-04-22 2016-09-28 安徽大学 Automobile feature identification method
CN109903275B (en) * 2019-02-13 2021-05-18 湖北工业大学 Fermented grain mildewing area detection method based on self-adaptive multi-scale filtering and histogram comparison
CN111104870B (en) * 2019-11-27 2024-02-13 珠海欧比特卫星大数据有限公司 Motion detection method, device, equipment and storage medium based on satellite video
CN111259866B (en) * 2020-03-06 2023-07-28 大连科技学院 Marine ship target detection method based on improved background difference method

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102521567A (en) * 2011-11-29 2012-06-27 Tcl集团股份有限公司 Human-computer interaction fingertip detection method, device and television
CN103714328A (en) * 2013-12-31 2014-04-09 江苏图云智能科技发展有限公司 Adaboost algorithm for human face detection
CN104167006A (en) * 2014-07-10 2014-11-26 华南理工大学 Gesture tracking method of any hand shape
CN109034016A (en) * 2018-07-12 2018-12-18 辽宁工业大学 A kind of hand back vein image-recognizing method based on S-CNN model of universality
CN110008824A (en) * 2019-02-20 2019-07-12 平安科技（深圳）有限公司 Palm grain identification method, device, computer equipment and storage medium
CN111860424A (en) * 2020-07-30 2020-10-30 厦门熵基科技有限公司 Training method and device for visible light palm recognition model

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
基于直线检测的手臂去除算法;王凤艳;《电子技术》;20171231;第46卷(第7期);第32页摘要、33页第2节及图2、34页图5 *
基于肤色分割的视频人体检测;王红梅等;《计算机工程与设计》;20111231;第32卷(第3期);第980-982页第3.1节 *
基于肤色椭圆边界模型和改进帧差分算法的手势分割方法;张忠军;《福建电脑》;20181231;第34卷(第5期);第8页第0节、第9页2-4节以及图3-4 *

Also Published As

Publication number Publication date
CN112232332A (en) 2021-01-15

Similar Documents

Publication Publication Date Title
CN112232332B (en) Non-contact palm detection method based on video sequence
Pan et al. A robust system to detect and localize texts in natural scene images
CN104732200B (en) A kind of recognition methods of skin type and skin problem
CN104680127A (en) Gesture identification method and gesture identification system
CN111126240B (en) Three-channel feature fusion face recognition method
CN105844242A (en) Method for detecting skin color in image
Monwar et al. Pain recognition using artificial neural network
CN101996317B (en) Method and device for identifying markers in human body
CN108256518A (en) Detection method and detection device for character region
Youlian et al. Face detection method using template feature and skin color feature in rgb color space
CN109753912B (en) Multispectral palm print matching method based on tensor
CN116665258A (en) Palm image finger seam segmentation method
CN109800771B (en) Spontaneous micro-expression positioning method of local binary pattern of mixed space-time plane
Işikdoğan et al. Automatic recognition of Turkish fingerspelling
CN108629780B (en) Tongue image segmentation method based on color decomposition and threshold technology
CN103020631A (en) Human movement identification method based on star model
CN106066887A (en) A kind of sequence of advertisements image quick-searching and the method for analysis
Yi et al. Face detection method based on skin color segmentation and facial component localization
Shemshaki et al. Lip segmentation using geometrical model of color distribution
Ghimire et al. A lighting insensitive face detection method on color images
CN110956095A (en) Multi-scale face detection method based on corner skin color detection
Lakshmi et al. Plant leaf image detection method using a midpoint circle algorithm for shape-based feature extraction
CN104866825B (en) A kind of sign language video frame sequence classification method based on Hu square
Zhifan et al. Research and implementation of an improved license plate recognition algorithm
CN117037049B (en) Image content detection method and system based on YOLOv5 deep learning

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant
TR01 Transfer of patent right

Effective date of registration: 20210910

Address after: 030032 room 0906, floor 9, building C, qingkong innovation base, No. 529, South Central Street, Taiyuan Xuefu Park, comprehensive reform demonstration zone, Taiyuan, Shanxi Province

Patentee after: Holy Point Century Technology Co.,Ltd.

Address before: 9 / F, unit 1, building 2, no.41-5, Jinsha North 2nd Road, Jinniu District, Chengdu, Sichuan 610000

Patentee before: Sichuan ShengDian Century Technology Co.,Ltd.

TR01 Transfer of patent right
TR01 Transfer of patent right

Effective date of registration: 20230330

Address after: Room 2309, 23rd Floor, Qidi Building, No. 99, South Tiancheng Road, High Speed Rail New Town, Xiangcheng District, Suzhou City, Jiangsu Province, 215000 - Work Station A029 (cluster registration)

Patentee after: Jiangsu Shengdian Century Technology Co.,Ltd.

Address before: 030032 room 0906, floor 9, building C, qingkong innovation base, No. 529, South Central Street, Taiyuan Xuefu Park, comprehensive reform demonstration zone, Taiyuan, Shanxi Province

Patentee before: Holy Point Century Technology Co.,Ltd.

TR01 Transfer of patent right