CN108255298B - Infrared gesture recognition method and device in projection interaction system - Google Patents
Infrared gesture recognition method and device in projection interaction system Download PDFInfo
- Publication number
- CN108255298B CN108255298B CN201711498318.7A CN201711498318A CN108255298B CN 108255298 B CN108255298 B CN 108255298B CN 201711498318 A CN201711498318 A CN 201711498318A CN 108255298 B CN108255298 B CN 108255298B
- Authority
- CN
- China
- Prior art keywords
- gesture
- image
- images
- threshold
- preprocessed
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/017—Gesture based interaction, e.g. based on a set of recognized hand gestures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
- G06F3/041—Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
- G06F3/042—Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means by opto-electronic means
- G06F3/0421—Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means by opto-electronic means by interrupting or reflecting a light beam, e.g. optical touch-screen
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/107—Static hand or arm
- G06V40/113—Recognition of static hand signs
Abstract
The invention discloses an infrared gesture recognition method and equipment in a projection interaction system, which belong to the technical field of data processing, and comprise the steps of respectively preprocessing acquired gesture images by adopting a dynamic threshold method, an Otsu threshold method and an adaptive threshold method to obtain different preprocessed images; carrying out binarization operation on different preprocessed images, and carrying out image feature extraction on the obtained binarized image to obtain gesture features; and calculating the distance between the gesture features and the geometric features of each gesture image in the gesture library, and taking the gesture in the standard library corresponding to the minimum value of the geometric feature distances as a gesture recognition result. The three thresholding methods are respectively operated, each thresholding method corresponds to a final gesture recognition rate, and the thresholding method with the highest gesture recognition rate is finally selected for use, so that the accuracy of a gesture recognition result is improved.
Description
Technical Field
The invention relates to the technical field of data processing, in particular to an infrared gesture recognition method in a projection interaction system.
Background
With the development of society, the application of human-computer interaction technology in daily life is increasingly wide, which mainly researches information interaction between a computer and a human body, and the development of human-computer interaction technology is further promoted due to the appearance of a touch display screen. Under the condition of huge potential market of human-computer interaction, the interactive projection system is produced. For the interactive projection system, the interactive projection system has the advantages of low cost, convenience in operation, good use effect and the like, is widely applied to enterprises, institutional academic conferences and daily teaching of schools, greatly improves the man-machine interaction efficiency, facilitates communication between users and audiences, and has a wide development prospect.
In the interactive projection system, a screen can be operated by using a special laser pen, and the screen can also be operated by using a finger. However, if the area pointed by the finger to the screen is not accurately positioned during the operation of the screen by the finger, the effect of the whole interactive projection is affected.
In the existing gesture recognition technology, generally, collected gesture images are processed, and the processed gesture images are matched with template gestures, so that gestures with the best similarity are found out and used as correct output of gesture recognition. The accuracy of gesture recognition mainly determines the gesture region positioning accuracy, and the key of the gesture region positioning accuracy is whether the thresholding operation of the region is accurate. However, in the actual operation process, because the processing effect on the acquired gesture image is not good, for example, some noise points may be introduced by operations such as image thresholding, and the like, thereby affecting the accuracy of the gesture recognition operation.
Disclosure of Invention
The invention aims to provide an infrared gesture recognition method and device in a projection interaction system so as to improve the accuracy of gesture recognition.
In order to realize the purpose, the invention adopts the technical scheme that:
in a first aspect, a method for infrared gesture recognition in a projection interaction system is adopted, which includes:
respectively preprocessing the acquired gesture images by adopting a dynamic threshold method, an Otsu threshold method and a self-adaptive threshold method to obtain different preprocessed images;
carrying out binarization operation on different preprocessed images, and carrying out image feature extraction on the obtained binarized image to obtain gesture features;
and calculating the distance between the gesture features and the geometric features of each gesture image in the gesture library, and taking the gesture GE in the standard library corresponding to the minimum distance between the geometric features as a gesture recognition result.
Preferably, the preprocessing the acquired gesture images by using a dynamic threshold method includes:
carrying out graying operation on the acquired gesture image to obtain a grayscale image;
establishing a rectangular coordinate system by taking the upper left corner of the gray level image as the origin of the coordinate system, and obtaining the position coordinates of the light source points in the rectangular coordinate system;
calculating Euclidean distance d between any point (x, y) in the gray level image and the light source point(x,y)And obtaining the farthest distance d between each point in the gray image and the point light source pointmax；
According to the maximum distance dmaxAnd the Euclidean distance d between each point and the light source point(x,y)And a set fixed threshold k, calculating a corresponding threshold k (x, y) at the point (x, y);
utilization point (x),y), performing thresholding operation on the gesture image to obtain the preprocessed image.
Preferably, the preprocessing the acquired gesture images by using the greater fluid threshold method includes:
setting different gray threshold values T, and comparing the gray value of each point pixel in the acquired gesture image with the different gray threshold values T;
the number N of pixels with the gray value of the pixels in the gesture image smaller than the threshold value T0Number N of pixels having a pixel gradation larger than threshold value T1；
Obtaining the number N of pixels according to the size of the gesture image and the comparison under the same gray threshold value T0Number of pixels N1Calculating the inter-class variance under the gray threshold T as g;
traversing the inter-class variance values under different gray threshold values T, and performing thresholding operation on the gesture image by taking the gray threshold value T when the inter-class variance value reaches the maximum as a threshold value to obtain the preprocessed image.
Preferably, the preprocessing the acquired gesture images by using an adaptive threshold method includes:
setting a block with the size of b and a constant c for the gesture image, wherein b is an odd number;
acquiring the sum of gray values corresponding to all coordinate points (i, j) in the gesture image within the range of b multiplied by b, and recording the sum as g (i, j);
calculating a corresponding threshold value t (i, j) at the coordinate point (i, j) in the b × b range according to the gray value sum and the constant c;
and (3) carrying out binarization operation on the gesture image by using a corresponding threshold value t (i, j) at the coordinate point (i, j) to obtain the preprocessed image.
Preferably, after the collected gesture images are respectively preprocessed by using a dynamic threshold method, an incredible threshold method and an adaptive threshold method to obtain different preprocessed images, the method further includes:
(1) detecting the connected regions of the preprocessed image, and extracting all the connected regions in the preprocessed image;
(2) comparing the area of each communication area with a set communication area threshold value, and judging whether the communication area is connected or not;
(3) when the connected domains are judged not to be connected, finger area positioning is carried out according to the number of the connected domains;
(4) and when judging that the connected domains are connected, comparing the size of each connected domain with a set connected domain size threshold value, distinguishing the connected domains, and then executing the step (3).
Preferably, after the collected gesture images are respectively preprocessed by using a dynamic threshold method, an incredible threshold method and an adaptive threshold method to obtain different preprocessed images, the method further includes:
and obtaining the size of the connected domain after background modeling or thresholding operation, and removing noise points.
Preferably, the moment feature extraction is performed on the binarized image, and specifically includes:
according toThe gesture image obtains a p + q order geometric moment mpqAnd central moment mupqWherein p + q is more than or equal to 2;
according to the central moment mupqObtaining a normalized center distance npq；
According to the central moment n of the second and third order scalespqObtaining u invariant moment groups;
and (3) selecting v moment groups from the u invariant moment groups to form invariant moment feature vectors, and using the invariant moment vectors as moment features, wherein v is more than or equal to 1 and less than or equal to u.
Preferably, the calculating the geometric feature distance between the gesture feature and each gesture image in the gesture library, and taking the gesture GE in the standard library corresponding to the minimum geometric feature distance as the gesture recognition result specifically includes:
calculating the distance D between the gesture features and the geometric features of the gesture images in the gesture librarymThe calculation formula is as follows:
wherein M isiIs the gesture image I(k)Geometric moment component of (G)iFor gesture image I in gesture library(l)Corresponding geometric moment component, ωiIs a component adjustment factor;
at a distance D of each geometric featuremFinds D inmAnd taking the gesture GE as a gesture recognition result when the gesture GE in the standard library corresponds to the minimum value.
In a second aspect, an infrared gesture recognition device in a projection interaction system is adopted, and the device comprises a preprocessing module, a feature extraction module and a recognition module;
the preprocessing module is used for respectively preprocessing the acquired gesture images by adopting a dynamic threshold method, an Otsu threshold method and an adaptive threshold method to obtain different preprocessed images;
the feature extraction module is used for carrying out binarization operation on different preprocessed images and carrying out image feature extraction on the obtained binarized images to obtain gesture features;
and the recognition module is used for calculating the distance between the gesture features and the geometric features of each gesture image in the gesture library, and taking the gesture GE in the standard library corresponding to the minimum geometric feature distance as a gesture recognition result.
In a third aspect, an infrared gesture recognition device in a projection interaction system is adopted, which includes a memory and a processor, wherein the memory stores a plurality of program instructions, and the processor is adapted to load the plurality of program instructions and execute:
respectively preprocessing the acquired gesture images by adopting a dynamic threshold method, an Otsu threshold method and a self-adaptive threshold method to obtain different preprocessed images;
carrying out binarization operation on different preprocessed images, and carrying out image feature extraction on the obtained binarized image to obtain gesture features;
and calculating the distance between the gesture features and the geometric features of each gesture image in the gesture library, and taking the gesture GE in the standard library corresponding to the minimum distance between the geometric features as a gesture recognition result.
Compared with the prior art, the invention has the following technical effects: the invention adopts three methods to simultaneously process the image thresholding method, namely a dynamic threshold method, an OTSU (Otsu threshold) method and an adaptive threshold method. The thresholding scenes are complex, and the absolute scene corresponding to each thresholding method is not well determined, so the scheme operates by three thresholding methods respectively, each thresholding method corresponds to a final gesture recognition rate, and a thresholding method with the highest gesture recognition rate is selected for use, so that the accuracy of a gesture recognition result is improved.
Drawings
The following detailed description of embodiments of the invention refers to the accompanying drawings in which:
FIG. 1 is a flow chart of a method for infrared gesture recognition in a projection interactive system;
FIG. 2 is a schematic diagram of a pretreatment process;
FIG. 3 is a pre-processed image resulting from performing a pre-processing operation;
fig. 4 is a schematic structural diagram of an infrared gesture recognition device in a projection interaction system.
Detailed Description
To further illustrate the features of the present invention, refer to the following detailed description of the invention and the accompanying drawings. The drawings are for reference and illustration purposes only and are not intended to limit the scope of the present disclosure.
As shown in fig. 1, the present embodiment discloses an infrared gesture recognition method in a projection interaction system, which includes the following steps S101 to S103:
s101, respectively preprocessing the acquired gesture images by adopting a dynamic threshold method, an Otsu threshold method and an adaptive threshold method to obtain different preprocessed images;
s102, carrying out binarization operation on different preprocessed images, and carrying out image feature extraction on the obtained binarized images to obtain gesture features;
s103, calculating the distance between the gesture features and the geometric features of the gesture images in the gesture library, and taking the gesture GE in the standard library corresponding to the minimum distance between the geometric features as a gesture recognition result.
It should be noted that the thresholding operation is mainly aimed at extracting the pixels of the finger in the screen as much as possible, and at the same time, the pixels of the noise point can be introduced to the minimum, so as to provide a basis for accurate positioning of the finger partial region. Therefore, a certain requirement is put on the thresholding operation in terms of how to extract the effective pixel value while introducing as few noise pixels as possible. If the threshold is set too large, much valid information is lost, and if the threshold is set too low, although valid information is retained, some noise point information is introduced.
In the scheme, three methods are mainly adopted to simultaneously process the image thresholding method, namely a dynamic threshold method, an OTSU (Otsu threshold) method and an adaptive threshold method. Because the thresholding scenes are complex and the absolute scene corresponding to each thresholding method is not well determined, the scheme adopts three thresholding methods to operate respectively, each thresholding method corresponds to a final gesture recognition rate, and a thresholding method with the highest gesture recognition rate is finally selected for use, so that the accuracy of gesture recognition is improved.
It should be noted that, in the dynamic threshold and adaptive threshold method, the thresholds corresponding to different pixel point positions of the original image are different, so that different thresholds should be used to perform thresholding operation on the original image, and the setting of the threshold is related to the positions of the surrounding pixels and the pixel points thereof, which can reduce the introduction of noise points to a certain extent and ensure that too much effective information is not lost. In the OTSU (atrazine threshold) method, the threshold is dynamically calculated, so that the inter-class variance between the background and the target is maximized, and the threshold is finally determined, thereby well ensuring that excessive noise points are not introduced. The three thresholding procedures are described in detail below:
in the fixed threshold method, the threshold is set to a fixed value, that is, the same threshold is used for thresholding for all screen regions. However, for the screen of the interactive projection system, since the laser is a point light source, the farther away from the point light source, the weaker the intensity of the received light, and the smaller the size of the reflection point, the lower the brightness threshold of the point should be set.
If the fixed threshold method is directly used, if the threshold setting is too high, much information is lost in the thresholding operation at positions farther away from the point light source, and if the threshold setting is too low, redundant information cannot be deleted in areas closer to the light source. Therefore, the dynamic thresholding method proposed in this embodiment specifically includes:
(a) the method comprises the following steps that a camera collects a screen picture and performs graying operation on the screen picture, the grayscale picture is marked as P, the size of the grayscale picture is X multiplied by Y, wherein X represents the length (length) of the grayscale picture in the horizontal direction, and Y represents the length (width) of the grayscale picture in the vertical direction;
(b) establishing a rectangular coordinate system for the gray picture P, and taking the upper left corner of the picture as the origin of the coordinate system;
(c) since the light source is positioned right above the middle of the projection screen, the coordinate position of the light source point is set to be
(d) Calculating Euclidean distance d between coordinates (x, y) of any point in the gray picture P and a light source point(x,y)Wherein
(e) Setting a fixed threshold k according to the early sampling experience, wherein under the threshold k, only a few sporadic white pixel points in the gray image are just realized, and the thresholding operation of all the pixel points into black is basically realized;
(f) obtaining the farthest distance d from the point light source point in the gray imagemaxWherein
(g) Since the farther away from the light source point, the lower the intensity of the light source corresponding to the light source point, the lower the threshold should be, and we consider that there is a linear relationship, the corresponding threshold at point (x, y)
It should be noted that, in this embodiment, a threshold method of OTSU in the prior art is adopted. In OTSU, an image is divided into two parts, a background and an object, according to the gray characteristics of the image. The larger the inter-class variance between the background and the object, the larger the difference between the two parts constituting the image, and the smaller the difference between the two parts when part of the object is mistaken for the background or part of the background is mistaken for the object. Thus, a segmentation that maximizes the inter-class variance means that the probability of false positives is minimized. The OTSU threshold method comprises the following specific processing procedures:
assuming that the background of the image is dark and the size of the image is M × N, a threshold T is set arbitrarily first, and the number of pixels in the image whose gray-scale value is smaller than the threshold T is denoted as N0The number of pixels having a pixel gray level greater than the threshold T is denoted by N1Then, there are:
ω0＝N0/M×N
ω1＝N1/M×N
N0+N1＝M×N
ω0+ω1＝1；
wherein, the ratio of the number of pixels belonging to the foreground in the whole image is marked as omega0Average gray level mu of0(ii) a The proportion of the number of background pixels to the whole image is omega1Average gray of μ1When the total average gray level of the image is recorded as mu and the inter-class variance is recorded as g, the following results are obtained:
μ＝ω0μ0+ω1μ1
g＝ω0(μ0-μ)2+ω1(μ1-μ)2；
according to these two equations:
g＝ω0ω1(μ0-μ1)2。
due to the difference in threshold T, the value of the inter-class variance also differs. Different threshold values T are set, and corresponding values of the inter-class variance are obtained. By traversing all the values of the inter-class variance, the threshold value T at the time when the value of the inter-class variance reaches the maximum is selected as the required threshold value.
Further, in the present embodiment, an adaptive threshold method of adaptive threshold of opencv self-contained AdaptiveThreshold is used to implement the thresholding operation of the image. The threshold values of different coordinate points in the image are different, and the implementation principle is as follows:
given an M × N digital image f (i, j), where f (i, j) is the gray level of the image at the coordinate point (i, j), a block size b and a constant c are set, where b is an odd number, then the corresponding threshold size t (i, j) at the coordinate point (i, j) is defined as follows:
(1) acquiring the sum of gray values corresponding to all coordinate points in the range of b & ltb & gt around the coordinate point (i, j), and recording the sum as g (i, j);
(2)
the corresponding threshold size at coordinate point (i, j) is obtained. In the adaptive thresholding method, the binarization threshold value at the pixel position is determined according to the pixel value distribution of the neighborhood block of the pixel. Thus the binarization threshold at each pixel position is not fixed but determined by the distribution of its surrounding neighborhood pixels. The binarization threshold value of the image area with higher brightness is generally higher, while the binarization threshold value of the image area with lower brightness is correspondingly smaller. Local image regions of different brightness, contrast, texture will have corresponding local binarization thresholds.
Further, the preprocessing process in the above embodiment further includes performing connected component detection on the image after the thresholding operation, when a plurality of users operate the projection interactive system at the same time, if the finger distance between different users triggering the projection interactive screen is too small (about half the finger nail size length), then after the camera acquires the image and the thresholding operation, it may happen that the region after thresholding adjacent fingertips may form a connected component, and if the finger distance is too large, this will not happen. Therefore, when the finger distance is too large, the finger area can be located by determining the number of connected domains, which is specifically as follows:
judging through the number of connected domains:
through early sampling, the average connected domain area of one finger is approximately estimated to be S2And setting a threshold value of the connected component size to be 1.2 × S2Since the sizes of fingers are different between different persons, a certain error range is set. Traversing the areas of all connected domains in the image, and if the areas of all connected domains are less than the threshold value of 1.2 multiplied by S2Then the distances between adjacent fingers are considered to be relatively smallThe problem that connected domains are connected together is not caused, so that the finger area can be positioned by judging the number of the connected domains, namely the position of each connected domain is the position of the finger area. If at least one connected domain has an area larger than the threshold value, the connected domains are connected together, so that the method is not suitable.
Judging by judging the size of the connected domain:
the finger distance between different users for triggering projection interactive screens is small (about half the length of finger nail), a connected domain may be formed by the thresholded regions of adjacent finger tips, and at this time, the subsequent operation cannot be performed by the number of the connected domains. In this case, therefore, we can use the judgment of the size and shape of the connected domain to separate the two connected domains.
For example: through early sampling, the average connected domain area of one finger is approximately estimated to be S2And setting a threshold value of the connected component size to be 1.2 × S2Since the sizes of fingers are different between different persons, a certain error range is set. If the obtained area of the connected domain is more than 1.2 multiplied by S2It is considered that two adjacent connected domains may be connected together, thereby facilitating the subsequent positioning operation. Alternatively, the threshold may be set by the length or width of the connected domain region, and if the two regions are connected together, the length or width is necessarily much larger than the length or width of the original region without merging.
Further, when the size of the connected domain is obtained through the detection, noise points are excluded from the connected domain, specifically:
A. method for modeling by adopting background
In the interactive projection system, if the laser is not accurately installed, noise point phenomena such as artifacts and the like are generated in the whole interactive projection system, so that the generation of the noise point phenomena is solved by adopting a background difference method, and the background difference is specifically implemented as follows:
when the laser is installedAfter the accuracy is achieved, a camera carried by the projection type interactive system is used for pre-collecting background images of a plurality of screens, and the background images are recorded as BnAveraging the multiple background images to obtain a background image with relatively stable image structure information, which is denoted as P1. Then, when a finger triggers the screen of the interactive projection system, a background image of the screen is also acquired and recorded as PnLet PnAnd P1And performing subtraction to obtain an image containing relatively less noise points, thereby laying a foundation for thresholding and positioning the finger region part behind.
B. Noise point rejection based on thresholded image size
Different from the background modeling method in the A, the scheme also adopts the method of obtaining the size of the connected domain after thresholding the image to eliminate the noise points. Through early sampling, the average connected component area S of one finger can be roughly estimated2If the size of the thresholded connected component obtained by us is far smaller than S2Then, we can consider this region as the region generated by the noise point, and can ignore it, and can also lay the foundation for the accurate positioning later.
It should be noted that, in the interactive projection system, since some non-finger regions may be introduced into the screen, such as cuffs of clothes, black dots on the projection screen, etc., the introduced regions undoubtedly bring noise hazards to the preprocessing of the image, and meanwhile, the interference of the noise dots also has a great influence on the stability of the system and the accuracy of finger position positioning, so how to solve the noise dots is very important. In the embodiment, the relevant image preprocessing obtained by the two methods is shown in fig. 3, in the drawing, the white area represents the preprocessed finger area, and the black area represents the screen area, so that the interference of noise points can be reduced to a greater extent after the preprocessing.
Further, after image preprocessing, moment features are extracted from the acquired binarized preprocessed image and are used as important bases for distinguishing different gestures. Because the coordinates of pixels are considered to be a two-dimensional random variable (x, y) for an image, a grayscale image can be represented by a two-dimensional grayscale density function, and thus the grayscale image can be characterized by moments. Invariant moment is a highly condensed image feature with translational, grayscale, scale, rotational invariance. The scheme adopts the Hu matrix as the extraction characteristic of the image, and the specific extraction process is as follows:
an M N digital image f (i, j), M, N representing the length and width of the image, respectively, with a geometric moment M of order p + qpqAnd central moment mupqRespectively as shown in the following formulas:
wherein f (i, j) is a gray value of the image at the coordinate point (i, j), and satisfies that i ═ m10/m00，j'＝m01/m00。
If m is equal to00And (i ', j') is the centroid coordinate of the image when the gray scale quality of the image is regarded. To eliminate the influence of image scale change, a normalization central moment eta is definedpqAs shown in the following equation:
the following 7 invariant moment sets (Φ 1, Φ 7) can be derived using the central moments of the second and third order gauges, respectively as follows:
Φ1＝η20+η02
Φ2＝(η20-η02)2+4η2 11
Φ3＝(η20-3η12)2+3(η21-η03)2
Φ4＝(η30+η12)2+(η21+η03)2
Φ5＝(η30+3η12)(η30+η12)[(η30+η12)2-3(η21+η03)2]+(3η21-η03)(η21+η03)[3(η30+η12)2-(η21+η03)2]
Φ6＝(η20-η02)[(η30+η12)2-(η21+η03)2]+4η11(η30+η12)(η21+η03)
Φ7＝(3η21-η03)(η30+η12)[(η30+η12)2-3(η21+η03)2]+(3η12-η30)(η21+η03)[3(η30+η12)2-(η21+η03)2]。
because the 7 invariant moment groups are kept unchanged when the image is translated, rotated and scaled, and in the scheme, in view of reducing the calculation complexity, the first four invariant moment feature quantities are adopted to form the feature vector of the gesture, namely (phi) in the embodiment1,Φ2,Φ3,Φ4) Thereby realizing the recognition of the gesture. In addition, the recognition of the gesture can be better realized by adopting the first four constant moment feature vectors.
Further, after the image features of the gesture are obtained, by defining a geometric feature distance between the gesture image and the gesture image in the gesture library, specific gesture expression contents are determined, specifically:
calculating the geometric feature distance D between the input gesture image and any one gesture image in the gesture librarymThe formula is as follows:
wherein M isiFor an input image I(k)Geometric moment component of (G)iFor gesture image I in gesture library(l)Corresponding geometric moment component, ωiThe component adjustment coefficients are used for adjusting the order of magnitude inconsistency of each moment component in the feature vector. Through the final geometric feature distance, recognition between different gestures can be finally realized.
When the geometric characteristic distance D between the input gesture image and any one gesture image in the gesture library is knownmThen, look for DmAnd when the value is minimum, the corresponding gesture GE in the standard library corresponds to the input gesture GE, so that the final gesture recognition is realized.
As shown in fig. 4, the present embodiment discloses an infrared gesture recognition device in a projection interaction system, which includes a preprocessing module 10, a feature extraction module 20, and a recognition module 30;
the preprocessing module 10 is configured to respectively preprocess the acquired gesture images by using a dynamic threshold method, an incredible threshold method and an adaptive threshold method to obtain different preprocessed images;
the feature extraction module 20 is configured to perform binarization operations on different preprocessed images, and perform image feature extraction on the obtained binarized images to obtain gesture features;
and the recognition module 30 is configured to calculate a distance between the gesture feature and a geometric feature of each gesture image in the gesture library, and use the gesture GE in the standard library corresponding to the minimum geometric feature distance as a gesture recognition result.
In addition, an infrared gesture recognition device in a projection interaction system is further adopted, which comprises a memory and a processor, wherein the memory stores a plurality of program instructions, and the processor is suitable for loading the plurality of program instructions and executing:
respectively preprocessing the acquired gesture images by adopting a dynamic threshold method, an Otsu threshold method and a self-adaptive threshold method to obtain different preprocessed images;
carrying out binarization operation on different preprocessed images, and carrying out image feature extraction on the obtained binarized image to obtain gesture features;
and calculating the distance between the gesture features and the geometric features of each gesture image in the gesture library, and taking the gesture GE in the standard library corresponding to the minimum distance between the geometric features as a gesture recognition result.
It should be understood that, in this embodiment, the infrared gesture recognition device in the projection interaction system corresponds to the infrared gesture recognition method, and the above and other operations and/or functions of each module in the device respectively implement corresponding processes of each method in fig. 1 to 2, which is not described herein again for brevity.
The above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the invention, and any modifications, equivalents, improvements and the like that fall within the spirit and principle of the present invention are intended to be included therein.
Claims (9)
1. An infrared gesture recognition method in a projection interaction system is characterized by comprising the following steps:
respectively preprocessing the acquired gesture images by adopting a dynamic threshold method, an Otsu threshold method and a self-adaptive threshold method to obtain different preprocessed images;
after the acquired gesture images are respectively preprocessed by adopting a dynamic threshold method, an Otsu threshold method and an adaptive threshold method to obtain different preprocessed images, the method further comprises the following steps:
(1) detecting the connected regions of the preprocessed image, and extracting all the connected regions in the preprocessed image;
(2) comparing the area of each communication area with a set communication area threshold value, and judging whether the communication area is connected or not;
(3) when the connected domains are judged not to be connected, finger area positioning is carried out according to the number of the connected domains;
(4) when the connected condition of the connected domains is judged, comparing the size of each connected domain with a set connected domain size threshold value, distinguishing the connected domains, and then executing the step (3);
carrying out binarization operation on different preprocessed images, and carrying out image feature extraction on the obtained binarized image to obtain gesture features;
and calculating the distance between the gesture features and the geometric features of each gesture image in the gesture library, and taking the gesture GE in the standard library corresponding to the minimum distance between the geometric features as a gesture recognition result.
2. The infrared gesture recognition method in the projection interaction system according to claim 1, wherein the preprocessing the acquired gesture images by using a dynamic threshold method respectively comprises:
carrying out graying operation on the acquired gesture image to obtain a grayscale image;
establishing a rectangular coordinate system by taking the upper left corner of the gray level image as the origin of the coordinate system, and obtaining the position coordinates of the light source points in the rectangular coordinate system;
calculating Euclidean distance d between any point (x, y) in the gray level image and the light source point(x,y)And obtaining the farthest distance d between each point in the gray image and the point light source pointmax；
According to the maximum distance dmaxAnd the Euclidean distance d between each point and the light source point(x,y)And a set fixed threshold k, calculating a corresponding threshold k (x, y) at the point (x, y);
and carrying out thresholding operation on the gesture image by using a corresponding threshold value k (x, y) at the point (x, y) to obtain the preprocessed image.
3. The infrared gesture recognition method in the projection interaction system according to claim 1, wherein the preprocessing is performed on the collected gesture images by using an atrazine threshold method, which specifically includes:
setting different gray threshold values T, and comparing the gray value of each point pixel in the acquired gesture image with the different gray threshold values T;
image the gesture imageNumber N of pixels having gray-scale values smaller than threshold T0Number N of pixels having a pixel gradation larger than threshold value T1；
Obtaining the number N of pixels according to the size of the gesture image and the comparison under the same gray threshold value T0Number of pixels N1Calculating the inter-class variance under the gray threshold T as g;
traversing the inter-class variance values under different gray threshold values T, and performing thresholding operation on the gesture image by taking the gray threshold value T when the inter-class variance value reaches the maximum as a threshold value to obtain the preprocessed image.
4. The infrared gesture recognition method in the projection interaction system according to claim 1, wherein the preprocessing the acquired gesture images by using an adaptive threshold method respectively comprises:
setting a block with the size of b and a constant c for the gesture image, wherein b is an odd number;
acquiring the sum of gray values corresponding to all coordinate points (i, j) in the gesture image within the range of b multiplied by b, and recording the sum as g (i, j);
calculating a corresponding threshold value t (i, j) at the coordinate point (i, j) in the b × b range according to the gray value sum and the constant c;
and (3) carrying out binarization operation on the gesture image by using a corresponding threshold value t (i, j) at the coordinate point (i, j) to obtain the preprocessed image.
5. The infrared gesture recognition method in the projection interaction system according to claim 2 or 3, wherein after the collected gesture images are respectively preprocessed by using a dynamic threshold method, an Otsu threshold method and an adaptive threshold method to obtain different preprocessed images, the method further comprises:
and obtaining the size of the connected domain after background modeling or thresholding operation, and removing noise points.
6. The infrared gesture recognition method in the projection interaction system as claimed in claim 1, wherein the moment feature extraction is performed on the binarized image, specifically comprising:
obtaining a p + q order geometric moment m according to the gesture imagepqAnd central moment mupqWherein p + q is more than or equal to 2;
according to the central moment mupqObtaining a normalized center distance npq；
According to the central moment n of the second and third order scalespqObtaining u invariant moment groups;
and (3) selecting v moment groups from the u invariant moment groups to form invariant moment feature vectors, and using the invariant moment vectors as moment features, wherein v is more than or equal to 1 and less than or equal to u.
7. The infrared gesture recognition method in the projection interaction system according to claim 6, wherein the calculating of the distance between the gesture feature and the geometric feature of each gesture image in the gesture library and the taking of the gesture GE in the standard library corresponding to the minimum geometric feature distance as the gesture recognition result specifically includes:
calculating the distance D between the gesture features and the geometric features of the gesture images in the gesture librarymThe calculation formula is as follows:
wherein M isiIs the gesture image I(k)Geometric moment component of (G)iFor gesture image I in gesture library(l)Corresponding geometric moment component, ωiIs a component adjustment factor;
at a distance D of each geometric featuremFinds D inmAnd taking the gesture GE as a gesture recognition result when the gesture GE in the standard library corresponds to the minimum value.
8. An infrared gesture recognition device in a projection interaction system, comprising: the device comprises a preprocessing module, a feature extraction module and an identification module;
the preprocessing module is used for respectively preprocessing the acquired gesture images by adopting a dynamic threshold method, an Otsu threshold method and an adaptive threshold method to obtain different preprocessed images, and is used for:
(1) detecting the connected regions of the preprocessed image, and extracting all the connected regions in the preprocessed image;
(2) comparing the area of each communication area with a set communication area threshold value, and judging whether the communication area is connected or not;
(3) when the connected domains are judged not to be connected, finger area positioning is carried out according to the number of the connected domains;
(4) when the connected condition of the connected domains is judged, comparing the size of each connected domain with a set connected domain size threshold value, distinguishing the connected domains, and then executing the step (3);
the feature extraction module is used for carrying out binarization operation on different preprocessed images and carrying out image feature extraction on the obtained binarized images to obtain gesture features;
and the recognition module is used for calculating the distance between the gesture features and the geometric features of each gesture image in the gesture library, and taking the gesture GE in the standard library corresponding to the minimum geometric feature distance as a gesture recognition result.
9. An infrared gesture recognition device in a projection interaction system, comprising: a memory storing a plurality of program instructions, and a processor adapted to load the plurality of program instructions and perform:
respectively preprocessing the acquired gesture images by adopting a dynamic threshold method, an Otsu threshold method and an adaptive threshold method to obtain different preprocessed images, and using the preprocessed images to:
(1) detecting the connected regions of the preprocessed image, and extracting all the connected regions in the preprocessed image;
(2) comparing the area of each communication area with a set communication area threshold value, and judging whether the communication area is connected or not;
(3) when the connected domains are judged not to be connected, finger area positioning is carried out according to the number of the connected domains;
(4) when the connected condition of the connected domains is judged, comparing the size of each connected domain with a set connected domain size threshold value, distinguishing the connected domains, and then executing the step (3);
carrying out binarization operation on different preprocessed images, and carrying out image feature extraction on the obtained binarized image to obtain gesture features;
and calculating the distance between the gesture features and the geometric features of each gesture image in the gesture library, and taking the gesture GE in the standard library corresponding to the minimum distance between the geometric features as a gesture recognition result.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711498318.7A CN108255298B (en) | 2017-12-29 | 2017-12-29 | Infrared gesture recognition method and device in projection interaction system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711498318.7A CN108255298B (en) | 2017-12-29 | 2017-12-29 | Infrared gesture recognition method and device in projection interaction system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108255298A CN108255298A (en) | 2018-07-06 |
| CN108255298B true CN108255298B (en) | 2021-02-19 |
Family
ID=62725746
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201711498318.7A Active CN108255298B (en) | 2017-12-29 | 2017-12-29 | Infrared gesture recognition method and device in projection interaction system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108255298B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109634410A (en) * | 2018-11-28 | 2019-04-16 | 上海鹰觉科技有限公司 | Unmanned plane photographic method and system based on gesture identification |
| CN111161332A (en) * | 2019-12-30 | 2020-05-15 | 上海研境医疗科技有限公司 | Homologous pathology image registration preprocessing method, device, equipment and storage medium |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101853071A (en) * | 2010-05-13 | 2010-10-06 | 重庆大学 | Gesture identification method and system based on visual sense |
| CN103294996A (en) * | 2013-05-09 | 2013-09-11 | 电子科技大学 | 3D gesture recognition method |
| CN103679145A (en) * | 2013-12-06 | 2014-03-26 | 河海大学 | Automatic gesture recognition method |
| CN104331193A (en) * | 2014-12-01 | 2015-02-04 | 湖北印象光电信息产业有限公司 | Infrared precise multi-point touch platform |
| CN105117685A (en) * | 2015-07-27 | 2015-12-02 | 华南理工大学 | Finger wrist joint automatic positioning method based on image template matching |
| CN106097382A (en) * | 2016-05-30 | 2016-11-09 | 重庆大学 | A kind of tunnel based on discrete region scene environment illumination disturbance restraining method |
| CN106874906A (en) * | 2017-01-17 | 2017-06-20 | 腾讯科技（上海）有限公司 | A kind of binarization method of picture, device and terminal |
| CN107092353A (en) * | 2017-03-31 | 2017-08-25 | 东南大学 | A kind of collection of hand sense of touch parameter and simulation reduction system and method |
| CN107462520A (en) * | 2017-07-25 | 2017-12-12 | 杭州电子科技大学 | Towards stainless steel plate on-line measuring device of the confined space based on machine vision |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9894255B2 (en) * | 2013-06-17 | 2018-02-13 | Industrial Technology Research Institute | Method and system for depth selective segmentation of object |
-
2017
- 2017-12-29 CN CN201711498318.7A patent/CN108255298B/en active Active
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101853071A (en) * | 2010-05-13 | 2010-10-06 | 重庆大学 | Gesture identification method and system based on visual sense |
| CN103294996A (en) * | 2013-05-09 | 2013-09-11 | 电子科技大学 | 3D gesture recognition method |
| CN103679145A (en) * | 2013-12-06 | 2014-03-26 | 河海大学 | Automatic gesture recognition method |
| CN104331193A (en) * | 2014-12-01 | 2015-02-04 | 湖北印象光电信息产业有限公司 | Infrared precise multi-point touch platform |
| CN105117685A (en) * | 2015-07-27 | 2015-12-02 | 华南理工大学 | Finger wrist joint automatic positioning method based on image template matching |
| CN106097382A (en) * | 2016-05-30 | 2016-11-09 | 重庆大学 | A kind of tunnel based on discrete region scene environment illumination disturbance restraining method |
| CN106874906A (en) * | 2017-01-17 | 2017-06-20 | 腾讯科技（上海）有限公司 | A kind of binarization method of picture, device and terminal |
| CN107092353A (en) * | 2017-03-31 | 2017-08-25 | 东南大学 | A kind of collection of hand sense of touch parameter and simulation reduction system and method |
| CN107462520A (en) * | 2017-07-25 | 2017-12-12 | 杭州电子科技大学 | Towards stainless steel plate on-line measuring device of the confined space based on machine vision |
Non-Patent Citations (2)
| Title |
|---|
| 《红外多点触摸技术的关键问题研究》;吴娟;《中国优秀硕士学位论文全文数据库 信息科技辑》;20130415;I135-30 * |
| 罗娜.《 基于OpenCV的自然手势识别与交互系统研究》.《中国优秀硕士学位论文全文数据库 信息科技辑》.2012,I138-592. * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN108255298A (en) | 2018-07-06 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109359538B (en) | Training method of convolutional neural network, gesture recognition method, device and equipment | |
| CN108227912B (en) | Device control method and apparatus, electronic device, computer storage medium | |
| Nai et al. | Fast hand posture classification using depth features extracted from random line segments | |
| JPH10214346A (en) | Hand gesture recognizing system and its method | |
| Liu et al. | Rgbd video based human hand trajectory tracking and gesture recognition system | |
| WO2023165616A1 (en) | Method and system for detecting concealed backdoor of image model, storage medium, and terminal | |
| Sharma et al. | Air-swipe gesture recognition using OpenCV in Android devices | |
| CN108614988A (en) | A kind of motion gesture automatic recognition system under complex background | |
| CN108255298B (en) | Infrared gesture recognition method and device in projection interaction system | |
| CN111027637A (en) | Character detection method and computer readable storage medium | |
| AUGMENTED | Grayscale image enhancement for enhancing features detection in marker-less augmented reality technology | |
| Belhedi et al. | Adaptive scene‐text binarisation on images captured by smartphones | |
| Hoque et al. | Computer vision based gesture recognition for desktop object manipulation | |
| Mu et al. | Finding autofocus region in low contrast surveillance images using CNN-based saliency algorithm | |
| US20230410561A1 (en) | Method and apparatus for distinguishing different configuration states of an object based on an image representation of the object | |
| Simion et al. | Finger detection based on hand contour and colour information | |
| Reddy et al. | Hand gesture recognition using skeleton of hand and distance based metric | |
| Wang et al. | Skin Color Weighted Disparity Competition for Hand Segmentation from Stereo Camera. | |
| Yeh et al. | Vision-based virtual control mechanism via hand gesture recognition | |
| Dehankar et al. | Detecting centroid for hand gesture recognition using morphological computations | |
| Xu et al. | MultiView-based hand posture recognition method based on point cloud | |
| Gui et al. | Finger detection in video sequences using a new sparse representation | |
| Elsayed et al. | Hybrid method based on multi-feature descriptor for static sign language recognition | |
| Vishal et al. | Paper piano—shadow analysis based touch interaction | |
| KR101357581B1 (en) | A Method of Detecting Human Skin Region Utilizing Depth Information |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information |
Address after: 230000 Yafu Park, Juchao Economic Development Zone, Chaohu City, Hefei City, Anhui Province Applicant after: ANHUI HUISHI JINTONG TECHNOLOGY Co.,Ltd. Address before: 102, room 602, C District, Hefei National University, Mount Huangshan Road, 230000 Hefei Road, Anhui, China Applicant before: ANHUI HUISHI JINTONG TECHNOLOGY Co.,Ltd. |
|
| CB02 | Change of applicant information | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |