CN104866826A - Static gesture language identification method based on KNN algorithm and pixel ratio gradient features - Google Patents

Abstract

The invention discloses a static gesture language identification method based on a KNN algorithm and pixel ratio gradient features. The method comprises following steps: 1. shooting color images; 2. performing binaryzation on the images according to color features of the images; 3. positioning hands according to shape features of the images and segmenting the image; 4. normalizing the segmented images; 5. extracting pixel ratio gradient features of the images and using the extracting pixel ratio gradient features as feature vectors of the images; 6. calculating the Euclidean distance between the input feature vectors and the a standard gesture language image feature library; 7. determining optimal matching results based on the KNN algorithm; and 8. outputting the identification results. According to the invention, feature matching is carried out with the combination of the color features, shape features and pixel ratio gradient features of the images and by the use of the KNN algorithm, so that the identification rate is improved, the adaptability for different environments is enhanced, the algorithm is relatively simple, the complexity is low, the system operation speed is rapid, and the equipment is low in cost.

Description

A kind of based on KNN and the pixel static sign Language Recognition Method than Gradient Features

Technical field

The invention belongs to static Sign Language Recognition technical field, the static sign Language Recognition Method of particularly a kind of Iamge Segmentation, location, feature extraction and pattern-recognition.

Background technology

It is reported, China's hearing, language physical disabilities are more than 2,000 ten thousand people, and annual also with the speed increase of 20,000 ~ 30,000.To sign language,---the most important communation means of this kind of crowd---are studied, not only contribute to life, the study and work condition of improving these individuals with disabilities, for they provide better service, also can be applied to all many-sides such as stunt process, cartoon making, medical research, Entertainment in the teaching of area of computer aided sign language, the bilingual broadcasting of TV programme, visual human's research, film making simultaneously.

The development of computer vision technique relates to multidisciplinary field, and content is quite extensive, all relevant to optics, signal transacting, artificial intelligence, pattern-recognition, image procossing and computer technology etc.Along with signal processing theory, the theory of computation, theories of vision, etc. the development of correlation technique, computer vision have also been obtained and develops fast, and, robotization untouchable with it, visual, real-time and the feature such as intelligent are widely used in the every field such as commercial production, health care, Aero-Space, scientific research and national defense construction, are subject to the great attention of a lot of country and industry.

According to the difference of gesture import pathway, Sign Language Recognition can be divided into based on data glove with based on the large class of computer vision two.Based in the Sign Language Recognition of data glove, when human body makes various gesture motion, the positional information of each for hand joint part can be passed to computing machine to make subsequent treatment by position tracker by data glove more exactly.The data that this implementation gets are more accurate, and be applicable to the identification of a large amount of vocabulary, discrimination is also higher.But this method requires that user wears the data gathering tools such as thick and heavy several handgrip covers, and convenience is poor, and these instrumental values are expensive.Based in the Sign Language Recognition of computer vision, digital photographing apparatus is used to obtain images of gestures, utilize computer vision and Digital Image Processing correlation technique to process image, obtain gesture feature, the technology of last Land use models identification identifies sign language.The method can make man-machine interaction naturally convenient, and only use some cheap apparatus such as digital photographing apparatus, economic input is low simultaneously; But be limited to the current situation of computer vision and graphics, the visual data obtained by digital photographing apparatus is accurate not enough, cause subsequent characteristics to be extracted and detection-phase robustness unsecured.Thus the less identification being applied to a large amount of vocabulary.But based on the trend that the Sign Language Recognition of computer vision will be Sign Language Recognition technical development, along with the development of computer technology and imaging technique, along with the maturation of mode identification technology, Sign Language Recognition approach based on computer vision can overcome poor robustness and the inefficient problem of detecting and tracking, realize the Sign Language Recognition of the vision of highly artificial property intelligence.

Summary of the invention

The object of the invention is to the shortcoming overcoming above-mentioned prior art, provide that a kind of discrimination is high, strong robustness, cheap easily based on KNN and the pixel static sign Language Recognition Method than Gradient Features.The static sign language image input computing machine that picture pick-up device is taken by the method, processes rear identifiable design to image and goes out static sign language and be converted into text output.

The object of the invention is to be achieved through the following technical solutions.

For improving the recognition effect of sign language systems, the present invention proposes a kind of based on KNN and the pixel static sign Language Recognition Method than Gradient Features.The prerequisite of method is sign language person to put on blue gloves, and the method includes the steps of: step 1: color image shot also inputs computing machine; Step 2: the color characteristic based on image carries out binaryzation to image; Step 3: based on the position of the shape facility location hand of image, and split; Step 4: the image after segmentation is normalized; Step 5: the pixel extracting image than and Gradient Features as the proper vector of image; Step 6: calculate the proper vector of input and the weighted euclidean distance in standard sign language characteristics of image storehouse; Step 7: based on KNN algorithm determination best matching result; Step 8: recognition result is exported.

Further, described step 2 comprises following sub-step: step 21: the rgb value calculating each pixel of coloured image; Step 22: judge whether its color is blue by the rgb value of pixel; Step 23: all being composed by the RGB of the pixel being judged as blueness in step 22 is 255, it is 0 that the rgb value of other pixels is composed, and obtains binary image.

Further, described step 3 comprises following steps:

Step 31: binary image is expanded;

Step 32: all profiles in traversing graph picture, goes out contour area frame by a minimum rectangle;

Step 33: the length and width and the area that calculate all rectangular areas, judges whether this region is hand region, and judgment criterion is:

The length length surrounding the described minimum rectangle of profile meets: len_min<length<len_max,

The width height surrounding the minimum rectangle of profile meets: hgh_min<height<hgh_max, and the judgement meeting this criterion is hand region;

Step 34: will the contours segmentation in hand region be judged to be out.

Further optimally, len_min=150, len_max=250; Hgh_min=100, hgh_max=200.Image is unified into 20*36 size by the image normalization in described step 4.

Further, described step 5 comprises following steps:

Step S51: the normalized image after segmentation is divided into 16 little rectangles by 4*4, namely each little rectangle is of a size of 5*9;

Step S52: calculate the pixel number in each little rectangle, the number of white pixel point is stored in array an, and the number of total pixel is stored in array bn;

Step S53: computed segmentation goes out the pixel number on 6 equal separated times of described 16 little rectangles, the number of white pixel point is stored in array an, and the number of total pixel is stored in array bn;

Step S54: using an/bn as the pixel ratio of image and gradient eigenvector.

Further, when described step 6 calculates the Euclidean distance in proper vector and standard sign language characteristics of image storehouse, the weight strengthening Gradient Features be considered, suppose that the proper vector of the sign language image inputted is the proper vector of certain template in sign language characteristics of image storehouse is weighted euclidean distance computing formula then between them is as follows:

$d=\sqrt{\underset{i=1}{\overset{16}{\mathrm{\&Sigma;}}}{(a_i-b_i)}^2+\underset{i=17}{\overset{22}{\mathrm{\&Sigma;}}}{\left[9(a_i-b_i)\right]}^2}.$

Preferably, image is unified into 20*36 size by the image normalization in described step 4.

Compared with prior art, provided by the present invention based on KNN and the pixel static sign Language Recognition Method than Gradient Features, combine color of image feature, shape facility and pixel and compare Gradient Features, and make use of KNN algorithm to carry out characteristic matching, improve discrimination and the adaptability to varying environment, and algorithm is relatively simple, complexity is low, system running speed is fast, and equipment cost is cheap, has certain advance.

Accompanying drawing explanation

Fig. 1 is of the present invention based on KNN and the pixel process flow diagram than the static sign Language Recognition Method of Gradient Features;

Fig. 2 is the sub-process figure of Fig. 1 step S102;

Fig. 3 is the sub-process figure of Fig. 1 step S103;

Fig. 4 is the sub-process figure of Fig. 1 step S105.

Embodiment

Illustrate below in conjunction with accompanying drawing and specific embodiment of the invention is described in further detail; but specific embodiment of the invention and protection are not limited thereto; if have process or the symbol of not special detailed description it is noted that following, be all that those skilled in the art can refer to existing techniques in realizing.

As shown in Figure 1, be the Organization Chart of sign Language Recognition preferred embodiment of the present invention.Of the present invention based on KNN and pixel than the static sign Language Recognition Method of Gradient Features, comprise following steps: step S101: color image shot; Step S102: binaryzation is carried out to image; Step S103: locate the position of hand and split; Step S104: to the image normalization after segmentation; Step S105: the pixel ratio and the Gradient Features that extract image; Step S106: calculate the proper vector of input and the weighted euclidean distance in standard sign language characteristics of image storehouse; Step S107: determine best matching result based on KNN; Step S108: export recognition result.

In step S101, be sign language person to need to put on secondary blue gloves, based in the Gesture Recognition of computer vision, Hand Gesture Segmentation under complex background is very difficult, and this is mainly various due to background, and environmental factor is also unpredictable, not only there is no ripe theory as guidance, and existing method realizes difficulty, computation complexity is high, and effect neither be very desirable.Therefore the present invention adopts the method increasing restriction, and namely put on secondary blue gloves, this can reduce the difficulty of Hand Gesture Segmentation greatly, utilizes color detection, just can be easy to locating segmentation and go out gesture part.In addition, be applied in Wearable device conveniently, when the present invention makes a video recording, camera be placed on and do sign language person front and take forward.

As shown in Figure 2, being utilize color characteristic to carry out the process flow diagram of binaryzation to image, is the further refinement of step 102.Specifically comprise following steps: step S201: the rgb value calculating each pixel of coloured image; Step S202: judge whether its color is blue by the rgb value of pixel; Step S203: all being composed by the RGB of the pixel being judged as blueness in step S202 is 255, it is 0 that the rgb value of other pixels is composed, and obtains binary image.Judge that whether a pixel is that blue criterion is as follows:

(1) in R, G, B value of pixel s, the value of B is maximum;

(2) B value should be greater than certain threshold value B_th.Preferably, B_th is set to 50.

(3) because region is significantly blue, so the variance of R, G, B should be greater than certain threshold value Var_th.Preferably, Var_th is set to 350.

Wherein, the formula of the variance VAR of RGB is:

$\mathrm{Var}=\frac{{(R-\mathrm{average})}^2+{(G-\mathrm{average})}^2+{(B-\mathrm{average})}^2}{3}$

In formula,

$\mathrm{average}=\frac{R+G+B}{3}$

Worn blue gloves owing to being sign language person, after above-mentioned binarization, hand region major part pixel gray-scale value is 255, is white pixel point, is conducive to follow-up orientation and segmentation.

As shown in Figure 3, being the process flow diagram utilizing shape facility opponent region to position and split, is the further refinement to step S103.Specifically comprise following steps: step S301: binary image is once expanded, under the environment that light is more weak, plication region on blue gloves does not meet the judgment criterion in step S202, by binarized be black pixel point, thus cause the fracture of hand region contour, therefore need expansion plans picture to connect the region of fracture, promote follow-up location; Step S302: all profiles in traversing graph picture, goes out contour area frame by a minimum rectangle, calculates the length of rectangular area and wide, judges whether this region is hand region.Because camera during shooting is in the front of being sign language person, the size in hand region in a certain scope, must can go out the hand region in profile by this conditional filtering.Step S303: will the contours segmentation in hand region be judged to be out.

Concrete filter criteria is:

1. the length of surrounding the minimum rectangle of profile meets: len_min<length<len_max, preferably, and len_min=150, len_max=250

2. the width surrounding the minimum rectangle of profile meets: hgh_min<height<hgh_max, preferably, and hgh_min=100, hgh_max=200

After above step, the binary image of a static sign language image conversion in order to only comprise hand region.

Step S104 is normalized the image after segmentation, and the hand area size that each sign language is partitioned into is different, therefore needs image to unify size, and this is the accuracy in order to improve during the identification of subsequent extracted characteristic sum.Preferably, image is unified into 20*36 size by normalization.

As shown in Figure 4, being extract the pixel ratio of image and the process flow diagram of Gradient Features, is the further refinement to step S105.Detailed extraction step is: step S401: image is divided into 16 little rectangles by 4*4, and each rectangular dimension is 5*9; Step S402: calculate the pixel number in each little rectangle, the number of white pixel point is stored in array an, and the number of total pixel is stored in array bn; Step S403: computed segmentation goes out the pixel number on 6 equal separated times of described 16 little rectangles, the number of white pixel point is stored in array an, and the number of total pixel is stored in array bn; Step S404: using an/bn as the pixel ratio of image and gradient eigenvector.So far, we obtain the proper vector of one 22 dimension, and the 1st dimension is to the 16th dimension for pixel is than feature, and the 17th dimension is Gradient Features to the 22nd dimension.This feature comparatively directly and the pixel distribution situation embodied meticulously in binary image, is very helpful with mating to follow-up pattern-recognition.In addition, be the ratio of pixel number but not simple pixel number due to what utilize here, this feature has good adaptability to different scene images.

Step S106 calculates the proper vector of input and the characteristic Euclidean distance in standard sign language characteristics of image storehouse, and Euclidean distance and euclidean metric are the most frequently used amounts for the difference between characterization data.The difference that this step obtains characterizes a pith to template matches, has vital effect to follow-up identification.Consider that the pixel number on equal separated time is less, when characterizing distance, the proportion that accounts for is very little, therefore needs for Gradient Features adds certain weight, and after debugging, weight is elected as 9 better.

Suppose that the proper vector of the sign language image inputted is the proper vector of certain template in sign language characteristics of image storehouse is weighted euclidean distance computing formula then between them is as follows:

$d=\sqrt{\underset{i=1}{\overset{16}{\mathrm{\&Sigma;}}}{(a_i-b_i)}^2+\underset{i=17}{\overset{22}{\mathrm{\&Sigma;}}}{\left[9(a_i-b_i)\right]}^2}$

Step S107 determines best matching result based on KNN algorithm.KNN algorithm, i.e. K arest neighbors sorting algorithm, the core concept of KNN algorithm is that then this sample also belongs to this classification, and has the characteristic of sample in this classification if the great majority in the k of sample in a feature space the most adjacent individual sample belong to some classifications.Being applied as in the present invention: the weighted euclidean distance sequence calculated in step S106 is sorted, find out minimum 20, corresponding 20 templates respectively, each template certain manual alphabet corresponding, just obtain 20 letters thus, get the maximum letter of wherein occurrence number as the result identified.

Step S108 exports recognition result with the form of word.

Claims (6)

1. based on KNN and pixel than a static sign Language Recognition Method for Gradient Features, it is characterized in that, comprise the following steps:

Step 1: color image shot also inputs computing machine;

Step 2: the color characteristic based on image carries out binaryzation to image;

Step 3: based on the position of the shape facility location hand of image, and split;

Step 4: the image after segmentation is normalized;

Step 5: the pixel extracting image than and Gradient Features as the proper vector of image;

Step 6: calculate the proper vector of input and the Euclidean distance in standard sign language characteristics of image storehouse;

Step 7: based on KNN algorithm determination best matching result;

Step 8: recognition result is exported.

2. as claimed in claim 1 based on KNN and pixel than the static sign Language Recognition Method of Gradient Features, it is characterized in that, described step 3 comprises following steps:

Step 31: binary image is expanded;

Step 32: all profiles in traversing graph picture, goes out contour area frame by a minimum rectangle;

Step 33: the length and width and the area that calculate all rectangular areas, judges whether this region is hand region, and judgment criterion is:

The length length surrounding the described minimum rectangle of profile meets: len_min<length<len_max,

The width height surrounding the minimum rectangle of profile meets: hgh_min<height<hgh_max, and the judgement meeting this criterion is hand region;

Step 34: will the contours segmentation in hand region be judged to be out.

3. as claimed in claim 1 based on KNN and pixel than the static sign Language Recognition Method of Gradient Features, it is characterized in that:

len_min＝150,len_max＝250；hgh_min＝100,hgh_max＝200。

4. as claimed in claim 1 based on KNN and pixel than the static sign Language Recognition Method of Gradient Features, it is characterized in that: image is unified into 20*36 size by the image normalization in described step 4.

5. as claimed in claim 1 based on KNN and pixel than the static sign Language Recognition Method of Gradient Features, it is characterized in that, described step 5 comprises following steps:

Step S51: the normalized image after segmentation is divided into 16 little rectangles by 4*4, and namely each little rectangle is of a size of 5*9;

Step S52: calculate the pixel number in each little rectangle, the number of white pixel point is stored in array an, and the number of total pixel is stored in array bn;

Step S53: computed segmentation goes out the pixel number on 6 equal separated times of described 16 little rectangles, the number of white pixel point is stored in array an, and the number of total pixel is stored in array bn;

Step S54: using an/bn as the pixel ratio of image and gradient eigenvector.

6. as claimed in claim 1 based on KNN and the pixel static sign Language Recognition Method than Gradient Features, it is characterized in that, when described step 6 calculates the Euclidean distance in proper vector and standard sign language characteristics of image storehouse, consider the weight strengthening Gradient Features, suppose that the proper vector of the sign language image inputted is the proper vector of the template in standard sign language characteristics of image storehouse is weighted euclidean distance computing formula then between them is as follows:

$d=\sqrt{\underset{i=1}{\overset{16}{\mathrm{\&Sigma;}}}{(a_i-b_i)}^2+\underset{i=17}{\overset{22}{\mathrm{\&Sigma;}}}{\left[9(a_i-b_i)\right]}^2}.$