CN103136986A

CN103136986A - Sign language identification method and sign language identification system

Info

Publication number: CN103136986A
Application number: CN2012100315958A
Authority: CN
Inventors: 雷敏娟; 周雷; 师丹玮
Original assignee: SHENZHEN TOL TECHNOLOGY Co Ltd
Current assignee: Shenzhen Taishan Sports Technology Co.,Ltd.
Priority date: 2011-12-02
Filing date: 2012-02-13
Publication date: 2013-06-05
Anticipated expiration: 2032-02-13
Also published as: CN103135758B; CN103135754A; CN103139508A; CN103127717B; CN103135882B; CN103135755A; CN103135453A; CN103135883A; CN103135883B; CN103135759B; CN103135754B; CN103136986B; CN103135453B; CN103135755B; CN103135758A; CN103135756B; CN103135756A; CN103135759A; CN103135881A; CN103135882A

Abstract

The invention relates to a sign language identification method which comprises the steps of collecting images comprising marked areas; identifying gestures of the marked areas; generating control commands corresponding to the gestures; and converting the control commands to natural language information. The invention further discloses a sign language identification system. The sign language identification method and the sign language identification system are capable of improving the accuracy rate of identification.

Description

Sign Language Recognition Method and system

[technical field]

The present invention relates to artificial intelligence field, particularly a kind of sign Language Recognition Method and system.

[background technology]

In daily life, the deaf-mute is due to the listening and speaking ability of having lost the normal person, with other people communication in, usually adopt sign language to exchange.The action that the deaf-mute makes standard by limbs represents corresponding semanteme.But do not understand the implication of sign language due to most of normal persons, so the deaf-mute can only exchange with other deaf-mutes by sign language usually, and still have obstacle with exchanging of normal person.

In conventional art, for the deaf-mute can be exchanged with the normal person, usually make the deaf-mute put on the data glove with a plurality of sensors, and gather movement track and the orientation of deaf-mute's limbs by data glove, and generate the text message with semanteme according to the movement track that gets and orientation, thereby converted sign language to normal person more intelligible natural language information.

Yet in above-mentioned sign Language Recognition Method, the data glove small volume causes the negligible amounts of the sensor on data glove.Therefore, data glove is when the movement track that gathers limbs and orientation, and the data that obtain are not accurate enough, make the error rate of the identification error of sign Language Recognition Method in conventional art larger.

[summary of the invention]

Based on this, be necessary to provide a kind of sign Language Recognition Method that can improve accuracy rate.

A kind of sign Language Recognition Method comprises the following steps:

Collection comprises the image of marked region;

The attitude in identification marking zone;

Generate steering order corresponding to described attitude;

Convert described steering order to natural language information.

Preferably, the described step that converts described steering order to natural language information comprises:

Described steering order is arranged in the steering order sequence, generates natural language information according to described steering order sequence.

Preferably, described step according to described steering order sequence generation natural language information comprises:

Calculate the eigenwert of described steering order sequence, generate natural language information according to described default eigenwert and the mapping relations of natural language information.

Preferably, also comprise before the step of the eigenwert of the described steering order sequence of described calculating:

The steering order that repeats in described steering order sequence is removed.

To remove greater than the steering order of threshold value with adjacent steering order difference in described steering order sequence.

Preferably, the described step that described steering order is arranged in the steering order sequence is specially:

With described steering order according to the one-tenth formation arranged sequentially that generates, when the formation end being detected the identical steering order of continuously default number arranged, arranges complete, and according to described formation generation steering order sequence.

Preferably, describedly also comprise after converting described steering order the step of natural language information to:

Show described natural language information by the mode of text and/or audio frequency.

In addition, also be necessary to provide a kind of sign Language Recognition that can improve accuracy rate.

A kind of sign Language Recognition comprises with lower module:.

Image capture module is used for gathering the image that comprises marked region;

The gesture recognition module is for the attitude in identification marking zone;

The instruction generation module is used for generating steering order corresponding to described attitude;

The instruction transformation module is used for converting described steering order to natural language information.

Preferably, described instruction transformation module also is used for described steering order is arranged in the steering order sequence, generates natural language information according to described steering order sequence.

Preferably, described instruction transformation module also is used for calculating the eigenwert of described steering order sequence, generates natural language information according to described default eigenwert and the mapping relations of natural language information.

Preferably, described instruction transformation module is also removed for the steering order that described steering order sequence is repeated.

Preferably, described instruction transformation module also is used for described steering order sequence and adjacent steering order difference are removed greater than the steering order of threshold value.

Preferably, described instruction transformation module also is used for described steering order according to the formation arranged sequentially that generates, when the formation end being detected the identical steering order of continuously default number arranged, arranges complete, and according to described formation generation steering order sequence.

Preferably, also comprise the information display module, be used for showing described natural language information by the mode of text and/or audio frequency.

Above-mentioned sign Language Recognition Method and system go out according to the image recognition that comprises marked region that collects the attitude that marked region produces, and generate steering order corresponding to attitude, then convert this steering order to normal person and hold intelligible natural language information.Owing to judging movement locus and the attitude of limb action by the image that obtains limb action, therefore, process to whole limb action all has record, thereby has avoided missing in default of sensor the situation of certain action or attitude, thus the accuracy rate when having improved the identification sign language.

[description of drawings]

Fig. 1 controls the schematic flow sheet of the method for browsing page in the present invention;

Fig. 2 is the schematic flow sheet of step S20 in an embodiment;

Fig. 3 is the structural representation of interactive device in an embodiment;

Fig. 4 builds the schematic diagram of coordinate system in an embodiment;

Fig. 5 is the structural representation of the interactive device in another embodiment;

Fig. 6 is the structural representation of the interactive device in another embodiment;

Fig. 7 is the schematic flow sheet of step S20 in another embodiment;

Fig. 8 builds the schematic diagram of coordinate system in another embodiment;

Fig. 9 is the schematic flow sheet of step S30 in an embodiment;

Figure 10 is the schematic flow sheet of step S30 in another embodiment;

Figure 11 is the structural representation of sign Language Recognition system in an embodiment;

Figure 12 is the structural representation of gesture recognition module in an embodiment;

Figure 13 is the structural representation of gesture recognition module in another embodiment;

Figure 14 is the structural representation of instruction generation module in an embodiment;

Figure 15 is the structural representation of instruction generation module in another embodiment;

Figure 16 is the structural representation of sign Language Recognition in another embodiment.

[embodiment]

Below in conjunction with specific embodiment and accompanying drawing, technical scheme is described in detail.

In one embodiment, as shown in Figure 1, a kind of sign Language Recognition Method comprises the following steps:

Step S10 gathers the image that comprises marked region.

In the present embodiment, marked region is a zone in the image that gathers, and this zone can be formed by interactive device.

Concrete, in one embodiment, interactive device can be hand-held device, part or all of hand-held device can be set as color or the shape of appointment, gather the image of hand-held device, this designated color in the hand-held device in image or the part of shape form marked region.In addition, interactive device can also be the hand-held device of tape label, namely attach the mark (as reflectorized material) of designated color or shape on hand-held device, gather the image of hand-held device, on the hand-held device in image, the mark of incidental designated color or shape forms marked region.

In another embodiment, interactive device can also be human body (such as people's face, palm, arm etc.), gathers the image of human body, and the human body in image forms marked region.In addition, interactive device can also be the human body of tape label, namely attaches the mark (as reflectorized material) of designated color or shape on human body, and when gathering the image of human body, this designated color in image or the mark of shape form marked region.

Step S20, the attitude in identification marking zone.

Concrete, the image that collects is processed, extract the marked region in image, then produce the attitude of marked region according to the pixel coordinate of the pixel in marked region in the image coordinate system that builds.So-called attitude refers to marked region formed posture state in image.Further, in two dimensional image, attitude is marked region and the angle between predeterminated position, the i.e. attitude angle in two dimensional image; In 3-D view, attitude is the vector that marked region in two dimensional image and a plurality of attitude angle between predeterminated position form, i.e. the attitude vector." attitude that marked region produces " said in the present invention, " attitude of marked region ", " attitude " all refer to described attitude, namely the attitude angle of different embodiment and attitude vector.

Step S30 generates steering order corresponding to attitude.

In the present embodiment, preset the attitude of marked region and the mapping relations between steering order, and these mapping relations are stored in database.After identifying the attitude of marked region, can search the steering order corresponding with attitude from database according to the attitude that identifies.

Step S40 converts steering order to natural language information.

Natural language information is that the normal person holds intelligible language message, as Chinese, English, Latin etc.Can set in advance the mapping table of steering order and natural language information, and be stored in database, then obtain the natural language information corresponding with it by inquiry steering order in database.

For example, the mapping table of default steering order and natural language information can be as shown in table 1:

Table 1

Steering order	Natural language information
		command_cir	Circle
command_heart	Like
		command_ok	Alright
......	......

With the finger of human body as interactive device, when user's finger has surrounded a circle, the corresponding circular steering order command_cir of expression that generated, then inquiry obtains corresponding natural language information and is " circle " in database.When user's finger surrounds one when heart-shaped, the corresponding steering order command_heart that generates, then inquiry obtains corresponding natural language information and is " love " in database.Be connected between user's forefinger and thumb, when other three fingers launch to make the OK shape, the corresponding steering order command_ok that generates, then inquiry can get corresponding natural language information and is " good " in database.

In one embodiment, steering order can be arranged in the steering order sequence, generate natural language information according to the steering order sequence.

First by generating a plurality of steering orders every default sampling interval T execution in step S10, step S20, step S30.Again with the steering order that generates according to the one-tenth steering order sequence arranged sequentially that generates, and generate natural language information according to the steering order sequence.

For example, when adopting user's finger and arm as interactive device, caught user's finger and the attitude of arm every 0.1 second, and generate steering order and be used for this attitude constantly of sign.After having generated within a certain period of time a plurality of steering orders, steering order has represented user's finger and the running orbit of arm according to the sequence that the order that generates forms.

Further, can calculate the eigenwert of steering order sequence, according to the mapping relations generation natural language information of default eigenwert and natural language information.

Comprise in steering order for the attitude part of expression attitude information with for the coordinate part of this attitude of expression at the coordinate information of image.When with finger and arm during as interactive device, the attitude of steering order part can represent to point shape that the attitude with arm consists of or abstract vector graphics, launches shape, points the ring-type that surrounds as the five fingers.The coordinate part of steering order has represented the position of shape on image that finger or arm surround.When image was two dimensional image, coordinate was two-dimensional coordinate.When image was 3-D view, coordinate was three-dimensional coordinate.

Eigenwert is used for the common feature of steering order sequence that expression has certain similarity.Can adopt the variation characteristic of the attitude part in posture feature value representation steering order sequence, adopt coordinate characteristic value to represent the variation characteristic of the coordinate part in the steering order sequence.Then by the eigenwert of posture feature value and coordinate characteristic value composition control instruction sequence.

Eigenwert and natural language information have default mapping relations.For example, can in advance the corresponding natural language of eigenwert 2 (1F@1L) " be thanks ".This eigenwert 2 (1F@1L) is based on the variation of the attitude part of steering order sequence, expression is stretched (being represented by the 1F in eigenwert) by thumb and is become thumb bending (being represented by the 1L in eigenwert), and crooked twice (multiply by 2 with bracket represents).Can in advance the corresponding natural language of eigenwert 5F_y-down " be pressed ".This eigenwert 5F_y-down is based on the variation of the part of the coordinate in the steering order sequence, and expression palm (being represented by the 5F in eigenwert) is pressed (being represented by the y-down in eigenwert) from top to bottom.Further, first the steering order that repeats in the steering order sequence is removed, and then calculated the eigenwert of steering order sequence.Remove the steering order that repeats and to reduce calculated amount.

Further, first will remove greater than the steering order of threshold value with adjacent steering order difference in the steering order sequence, and then calculate the eigenwert of steering order sequence.When default sampling interval hour, if the attitude of certain steering order in steering order sequence part or coordinate part during greater than threshold value, judge that this steering order is the steering order of mistake with adjacent steering order difference.False command will be filtered out the steering order sequence, and can not be used to calculate the eigenwert of steering order sequence.

For example, when the user adopts finger as input media, if the steering order sequence that gets is: [3F (0,0), 3F (1,0), 3F (2,0), 2F (3,0), 3F (4,0), 3F (5,0)].Wherein attitude part 3F represents the attitude of 3 fingers, and 2F represents the attitude of 2 fingers, and coordinate part (0,0)-(5,0) expression finger gesture is in the coordinate in the image of catching.This steering order sequence is used for three tracks that the finger horizontal translation forms of expression user, and each steering order is the sampled point of this track.Wherein, may be due to the user in moving process, the situation that has finger to close up, steering order 2F (3,0) 3F (2,0), the 3F (4,0) with adjacent is larger in the upper gap of the number (the attitude part of steering order) of finger, therefore be judged as wrong steering order, and be moved out of the steering order sequence.When certain steering order and adjacent steering order have than big difference, normally because the nonstandard sign language gesture of having caught the user, or because interactive device is blocked when mobile, can not obtain its shape fully and cause.For example, in deaf and dumb sign language, some gesture needs two hands alternately mobile, in movement, and the situation that may occur blocking, therefore remove and the steering order of adjacent steering order gap greater than threshold value (shape different or coordinate distance greater than threshold value), can be more accurate so that eigenwert is calculated.

When calculating the eigenwert of steering order sequence, first calculate steering order in the steering order sequence the posture feature value, then according to the posture feature value with the segmentation of steering order sequence, the steering order in the steering order subsequence after segmentation has identical posture feature value.The monotonicity of the coordinate of then partly describing according to the coordinate of the steering order in the steering order subsequence, the variance of the distance of a certain reference coordinate is calculated the coordinate characteristic value of steering order subsequence in the image.Then the eigenwert that posture feature value and the coordinate characteristic value of all steering order subsequences is integrated into the steering order sequence.

For example, take finger as interactive device, the user makes following action:

Stretch out one and point translation from left to right, move on to and stretch the five fingers after certain position and do the quadrant motion, fix at last, and crooked forefinger and thumb, forefinger and thumb finger tip are collided forms OK shape gesture.

The steering order sequence that gets is: [1F (0,0), 1F (1,0), 1F (2,0), 1F (3,0), 5F (2.5,1.5), 5F (1.5,2.5), 5F (0,3), OK (0,3), OK (0,3), OK (0,3)].Wherein 1F is the posture feature value of 1 finger, and 5F is the posture feature value of 5 fingers, and OK represents the posture feature value of OK shape gesture, and bracket inner digital represents the coordinate figure of attitude in image.Remove repetition steering order and with the adjacent steering order of steering order difference greater than threshold value after, then according to the difference of posture feature value, the steering order sequence is divided into three sub-steering order sequences:

Subsequence 1:[1F (0,0), 1F (1,0), 1F (2,0), 1F (3,0)].

Subsequence 2:[5F (2.5,1.5), 5F (1.5,2.5), 5F (0,3)].

Subsequence 3:[OK (0,3)].

The coordinate characteristic value of subsequence 1 is x-right, and x-right represents that the monotonicity of the coordinate of subsequence 1 is that horizontal ordinate increases progressively (x axle), ordinate constant (y axle).The coordinate characteristic value of subsequence 2 is q-cir, q-cir represent track that the coordinate of subsequence 2 consists of to the variance of reference point coordinate (0,0) less than threshold value, namely track is 1/4th circles to the center of circle (0,0).The coordinate characteristic value of subsequence 3 is hold, namely represents the last attitude that fixes.

Then posture feature value and the coordinate characteristic value of subsequence 1, subsequence 2 and subsequence 3 are integrated, the eigenwert of controlled instruction sequence is: 1F_x-right@5F_q-cir@OK_hold.Wherein _ and represent separator with@, be used for distinguishing posture feature value and coordinate characteristic value.

After the eigenwert of controlled instruction sequence, then obtain the natural language corresponding with the eigenwert of steering order sequence according to default eigenwert with the mapping relations of natural language.As above in example, natural language corresponding to 1F_x-right@5F_q-cir@OK_hold is " perfection ", the user makes stretches out one and points translation from left to right, after moving on to certain position, the stretching, extension the five fingers are done the quadrant motion, fix at last, and crooked forefinger and thumb namely have been converted into natural language " perfection " with the sign language that forms OK shape gesture of colliding of forefinger and thumb finger tip.

Need to prove, sign language herein is not limited to the deaf and dumb sign language of standard, can be also user-defined sign language.The corresponding relation of sign language and natural language depends on default eigenwert and the mapping relations of natural language.

Further, when steering order is arranged in the steering order sequence, can be with steering order according to the one-tenth formation arranged sequentially that generates, when the formation end being detected the identical steering order of continuously default number arranged, arrange completely, and generate the steering order sequence according to formation.

Set in advance buffer area, this steering order is stored in buffer area in order whenever generating a steering order, form formation.Simultaneously, detect this buffer area, if when there is the identical steering order of continuously default number at the formation end, the as above steering order OK in example (0,3), formation is arranged complete, then takes out this formation formation control instruction sequence from buffer area.

Intercept the steering order sequence by the continuous identical steering order of default number, make when distinguishing a plurality of sign language gestures (pause between similar English word) convenient, thereby the wrong identification when having avoided many gestures.

In one embodiment, after converting steering order to natural language information, show natural language information by the mode of text and/or audio frequency.

Can natural language information be showed with the form of word by display screen, also can natural language information be played back by audio frequency apparatus.

For example, after steering order is changed into natural language information, natural language information is encoded, and send out this coding to remote terminal (mobile phone, computer, TV etc.), remote terminal first becomes natural language information with this coding and decoding after receiving this coding, then shows this natural language information or plays this natural language information by audio frequency apparatus with the form of voice by the form of display screen with text.

Mode by text and/or audio frequency is showed natural language information, can facilitate other people to understand natural language information.Send to remote terminal with natural language information coding and by telephone network or internet or television network, and then show this natural language information by remote terminal by decoding, make the deaf-mute to realize distance communicating with the normal person who is ignorant of sign language, thereby facilitated deaf-mute and normal person's communication.

As shown in Figure 2, in one embodiment, the image that comprises marked region that collects is two dimensional image, and the detailed process of above-mentioned steps S20 comprises:

Step S202 extracts the pixel of mating with default color model in image, the pixel of obtaining is carried out connected domain detect, and extracts the marked region that detects in the connected domain that obtains.

Concrete, can comprise by camera acquisition the image of marked region, the image that obtains is the two-dimensional visible light image.Preferably, also can add infrared fileter before the camera lens of video camera, be used for elimination except other wave band light of infrared band, the image that gathers is the two-dimensional infrared image.In visible images, the object in scene can form the identification of marked region and disturb, and infrared image has been because having filtered out visible light information, disturbs lessly, so the two-dimensional infrared image more is conducive to extract marked region.

In the present embodiment, set up in advance color model.For example the color of marked region is red, sets up in advance red model, and in this model, the rgb value component of pixel can be between 200 to 255, and G, B component can be close to zero; Obtain the pixel that satisfies the rgb value of this redness model in the image of collection and be red pixel.In addition, when forming marked region by human body in the image that gathers, can obtain the pixel of mating with default complexion model in the image of collection.The pixel of obtaining is carried out connected domain detect, obtain a plurality of connected domains, if connected domain is the set that individual continuous pixel forms.

In the present embodiment, because the size and shape of marked region should be roughly changeless, the pixel of obtaining is being carried out connected domain when detecting, can calculate girth and/or the area of all connected domains in the pixel of obtaining.Concrete, the girth of connected domain can be the number of connected domain boundary pixel, and the area of connected domain can be the number of the whole pixels in connected domain.Further, the girth of the connected domain obtained and/or girth and/or the area of area and default marked region can be compared, obtain the girth that satisfies default marked region and/or the connected domain of area and be marked region.Preferably, also can with girth square with the ratio of area as judgment criterion, this ratio of connected domain satisfies this ratio of default marked region, this connected domain is marked region.

Step S204 obtains the pixel coordinate in marked region, produces the marked region attitude according to this pixel coordinate.

Concrete, in one embodiment, as shown in Figure 3, interactive device comprises portion of the handle and the mark that is attached to portion of the handle, wherein, mark can be the reflectorized material of elongate in shape, and is preferred, can be ellipse or rectangular shape.In other embodiments, interactive device also can be human body, and as face, palm, arm etc., the marked region in the image that collects is the zone of human body.

In the present embodiment, marked region is a continuum, the process that produces the attitude of marked region according to pixel coordinate is: the covariance matrix that calculates pixel coordinate, obtain covariance matrix eigenvalue of maximum characteristic of correspondence vector, produce the attitude of marked region according to proper vector, the attitude of this marked region is an attitude angle.

Concrete, as shown in Figure 4, build the two dimensional image coordinate system, for two the some A (u1, v1) on this coordinate system and B (u2, v2), the attitude angle of its formation is the arc tangent of slope, i.e. arctan ((v2-v1)/(u2-u1)).Concrete, in the present embodiment, calculate the covariance matrix of the pixel coordinate in the marked region that extracts, obtain covariance matrix eigenvalue of maximum characteristic of correspondence vector, the direction of this proper vector is the direction of marked region major axis place straight line.As shown in Figure 4, marked region major axis place rectilinear direction is the direction of A, 2 place straight lines of B, establishes proper vector and is [dir_u, dir_v] ^T, wherein, the projection of direction on the u axle of the regional major axis of dir_u descriptive markup, its absolute value is proportional to the projection (be u2-u1) of vector on the u change in coordinate axis direction of pointing to B from A; The projection of direction on the v axle of dir_v descriptive markup zone major axis, its absolute value is proportional to the projection (be v2-v1) of vector on the v change in coordinate axis direction of pointing to B from A.If dir_u or dir_v less than 0, are modified to [dir_u ,-dir_v] ^T, the attitude angle of marked region is: arctan (dir_v/dir_u).

In another embodiment, marked region comprises the first continuum and the second continuum, the detailed process that produces the attitude of marked region according to described pixel coordinate is: calculate the center of gravity of the first continuum and the center of gravity of the second continuum according to pixel coordinate, produce the attitude of marked region according to the pixel coordinate of the center of gravity of the pixel coordinate of the center of gravity of the first continuum and the second continuum.Concrete, in one embodiment, interactive device comprises portion of the handle and is attached to two marks of portion of the handle.As shown in Figure 5, be labeled as two, be attached to respectively the portion of the handle front end, the shape of mark can be ellipse or rectangle.Preferably, mark can be for being positioned at two round dots of handgrip part front end.As shown in Figure 6, mark can be arranged on the two ends of portion of the handle.In other embodiments, also mark can be arranged on human body, for example be arranged on people's face, palm or arm.Should be noted that two set marks can size, inconsistent on the feature such as shape, color.

In the present embodiment, the marked region of extraction comprises two continuums, is respectively the first continuum and the second continuum.Further, calculate the center of gravity of these two continuums according to pixel coordinate.Concrete, calculate the mean value of the whole pixel coordinates in the continuum, resulting pixel coordinate is the center of gravity of continuum.As shown in Figure 4, the center of gravity of two continuums that calculate is respectively A (u1, v1) and B (u2, v2), and the attitude angle of marked region is the arc tangent of slope, i.e. arctan ((v2-v1)/(u2-u1)).

In another embodiment, the image that gathers can be 3-D view.Concrete, can utilize traditional stereo visual system (being formed by two known video camera and Correlation method for data processing equipment in locus), structured-light system (a right video camera, light source and Correlation method for data processing equipment form) or TOF (time of flight, flight time) depth camera collection 3-D view (being the three dimensional depth image).

In the present embodiment, as shown in Figure 7, the detailed process of step S20 comprises:

Step S210 to Image Segmentation Using, extracts the connected domain in this image, calculates the property value of connected domain, and the property value of connected domain and default marked region property value are compared, and this marked region is the connected domain that meets this default marked region property value.

Concrete, when in the three dimensional depth image, two adjacent pixel depths differ less than predefined threshold value, for example 5 centimetres, think that two pixels are communicated with, whole image is carried out connected domain detect, can obtain comprising a series of connected domains of mark connected domain.

In the present embodiment, the property value of connected domain comprises the size and dimension of connected domain.Concrete, calculate the size/shape of connected domain, compare with the size/shape of mark on interactive device, the connected domain that obtains meeting the size/shape of mark is the connected domain (marked region) of marked region.Take rectangle marked as example, be to be rectangle in the image that is marked at collection on interactive device, the length of pre-set mark and width, calculate length and the width of physical region corresponding to connected domain, length and the width of this length and width and mark are more approaching, and connected domain is more similar to marked region.

Further, the length of the physical region that the calculating connected domain is corresponding and the process of width are as follows: calculate the covariance matrix of the three-dimensional coordinate of connected domain pixel, adopt following formula to calculate length and the width of physical region corresponding to connected domain:

Wherein, k is predefined coefficient, for example is made as 4, and when λ was the covariance matrix eigenvalue of maximum, l was the length of connected domain, and when λ was the second largest eigenwert of covariance matrix, l was the width of connected domain.

Further, also can preset the length breadth ratio of rectangle marked, for example length breadth ratio is 2, the length breadth ratio of physical region corresponding to connected domain is more close to the length breadth ratio of the rectangle marked of default settings, connected domain is more similar to marked region, concrete, adopt following formula to calculate the length breadth ratio of physical region corresponding to connected domain:

Wherein, r is the length breadth ratio of connected domain, λ ₀Be the eigenvalue of maximum of covariance matrix, λ ₁Second Largest Eigenvalue for covariance matrix.

Step S220 obtains the pixel coordinate in marked region, produces the attitude of marked region according to this pixel coordinate.

Concrete, in the present embodiment, the attitude of marked region is the attitude vector.As shown in Figure 8, build the 3-D view coordinate system, this coordinate is right-handed coordinate system.In this coordinate system, establish space vector OP, P is at the p that is projected as of plane X OY, and the attitude vector with polar coordinate representation vector OP is [α, θ] ^T, α is angle XOp, and namely X-axis is to the Op angle, and span is 0 to 360 degree, and θ is angle pOP, i.e. the angle of OP and XOY plane, span be-90 to spend to 90 and spend.If 2 on the space ray in this coordinate system is A (x1, y1, z1) and B (x2, y2, z2), this attitude of 2 vector [α, θ] ^TAvailable following formula is unique to be determined:

\cos (α) = \frac{x 2 - x 1}{\sqrt{{(x 2 - x 1)}^{2} + {(y 2 - y 1)}^{2}}}

\sin (α) = \frac{y 2 - y 1}{\sqrt{{(x 2 - x 1)}^{2} + {(y 2 - y 1)}^{2}}} - - - (1)

θ = \arctan (\frac{z 2 - z 1}{\sqrt{{(x 2 - x 1)}^{2} + {(y 2 - y 1)}^{2}}}) - - - (2)

In the present embodiment, after extracting marked region, calculate the covariance matrix of the pixel coordinate in marked region, obtain covariance matrix eigenvalue of maximum characteristic of correspondence vector, and this proper vector is converted to the attitude vector.Concrete, establish the attitude vector that obtains and be [dir _x, dir _y, dir _z] ^T, wherein, dir _xRepresent 2 distances on the x direction of principal axis, dir _yRepresent 2 distances on the y direction of principal axis, dir _zRepresent 2 distances on the z direction of principal axis.Can think has two points on the ray of this attitude vector description, namely (0,0,0) and

Be that ray triggers from initial point, point to (dir _x, dir _y, dir _z), attitude angle need satisfy above-mentioned formula (1) and (2), makes the x1=0 in above-mentioned formula (1) and (2), y1=0, z1=0, x2=dir _x, y2=dir _y, z2=dir _z, can obtain attitude vector [α, θ] ^T

In one embodiment, marked region is a continuum, the process that produces the attitude of marked region according to pixel coordinate is: the covariance matrix that calculates pixel coordinate, obtain covariance matrix eigenvalue of maximum characteristic of correspondence vector, produce the attitude of marked region according to proper vector.As mentioned above, the attitude of this marked region is an attitude vector.

In another embodiment, marked region comprises the first continuum and the second continuum, the detailed process that produces the attitude of marked region according to described pixel coordinate is: calculate the center of gravity of the first continuum and the center of gravity of the second continuum according to pixel coordinate, calculate the attitude of marked region according to the pixel coordinate of the center of gravity of the pixel coordinate of the center of gravity of the first continuum and the second continuum.As shown in Figure 8, in the present embodiment, the pixel coordinate in marked region is three-dimensional coordinate, and is concrete, can produce the attitude of marked region according to the pixel coordinate of the center of gravity of two continuums that calculate, and this attitude is an attitude vector.

In one embodiment, also can comprise before the step of the attitude in identification marking zone: the image that judgement gathers is two dimensional image or the step of 3-D view.Concrete, if the image that gathers is two dimensional image, carry out above-mentioned steps S202 to step S204, if the image that gathers is 3-D view, carry out above-mentioned steps S210 to S220.

As shown in Figure 9, in one embodiment, the detailed process of above-mentioned steps S30 comprises:

Step S302 obtains the attitude of this marked region in current frame image.

As mentioned above, the attitude of obtaining in step S302 can be the attitude (being attitude angle) of the marked region in the two dimensional image of present frame, can be also the attitude (being the attitude vector) of the marked region in the three-dimensional dark image of present frame.In the present embodiment, the mapping relations between attitude and steering order have been preset.This attitude also can be described as absolute attitude.

Step S304, the steering order corresponding with this attitude with the mapping relations generation between steering order according to default attitude.

For example, steering order is left mouse button instruction and right button instruction.Take two dimensional image as example, the span of attitude angle is that-180 degree are to 180 degree.Can preset attitude angle in current frame image in the scope of (a, b), trigger the left button instruction, the attitude angle in current frame image triggers the right button instruction in the scope of (c, d).Wherein, a, b, c, d are predefined angle, satisfy a＜b, c＜d, and the common factor of set [a, b] and set [c, d] is empty.

In addition, in 3-D view, the attitude that identifies comprises two attitude angle, can obtain steering order with one of them attitude angle, also can obtain steering order with two attitude angle.Use Method And Principle and the two dimensional image of one of them attitude angle similar, repeat no more at this.When using two attitude angle, if two attitude angle can be set all in predefined instruction triggers scope the time, just trigger steering order.

As shown in figure 10, in another embodiment, the image that comprises marked region of collection is image sequence, and the detailed process of above-mentioned steps S30 comprises:

Step S310 obtains the relative attitude between the attitude of the attitude of this marked region in current frame image and this marked region in the previous frame image.

In the present embodiment, but the image sequence that Real-time Collection is comprised of a plurality of images that comprise marked region.As mentioned above, the attitude of obtaining in step S310 can be the attitude angle of the marked region in current frame image and previous frame image, can be also the attitude vector of the marked region in current frame image and previous frame image.Relative attitude between attitude in attitude in current frame image and previous frame image is both differences.

Step S320, the steering order corresponding with this relative attitude with the mapping relations generation between steering order according to default relative attitude.

For example, take two dimensional image as example, relative attitude is the relative attitude angle, the attitude angle that can preset current frame image is spent greater than 30 than the attitude angle increase of previous frame, be relative attitude angle when spending greater than 30, trigger the instruction that the roller of mouse rolls counterclockwise, the attitude angle of current frame image reduces when spending greater than 40 than the attitude angle of previous frame, be relative attitude angle when spending less than-40, trigger the instruction that the roller of mouse rolls clockwise.The principle of 3-D view is similar with it, repeats no more at this.

In 3-D view, the attitude that identifies comprises two attitude angle, can obtain steering order with one of them attitude angle, also can obtain steering order with two attitude angle.Use Method And Principle and the two dimensional image of one of them attitude angle similar, repeat no more at this.When using two attitude angle, change and all satisfy when pre-conditioned if two attitude angle can be set, for example first attitude angle changes greater than predefined first threshold, and second attitude angle changes greater than predefined Second Threshold, triggers steering order.

In one embodiment, as shown in figure 11, a kind of system that controls browsing page comprises image capture module 10, gesture recognition module 20, instruction generation module 30 and instruction transformation module 40, wherein:

Image capture module 10 is used for gathering the image that comprises marked region.

In the present embodiment, marked region is a zone in the image that gathers, and this zone can be formed by interactive device.Concrete, in one embodiment, interactive device can be hand-held device, part or all of hand-held device can be set as color or the shape of appointment, gather the image of hand-held device, this designated color in the hand-held device in image or the part of shape form marked region.In addition, interactive device can also be the hand-held device of tape label, namely attach the mark (as reflectorized material) of designated color or shape on hand-held device, gather the image of hand-held device, on the hand-held device in image, the mark of incidental designated color or shape forms marked region.

Gesture recognition module 20 is used for the attitude in identification marking zone.

Concrete, the image that collects is processed, extract the marked region in image, then obtain the attitude of marked region according to the pixel coordinate of the pixel in marked region in the image coordinate system that builds.So-called attitude refers to marked region formed posture state in image.Further, in two dimensional image, attitude is marked region and the angle between predeterminated position, the i.e. attitude angle in two dimensional image; In 3-D view, attitude is the vector that marked region in two dimensional image and a plurality of attitude angle between predeterminated position form, i.e. the attitude vector." attitude that marked region produces " said in the present invention, " attitude of marked region " all refers to described attitude, namely the attitude angle of different embodiment and attitude vector.

Instruction generation module 30 is used for generating steering order corresponding to attitude.

In the present embodiment, preset the attitude of marked region and the mapping relations between steering order, and these mapping relations are stored in the database (not shown).After identifying the attitude of marked region, the attitude that instruction generation module 30 can be used for identifying according to gesture recognition module 20 is searched the steering order corresponding with attitude from database.

Instruction transformation module 40 is used for converting steering order to natural language information.

For example, the mapping table of default steering order and natural language information can be as shown in table 2:

Table 2

In one embodiment, instruction transformation module 40 also can be used for steering order is arranged in the steering order sequence, generates natural language information according to the steering order sequence.

Predeterminable sampling interval T generates a plurality of steering orders every sampling interval T, then with the steering order that generates according to the one-tenth steering order sequence arranged sequentially that generates, then can generate natural language information according to the steering order sequence.

For example, can adopt user's finger and arm as interactive device, then catch user's finger and the attitude of arm every 0.1 second, and generate steering order and be used for this attitude constantly of sign.After having generated within a certain period of time a plurality of steering orders, steering order has represented user's finger and the running orbit of arm according to the sequence that the order that generates forms.

Further, instruction transformation module 40 also can be used for calculating the eigenwert of steering order sequence, according to the mapping relations generation natural language information of default eigenwert and natural language information.

Can comprise two parts information in steering order, a part is for being used for the information attitude part of expression attitude, and another part represents the coordinate part of the coordinate information of this attitude in image.When with finger and arm during as interactive device, the attitude of steering order part can represent to point shape that the attitude with arm consists of or abstract vector graphics, launches shape, points the ring-type that surrounds as the five fingers.The coordinate part of steering order has represented the position of shape on image that finger or arm surround.When image was two dimensional image, coordinate was two-dimensional coordinate.When image was 3-D view, coordinate was three-dimensional coordinate.

Eigenwert and natural language information have default mapping relations.For example, can in advance the corresponding natural language of eigenwert 2 (1F@1L) " be thanks ".This eigenwert 2 (1F@1L) is based on the variation of the attitude part of steering order sequence, expression is stretched (being represented by the 1F in eigenwert) by thumb and is become thumb bending (being represented by the 1L in eigenwert), and crooked twice (multiply by 2 with bracket represents).Can in advance the corresponding natural language of eigenwert 5F_y-down " be pressed ".This eigenwert 5F_y-down is based on the variation of the part of the coordinate in the steering order sequence, and expression palm (being represented by the 5F in eigenwert) is pressed (being represented by the y-down in eigenwert) from top to bottom.

Further, the steering order that instruction transformation module 40 also is used for first the steering order sequence being repeated is removed, and then calculates the eigenwert of steering order sequence.Remove the steering order that repeats and to reduce calculated amount.

Further, instruction transformation module 40 also is used for first steering order sequence and adjacent steering order difference being removed greater than the steering order of threshold value, and then calculates the eigenwert of steering order sequence.When default sampling interval hour, if the attitude of certain steering order in steering order sequence part or coordinate part during greater than threshold value, judge that this steering order is the steering order of mistake with adjacent steering order difference.False command will be filtered out the steering order sequence, and can not be used to calculate the eigenwert of steering order sequence.

For example, when the user adopts finger as input media, if the steering order sequence that gets is: [3F (0,0), 3F (1,0), 3F (2,0), 2F (3,0), 3F (4,0), 3F (5,0)].Wherein attitude part 3F represents the attitude of 3 fingers, and 2F represents the attitude of 2 fingers, and coordinate part (0,0)-(5,0) expression finger gesture is in the coordinate in the image of catching.This steering order sequence is used for three tracks that the finger horizontal translation forms of expression user, and each steering order is the sampled point of this track.Wherein, may be due to the user in moving process, the situation that has finger to close up, steering order 2F (3,0) 3F (2,0), the 3F (4,0) with adjacent is larger in the upper gap of the number (the attitude part of steering order) of finger, therefore be judged as wrong steering order, and be moved out of the steering order sequence.

When certain steering order and adjacent steering order have than big difference, normally because the nonstandard sign language gesture of having caught the user, or because interactive device is blocked when mobile, can not obtain its shape fully and cause.For example, in deaf and dumb sign language, some gesture needs two hands alternately mobile, in movement, and the situation that may occur blocking, therefore remove and the steering order of adjacent steering order gap greater than threshold value (shape different or coordinate distance greater than threshold value), can be more accurate so that eigenwert is calculated.

When calculating the eigenwert of steering order sequence, instruction transformation module 40 also is used for the posture feature value of the steering order of calculating steering order sequence, then according to the posture feature value with the segmentation of steering order sequence, the steering order in the steering order subsequence after segmentation has identical posture feature value.The monotonicity of the coordinate of then partly describing according to the coordinate of the steering order in the steering order subsequence, the variance of the distance of a certain reference coordinate is calculated the coordinate characteristic value of steering order subsequence in the image.Then the eigenwert that posture feature value and the coordinate characteristic value of all steering order subsequences is integrated into the steering order sequence.

The steering order sequence that gets is: [1F (0,0), 1F (1,0), 1F (2,0), 1F (3,0), 5F (2.5,1.5), 5F (1.5,2.5), 5F (0,3), OK (0,3), OK (0,3), OK (0,3)].Wherein 1F is the posture feature value of 1 finger, and 5F is the posture feature value of 5 fingers, and OK represents the posture feature value of OK shape gesture, and bracket inner digital represents the coordinate figure of attitude in image.Remove repetition steering order and with the adjacent steering order of steering order difference greater than threshold value after, can the steering order sequence be divided into three sub-steering order sequences according to the difference of posture feature value:

Subsequence 1:[1F (0,0), 1F (1,0), 1F (2,0), 1F (3,0)].

Subsequence 2:[5F (2.5,1.5), 5F (1.5,2.5), 5F (0,3)].

Subsequence 3:[OK (0,3)].

Then instruction transformation module 40 is integrated posture feature value and the coordinate characteristic value of subsequence 1, subsequence 2 and subsequence 3, and the eigenwert of controlled instruction sequence is: 1F_x-right@5F_q-cir@OK_hold.Wherein _ and represent separator with@, be used for distinguishing posture feature value and coordinate characteristic value.

Further, when steering order is arranged in the steering order sequence, instruction transformation module 40 also can be used for steering order according to the one-tenth formation arranged sequentially that generates, when the formation end being detected the identical steering order of continuously default number arranged, arrange completely, and generate the steering order sequence according to formation.

In one embodiment, as shown in figure 16, sign Language Recognition also comprises information display module 50, is used for showing natural language information by the mode of text and/or audio frequency after converting steering order to natural language information.

Information display module 50 also is used for by display screen, natural language information being showed with the form of word, also can natural language information be played back by audio frequency apparatus.

As shown in figure 12, in one embodiment, the image that image capture module 10 collects is two dimensional image, and gesture recognition module 20 comprises the first image processing module 202 and the first attitude generation module 204, wherein:

The first image processing module 202 is used for extracting image and the pixel that default color model mates, and the pixel of obtaining is carried out connected domain detect, and extracts the marked region that detects in the connected domain that obtains.

Concrete, image capture module 10 can be video camera, and its image that collects can be the two-dimensional visible light image.Preferably, also can add infrared fileter before the camera lens of video camera, be used for elimination except other wave band light of infrared band, the image of image capture module 10 collections is the two-dimensional infrared image.In visible images, the object in scene can form the identification of marked region and disturb, and infrared image has been because having filtered out visible light information, disturbs lessly, so the two-dimensional infrared image more is conducive to extract marked region.

Concrete, the first image processing module 202 is used for setting up in advance color model.For example the color of marked region is red, sets up in advance red model, and in this model, the rgb value component of pixel can be between 200 to 255, and G, B component can be close to zero; The first 202 of image processing modules are used for obtaining the pixel that two field picture satisfies the rgb value of this redness model and are red pixel.In addition, when forming marked region by human body in the image that gathers, the first 202 of image processing modules are for obtaining the pixel of image with default complexion model coupling.The first image processing module 202 is used for that also the pixel of obtaining is carried out connected domain and detects, and obtains a plurality of connected domains, if connected domain is the set that individual continuous pixel forms.

In the present embodiment, because the size and shape of marked region should be roughly changeless, the first image processing module 202 is carrying out connected domain when detecting to the pixel of obtaining, can calculate girth and/or the area of all connected domains in the pixel of obtaining.Concrete, the girth of connected domain can be the number of connected domain boundary pixel, and the area of connected domain can be the number of the whole pixels in connected domain.Further, the first image processing module 202 can be used for the girth of the connected domain that will obtain and/or girth and/or the area of area and default marked region compares, and obtains the girth that satisfies default marked region and/or the connected domain of area and is marked region.Preferably, the first image processing module 202 also can be used for girth square with the ratio of area as judgment criterion, this ratio of connected domain satisfies this ratio of default marked region, this connected domain is marked region.

The first attitude generation module 204 is used for obtaining the pixel coordinate of marked region, produces the attitude of marked region according to this pixel coordinate.

In the present embodiment, the attitude that marked region produces is attitude angle.In one embodiment, marked region is a continuum, the first attitude generation module 204 is used for calculating the covariance matrix of pixel coordinate, obtain covariance matrix eigenvalue of maximum characteristic of correspondence vector, produce the attitude of marked region according to proper vector, the attitude of this marked region is an attitude angle.

In another embodiment, marked region comprises the first continuum and the second continuum, the first attitude generation module 204 is used for calculating the center of gravity of the first continuum and the center of gravity of the second continuum according to pixel coordinate, calculates the attitude of marked region according to the pixel coordinate of the center of gravity of the pixel coordinate of the center of gravity of the first continuum and the second continuum.Concrete, calculate the mean value of the whole pixel coordinates in the continuum, resulting pixel coordinate is the center of gravity of continuum.

In another embodiment, the image that collects of image capture module 10 is 3-D view.Concrete, image capture module 10 can adopt traditional stereo visual system (being comprised of two known video camera and related softwares in control position), structured-light system (a right video camera, light source and related software form) or TOF (time of flight, flight time) depth camera to realize collection 3-D view (being the three dimensional depth image).

In the present embodiment, as shown in figure 13, gesture recognition module 20 comprises the second image processing module 210 and the second attitude generation module 220, wherein:

The second image processing module 210 is used for described Image Segmentation Using, extract the connected domain in image, and the property value of calculating connected domain, the property value of connected domain and default marked region property value are compared, and described marked region is the connected domain that meets described default marked region property value.

Concrete, the second image processing module 210 is used for when two adjacent pixel depths of 3-D view differ less than predefined threshold value, for example 5 centimetres, thinks that two pixels are communicated with, whole image is carried out connected domain detect, can obtain comprising a series of connected domains of mark connected domain.

In the present embodiment, the property value of connected domain comprises the size and dimension of connected domain.Concrete, the second image processing module 210 is used for calculating the size/shape of connected domain, compares with the size/shape of mark on interactive device, and the connected domain that obtains meeting the size/shape of mark is the connected domain (marked region) of marked region.Take rectangle marked as example, be to be rectangle in the image that is marked at collection on interactive device, the length of pre-set mark and width, the second 210 of image processing modules are used for calculating length and the width of physical region corresponding to connected domain, length and the width of this length and width and mark are more approaching, and connected domain is more similar to marked region.

Further, the second image processing module 210 is as follows for the process of the length of calculating physical region corresponding to connected domain and width: calculate the covariance matrix of the three-dimensional coordinate of connected domain pixel, adopt following formula to calculate length and the width of physical region corresponding to connected domain: Wherein, k is predefined coefficient, for example is made as 4, and when λ was the covariance matrix eigenvalue of maximum, l was the length of connected domain, and when λ was the second largest eigenwert of covariance matrix, l was the width of connected domain.

Further, the second image processing module 210 also can be used for presetting the length breadth ratio of rectangle marked, for example length breadth ratio is 2, the length breadth ratio of physical region corresponding to connected domain is more close to the length breadth ratio of the rectangle marked of default settings, connected domain is more similar to marked region, concrete, attribute matching module 234 is used for adopting following formula to calculate the length breadth ratio of physical region corresponding to connected domain:

The second attitude generation module 220 is used for obtaining the pixel coordinate of marked region, produces the attitude of marked region according to described pixel coordinate.

In the present embodiment, the attitude of marked region is the attitude vector.In one embodiment, marked region is a continuum, the second attitude generation module 220 is used for calculating the covariance matrix of pixel coordinate, obtains covariance matrix eigenvalue of maximum characteristic of correspondence vector, produces the attitude of marked region according to proper vector.As mentioned above, the attitude of this marked region is an attitude vector.

In another embodiment, marked region comprises the first continuum and the second continuum, the second attitude generation module 220 is used for calculating the center of gravity of the first continuum and the center of gravity of the second continuum according to pixel coordinate, produces the attitude of marked region according to the pixel coordinate of the center of gravity of the pixel coordinate of the center of gravity of the first continuum and the second continuum.In the present embodiment, the pixel coordinate in marked region is three-dimensional coordinate, and is concrete, can produce the attitude of marked region according to the pixel coordinate of the center of gravity of two continuums that calculate, and this attitude is an attitude vector.

In one embodiment, gesture recognition module 20 also comprises the judge module (not shown), and the image that is used for the judgement collection is two dimensional image or 3-D view.Concrete, in the present embodiment, when the image that determines collection when judge module is two dimensional image, the marked region of notifying the first image processing module 202 to extract in two dimensional images, and then produce the attitude of these marked regions by the first attitude generation module 204.When the image that determines collection when judge module is two dimensional image, the marked region of notifying the second image processing module 210 to extract in 3-D views, and then produce the attitude of these marked regions by the second attitude generation module 220.Understandable, in the present embodiment, gesture recognition module 20 comprises judge module (not shown), the first image processing module 202, the first attitude generation module 204, the second image processing module 210 and the second attitude generation module 220 simultaneously.The present embodiment both can by the attitude in two dimensional image identification marking zone, can pass through again the attitude in two dimensional image identification marking zone.

As shown in figure 14, in one embodiment, instruction generation module 30 comprises that the first attitude acquisition module 302 and the first instruction search module 304, wherein:

The first attitude acquisition module 302 is used for obtaining from gesture recognition module 20 attitude of the described marked region current frame image.

Concrete, this attitude can be the attitude angle of the marked region in the two dimensional image of present frame, can be also the attitude vector of the marked region in the three dimensional depth image of present frame.In the present embodiment, the mapping relations between attitude and steering order have been preset.This attitude also can be described as absolute attitude.

The first instruction is searched module 304 and is used for the steering order corresponding with described attitude with the mapping relations generation between steering order according to default attitude.

In the present embodiment, the image that comprises marked region that gathers can be image sequence.The first attitude acquisition module 302 is also for the relative attitude between the attitude of the attitude of obtaining the marked region current frame image from gesture recognition module 20 and the marked region in the previous frame image.The first instruction is searched module 304 and also is used for the steering order corresponding with relative attitude with the mapping relations generation between steering order according to default relative attitude.

In another embodiment, the image that comprises marked region that gathers can be image sequence.As shown in figure 15, instruction generation module 30 comprises that the second attitude acquisition module 310 and the second instruction search module 320, wherein:

The second attitude acquisition module is for the relative attitude between the attitude of the attitude of obtaining the marked region current frame image from gesture recognition module 20 and the marked region in the previous frame image.

The second instruction is searched module 320 and is used for the steering order corresponding with relative attitude with the mapping relations generation between steering order according to default relative attitude.

The above embodiment has only expressed several embodiment of the present invention, and it describes comparatively concrete and detailed, but can not therefore be interpreted as the restriction to the scope of the claims of the present invention.Should be pointed out that for the person of ordinary skill of the art, without departing from the inventive concept of the premise, can also make some distortion and improvement, these all belong to protection scope of the present invention.Therefore, the protection domain of patent of the present invention should be as the criterion with claims.

Claims

1. sign Language Recognition Method comprises the following steps:

Collection comprises the image of marked region;

The attitude in identification marking zone;

Generate steering order corresponding to described attitude;

Convert described steering order to natural language information.

2. sign Language Recognition Method according to claim 1, is characterized in that, the described step that converts described steering order to natural language information comprises:

3. the sign Language Recognition Method described according to claim 2, is characterized in that, the described step that generates natural language information according to described steering order sequence comprises:

4. the sign Language Recognition Method described according to claim 3, is characterized in that, also comprises before the step of the eigenwert of the described steering order sequence of described calculating:

5. the sign Language Recognition Method described according to claim 3, is characterized in that, also comprises before the step of the eigenwert of the described steering order sequence of described calculating:

6. the sign Language Recognition Method described according to claim 2, is characterized in that, the described step that described steering order is arranged in the steering order sequence comprises:

According to claim 1 to 6 described sign Language Recognition Method, it is characterized in that, describedly also comprise after converting described steering order the step of natural language information to:

8. a sign Language Recognition, is characterized in that, comprises with lower module:

9. sign Language Recognition according to claim 8, is characterized in that, described instruction transformation module also is used for described steering order is arranged in the steering order sequence, generates natural language information according to described steering order sequence.

10. the sign Language Recognition described according to claim 9, it is characterized in that, described instruction transformation module also is used for calculating the eigenwert of described steering order sequence, generates natural language information according to described default eigenwert and the mapping relations of natural language information.

11. the sign Language Recognition according to claim 10 is characterized in that, the steering order that described instruction transformation module also is used for described steering order sequence is repeated is removed.

12. the sign Language Recognition Method according to claim 10 is characterized in that, described instruction transformation module also is used for described steering order sequence and adjacent steering order difference are removed greater than the steering order of threshold value.

13. the sign Language Recognition according to claim 9, it is characterized in that, described instruction transformation module also is used for described steering order according to the formation arranged sequentially that generates, when the formation end being detected the identical steering order of continuously default number arranged, arrange completely, and generate the steering order sequence according to described formation.

14. the described sign Language Recognition of according to claim 8 to 13 any one is characterized in that, also comprises the information display module, is used for showing described natural language information by the mode of text and/or audio frequency.