CN103135758A - Method and system for achieving shortcut function - Google Patents
Method and system for achieving shortcut function Download PDFInfo
- Publication number
- CN103135758A CN103135758A CN2012100329325A CN201210032932A CN103135758A CN 103135758 A CN103135758 A CN 103135758A CN 2012100329325 A CN2012100329325 A CN 2012100329325A CN 201210032932 A CN201210032932 A CN 201210032932A CN 103135758 A CN103135758 A CN 103135758A
- Authority
- CN
- China
- Prior art keywords
- steering order
- input instruction
- attitude
- instruction
- input
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Landscapes
- Image Analysis (AREA)
- User Interface Of Digital Computer (AREA)
- Processing Or Creating Images (AREA)
- Position Input By Displaying (AREA)
Abstract
The invention relates to a method for achieving a shortcut function. The method comprises the steps of collecting an image containing a marked area, identifying a gesture of the marked area, generating a control instruction corresponding to the gesture, obtaining an input instruction corresponding to the control instruction according to the control instruction and a mapping relation between a preset control instruction and a preset input instruction, and sending the input instruction to an instruction processing system. In addition, provided is a system for achieving the shortcut function. The method and the system for achieving the shortcut function facilitate user operation.
Description
[technical field]
The present invention relates to artificial intelligence field, particularly a kind of method and system that realize shortcut function.
[background technology]
Existing computer technology is all based on Instruction System Design.Computing machine first obtains the input instruction of user's input, then resolves the input instruction and carries out corresponding operation by its inner instruction process system.And along with the development of computer technology, the executable operation of computing machine is more and more, inputs accordingly instruction more and more, character string long and more uncommon input instruction make the user be not easy memory, the impact operation.
In conventional art, input for the convenience of the user the instruction executable operations, by adding self-defined another name for the input instruction or adding shortcut and realize shortcut function in graphic interface.For example, in Windows, can be the shortcut that file set up is positioned over desktop.The user can input instruction by choice menus fast to the shortcut on desktop when select File.
Yet the method that realizes shortcut function in conventional art still needs the user to knock the another name of input instruction by keyboard or uses mouse to choose shortcut on desktop.As the user during away from computing machine, just can complete input, inconvenient user's operation before needing the user first to move to the input equipment of computing machine.
[summary of the invention]
Based on this, be necessary to provide a kind of user-friendly method that realizes shortcut function.
A kind of method that realizes shortcut function comprises the following steps:
Collection comprises the image of marked region;
The attitude in identification marking zone;
Generate steering order corresponding to described attitude;
Obtain the input instruction corresponding with described steering order with the mapping relations of input instruction, and send to instruction process system according to described steering order, default steering order.
Preferably, steering order corresponding to the described attitude of described generation also comprises afterwards:
Detect whether the configuration-direct input is arranged;
When the configuration-direct input having been detected, obtain the input instruction, and generate described steering order and the mapping relations of inputting instruction according to described steering order and described input instruction;
When the configuration-direct input not detected, obtain the input instruction corresponding with described steering order with the mapping relations of input instruction, and send to instruction process system according to described steering order, default steering order.
Preferably, the described step of obtaining the input instruction comprises:
Show the visual configuration interface, obtain the input instruction by described visual configuration interface.
Preferably, described input instruction is many;
The described step that generates described steering order and the mapping relations of input instruction according to described steering order and described input instruction comprises:
Described input instruction is generated the input instruction sequence according to the time sequencing of obtaining, and generate the mapping relations of described steering order and described input instruction sequence.
Preferably, the described step of obtaining with input instruction corresponding to described steering order according to described steering order, default steering order and the mapping relations of input instruction comprises:
Obtain the input instruction sequence corresponding with described steering order according to described steering order, described steering order with the mapping relations of input instruction sequence, extract the input instruction by the described time sequencing of obtaining from described input instruction sequence.
Preferably, described also comprising after the step of described input instruction according to the time sequencing generation input instruction sequence of obtaining:
With the form storage of described input instruction sequence with script.
Preferably, the described step of obtaining with input instruction sequence corresponding to described steering order according to described steering order, described steering order and the mapping relations of input instruction sequence comprises:
Obtain the input instruction sequence corresponding with described steering order according to described steering order, described steering order with the mapping relations of input instruction sequence from script.
In addition, also be necessary to provide a kind of user-friendly system that realizes shortcut function.
A kind of system that realizes shortcut function comprises with lower module:
Image capture module is used for gathering the image that comprises marked region;
The gesture recognition module is for the attitude in identification marking zone;
The instruction generation module is used for generating steering order corresponding to described attitude;
The instruction transformation module is used for obtaining the input instruction corresponding with described steering order with the mapping relations of input instruction, and sending to instruction process system according to described steering order, default steering order.
Preferably, also comprise the mapping configuration module, for detection of whether the configuration-direct input is arranged; When the configuration-direct input having been detected, obtain the input instruction, and generate described steering order and the mapping relations of inputting instruction according to described steering order and described input instruction; When the configuration-direct input not detected, the call instruction modular converter.
Preferably, described mapping configuration module also be used for to be showed the visual configuration interface, obtains the input instruction by described visual configuration interface.
Preferably, described input instruction is many;
Described mapping configuration module also is used for described input instruction is generated the input instruction sequence according to the time sequencing of obtaining, and generates the mapping relations of described steering order and described input instruction sequence.
Preferably, described instruction transformation module also is used for obtaining the input instruction sequence corresponding with described steering order according to described steering order, described steering order with the mapping relations of input instruction sequence, extracts the input instruction by the described time sequencing of obtaining from described input instruction sequence.
Preferably, described mapping configuration module also is used for the form storage of described input instruction sequence with script.
Preferably, described instruction transformation module also is used for obtaining with described steering order corresponding input instruction sequence with the mapping relations of input instruction sequence from script according to described steering order, described steering order.
Above-mentioned shortcut function method and the system of realizing, go out according to the image recognition that comprises marked region that collects the attitude that marked region produces, and generate steering order corresponding to attitude, then the mapping relations according to default steering order and input instruction convert this steering order to input instruction that instruction process system can be identified.Owing to judging movement locus and the attitude of limb action by the image that obtains limb action, therefore, the user only need to make corresponding limb action just can complete quick input, need to not use keyboard or mouse input, and makes that the user's is more convenient to operate.
[description of drawings]
Fig. 1 realizes the schematic flow sheet of the method for shortcut function in the present invention;
Fig. 2 is the schematic flow sheet of step S20 in an embodiment;
Fig. 3 is the structural representation of interactive device in an embodiment;
Fig. 4 builds the schematic diagram of coordinate system in an embodiment;
Fig. 5 is the structural representation of the interactive device in another embodiment;
Fig. 6 is the structural representation of the interactive device in another embodiment;
Fig. 7 is the schematic flow sheet of step S20 in another embodiment;
Fig. 8 builds the schematic diagram of coordinate system in another embodiment;
Fig. 9 is the schematic flow sheet of step S30 in an embodiment;
Figure 10 is the schematic flow sheet of step S30 in another embodiment;
Figure 11 realizes the structural representation of the system of shortcut function in an embodiment;
Figure 12 is the structural representation of gesture recognition module in an embodiment;
Figure 13 is the structural representation of gesture recognition module in another embodiment;
Figure 14 is the structural representation of instruction generation module in an embodiment;
Figure 15 is the structural representation of instruction generation module in another embodiment;
Figure 16 is for realizing the structural representation of the system of shortcut function in another embodiment.
[embodiment]
Below in conjunction with specific embodiment and accompanying drawing, technical scheme is described in detail.
In one embodiment, as shown in Figure 1, a kind of shortcut function method of realizing comprises the following steps:
Step S10 gathers the image that comprises marked region.
In the present embodiment, marked region is a zone in the image that gathers, and this zone can be formed by interactive device.
Concrete, in one embodiment, interactive device can be hand-held device, part or all of hand-held device can be set as color or the shape of appointment, gather the image of hand-held device, this designated color in the hand-held device in image or the part of shape form marked region.In addition, interactive device can also be the hand-held device of tape label, namely attach the mark (as reflectorized material) of designated color or shape on hand-held device, gather the image of hand-held device, on the hand-held device in image, the mark of incidental designated color or shape forms marked region.
In another embodiment, interactive device can also be human body (such as people's face, palm, arm etc.), gathers the image of human body, and the human body in image forms marked region.In addition, interactive device can also be the human body of tape label, namely attaches the mark (as reflectorized material) of designated color or shape on human body, and when gathering the image of human body, this designated color in image or the mark of shape form marked region.
Step S20, the attitude in identification marking zone.
Concrete, the image that collects is processed, extract the marked region in image, then produce the attitude of marked region according to the pixel coordinate of the pixel in marked region in the image coordinate system that builds.So-called attitude refers to marked region formed posture state in image.Further, in two dimensional image, attitude is marked region and the angle between predeterminated position, the i.e. attitude angle in two dimensional image; In 3-D view, attitude is the vector that marked region in two dimensional image and a plurality of attitude angle between predeterminated position form, i.e. the attitude vector." attitude that marked region produces " said in the present invention, " attitude of marked region ", " attitude " all refer to described attitude, namely the attitude angle of different embodiment and attitude vector.
Step S30 generates steering order corresponding to attitude.
In the present embodiment, preset the attitude of marked region and the mapping relations between steering order, and these mapping relations are stored in database.After identifying the attitude of marked region, can search the steering order corresponding with attitude from database according to the attitude that identifies.
Step S40 obtains the input instruction corresponding with steering order with the mapping relations of input instruction, and sends to instruction process system according to steering order, default steering order.
The input instruction is the instruction that operating system can be identified.For example, order line input instruction " dir/p " expression split screen is showed file and the sub-directory under current directory; The current file of choosing of the input instruction " deletion " of right button popup menu expression deletion after the mouse filesselected.
The mapping relations of steering order and input instruction are for presetting, and store with the form of key-value pair.After the steering order that gets generation, find the input instruction corresponding with this steering order according to this key-value pair, then send to instruction process system.
For example, shown the attitude of L when user's limb action, when having generated corresponding steering order A001, find corresponding with it input instruction and be " del*.txt " (deleting the file of all expansions txt by name) in default key-value pair, then " del*.txt " sent to instruction process system, instruction process system is carried out the file of all suffix of deletion txt by name by parsing.
The above-mentioned shortcut function method of realizing, go out according to the image recognition that comprises marked region that collects the attitude that marked region produces, and generate steering order corresponding to attitude, then the mapping relations according to default steering order and input instruction convert this steering order to input instruction that instruction process system can be identified.Owing to judging movement locus and the attitude of limb action by the image that obtains limb action, therefore, the user only need to make corresponding limb action just can complete quick input, need to not use keyboard or mouse input, and makes that the user's is more convenient to operate.
In one embodiment, after generating steering order corresponding to attitude, detect also whether the configuration-direct input is arranged; When the configuration-direct input having been detected, obtain the input instruction, and generate the mapping relations of steering order and input instruction according to steering order and input instruction; When the configuration-direct input not detected, execution in step S40.
Can pre-defined steering order and the mapping relations of inputting instruction.Obtain according to step S10, S20 and S30 steering order corresponding to attitude that consists of with user's limb action.Before execution in step S40, first judge whether to have detected the configuration-direct input.Whether configuration-direct is a switch order, be used for the expression configuration and begin.The user can trigger by the button on the devices such as telepilot, also can be by predetermined attitude action triggers.When the configuration-direct input having been detected, the steering order that expression is generated by step S10, S20 and S30 is used for arranging the mapping relations of steering order and input instruction; When not detecting, expression is used for searching the input instruction corresponding with this steering order by the steering order that step S10, S20 and S30 generate.
Further, show the visual configuration interface, obtain the input instruction by the visual configuration interface.
For example, user's limb action has been shown the attitude that opens arms, and then the steering order 002 that is then generated by step S10, S20 and S30 detects the user and whether triggering configuration-direct by pressing button default on telepilot or keyboard before.If the configuration-direct input detected, show visual input instruction input frame, can obtain the input instruction of character string type, also can obtain the cursor track type input instruction that in input frame, cursor movement forms, then with steering order 002 and the form storage of the input instruction that gets with key-value pair.
In one embodiment, the input instruction is many.When generating steering order and inputting the mapping relations of instruction, the input instruction is generated the input instruction sequence according to the time sequencing of inputting, and generation steering order and the mapping relations of inputting instruction sequence.
When obtaining the input instruction corresponding with steering order, obtain the input instruction sequence corresponding with steering order according to steering order, steering order with the mapping relations of input instruction sequence, extract the input instruction by the time sequencing of inputting from the input instruction sequence.
For example, can obtain by the visual configuration interface many input instruction B001, B002, the B003......B00N of user input, and will input instruction and be arranged in the input instruction sequence according to the time sequencing of inputting.Then by steering order and input instruction sequence are generated the mapping relations of steering order and input instruction with the form storage of key-value pair.When configuration finishes, when obtaining the input instruction according to steering order, first obtain the input instruction sequence corresponding with it according to steering order, extract the input instruction and send to instruction process system by the time sequencing of inputting from the input instruction sequence, instruction process system namely carries out by the time sequencing of input the input instruction of inputting in instruction sequence.
Further, after the time sequencing generation input instruction sequence of input instruction according to input, also will input instruction sequence with the form storage of script.
When obtaining the input instruction sequence corresponding with steering order, obtain the input instruction sequence corresponding with steering order according to steering order, steering order with the mapping relations of input instruction sequence from script.
Adopt script storage input instruction sequence, can facilitate the user to preserve and transmission configuration.On the user can upload onto the server the script that configures, and the synchronizeing of version on maintenance and server.When wishing on a plurality of equipment as the user to adopt identical configuration, can directly download this configuration on server, thereby the mapping relations that make the user avoid frequently configuring steering order and input instruction are more convenient to operate.
Further, script also comprises sequence control instruction, after getting the input instruction, also obtains order and describes instruction, describes instruction according to order and will input instruction and be arranged in the input instruction sequence.
When obtaining the input instruction sequence corresponding with steering order, describe instruction according to order and resolve script, then obtain the input instruction sequence corresponding with steering order according to steering order, steering order with the mapping relations of input instruction sequence from script.
Order is described instruction and is comprised that looping discription instruction, branch describe instruction.For example, create fast 10000 files if wish by the limbs attitude, the filename of the file that creates is natural number to be arranged, and can limit the input instruction that creates file with the looping discription instruction, and the file name parameter that each circulation time will create in the input instruction of file adds 1.Added sequentially instruction is described after, the user carries out a plurality of input instructions when the mapping relations of configuration steering order and input instruction according to the order of appointment, facilitate the user to be configured, and simultaneously, has also expanded input instruction corresponding to steering order.
As shown in Figure 2, in one embodiment, the image that comprises marked region that collects is two dimensional image, and the detailed process of above-mentioned steps S20 comprises:
Step S202 extracts the pixel of mating with default color model in image, the pixel of obtaining is carried out connected domain detect, and extracts the marked region that detects in the connected domain that obtains.
Concrete, can comprise by camera acquisition the image of marked region, the image that obtains is the two-dimensional visible light image.Preferably, also can add infrared fileter before the camera lens of video camera, be used for elimination except other wave band light of infrared band, the image that gathers is the two-dimensional infrared image.In visible images, the object in scene can form the identification of marked region and disturb, and infrared image has been because having filtered out visible light information, disturbs lessly, so the two-dimensional infrared image more is conducive to extract marked region.
In the present embodiment, set up in advance color model.For example the color of marked region is red, sets up in advance red model, and in this model, the rgb value component of pixel can be between 200 to 255, and G, B component can be close to zero; Obtain the pixel that satisfies the rgb value of this redness model in the image of collection and be red pixel.In addition, when forming marked region by human body in the image that gathers, can obtain the pixel of mating with default complexion model in the image of collection.The pixel of obtaining is carried out connected domain detect, obtain a plurality of connected domains, if connected domain is the set that individual continuous pixel forms.
In the present embodiment, because the size and shape of marked region should be roughly changeless, the pixel of obtaining is being carried out connected domain when detecting, can calculate girth and/or the area of all connected domains in the pixel of obtaining.Concrete, the girth of connected domain can be the number of connected domain boundary pixel, and the area of connected domain can be the number of the whole pixels in connected domain.Further, the girth of the connected domain obtained and/or girth and/or the area of area and default marked region can be compared, obtain the girth that satisfies default marked region and/or the connected domain of area and be marked region.Preferably, also can with girth square with the ratio of area as judgment criterion, this ratio of connected domain satisfies this ratio of default marked region, this connected domain is marked region.
Step S204 obtains the pixel coordinate in marked region, produces the marked region attitude according to this pixel coordinate.
Concrete, in one embodiment, as shown in Figure 3, interactive device comprises portion of the handle and the mark that is attached to portion of the handle, wherein, mark can be the reflectorized material of elongate in shape, and is preferred, can be ellipse or rectangular shape.In other embodiments, interactive device also can be human body, and as face, palm, arm etc., the marked region in the image that collects is the zone of human body.
In the present embodiment, marked region is a continuum, the process that produces the attitude of marked region according to pixel coordinate is: the covariance matrix that calculates pixel coordinate, obtain covariance matrix eigenvalue of maximum characteristic of correspondence vector, produce the attitude of marked region according to proper vector, the attitude of this marked region is an attitude angle.
Concrete, as shown in Figure 4, build the two dimensional image coordinate system, for two the some A (u1, v1) on this coordinate system and B (u2, v2), the attitude angle of its formation is the arc tangent of slope, i.e. arctan ((v2-v1)/(u2-u1)).Concrete, in the present embodiment, calculate the covariance matrix of the pixel coordinate in the marked region that extracts, obtain covariance matrix eigenvalue of maximum characteristic of correspondence vector, the direction of this proper vector is the direction of marked region major axis place straight line.As shown in Figure 4, marked region major axis place rectilinear direction is the direction of A, 2 place straight lines of B, establishes proper vector and is [dir_u, dir_v]
T, wherein, the projection of direction on the u axle of the regional major axis of dir_u descriptive markup, its absolute value is proportional to the projection (be u2-u1) of vector on the u change in coordinate axis direction of pointing to B from A; The projection of direction on the v axle of dir_v descriptive markup zone major axis, its absolute value is proportional to the projection (be v2-v1) of vector on the v change in coordinate axis direction of pointing to B from A.If dir_u or dir_v less than 0, are modified to [dir_u ,-dir_v]
T, the attitude angle of marked region is: arctan (dir_v/dir_u).
In another embodiment, marked region comprises the first continuum and the second continuum, the detailed process that produces the attitude of marked region according to described pixel coordinate is: calculate the center of gravity of the first continuum and the center of gravity of the second continuum according to pixel coordinate, produce the attitude of marked region according to the pixel coordinate of the center of gravity of the pixel coordinate of the center of gravity of the first continuum and the second continuum.Concrete, in one embodiment, interactive device comprises portion of the handle and is attached to two marks of portion of the handle.As shown in Figure 5, be labeled as two, be attached to respectively the portion of the handle front end, the shape of mark can be ellipse or rectangle.Preferably, mark can be for being positioned at two round dots of handgrip part front end.As shown in Figure 6, mark can be arranged on the two ends of portion of the handle.In other embodiments, also mark can be arranged on human body, for example be arranged on people's face, palm or arm.Should be noted that two set marks can size, inconsistent on the feature such as shape, color.
In the present embodiment, the marked region of extraction comprises two continuums, is respectively the first continuum and the second continuum.Further, calculate the center of gravity of these two continuums according to pixel coordinate.Concrete, calculate the mean value of the whole pixel coordinates in the continuum, resulting pixel coordinate is the center of gravity of continuum.As shown in Figure 4, the center of gravity of two continuums that calculate is respectively A (u1, v1) and B (u2, v2), and the attitude angle of marked region is the arc tangent of slope, i.e. arctan ((v2-v1)/(u2-u1)).
In another embodiment, the image that gathers can be 3-D view.Concrete, can utilize traditional stereo visual system (being formed by two known video camera and Correlation method for data processing equipment in locus), structured-light system (a right video camera, light source and Correlation method for data processing equipment form) or TOF (time of flight, flight time) depth camera collection 3-D view (being the three dimensional depth image).
In the present embodiment, as shown in Figure 7, the detailed process of step S20 comprises:
Step S210 to Image Segmentation Using, extracts the connected domain in this image, calculates the property value of connected domain, and the property value of connected domain and default marked region property value are compared, and this marked region is the connected domain that meets this default marked region property value.
Concrete, when in the three dimensional depth image, two adjacent pixel depths differ less than predefined threshold value, for example 5 centimetres, think that two pixels are communicated with, whole image is carried out connected domain detect, can obtain comprising a series of connected domains of mark connected domain.
In the present embodiment, the property value of connected domain comprises the size and dimension of connected domain.Concrete, calculate the size/shape of connected domain, compare with the size/shape of mark on interactive device, the connected domain that obtains meeting the size/shape of mark is the connected domain (marked region) of marked region.Take rectangle marked as example, be to be rectangle in the image that is marked at collection on interactive device, the length of pre-set mark and width, calculate length and the width of physical region corresponding to connected domain, length and the width of this length and width and mark are more approaching, and connected domain is more similar to marked region.
Further, the length of the physical region that the calculating connected domain is corresponding and the process of width are as follows: calculate the covariance matrix of the three-dimensional coordinate of connected domain pixel, adopt following formula to calculate length and the width of physical region corresponding to connected domain:
Wherein, k is predefined coefficient, for example is made as 4, and when λ was the covariance matrix eigenvalue of maximum, l was the length of connected domain, and when λ was the second largest eigenwert of covariance matrix, l was the width of connected domain.
Further, also can preset the length breadth ratio of rectangle marked, for example length breadth ratio is 2, the length breadth ratio of physical region corresponding to connected domain is more close to the length breadth ratio of the rectangle marked of default settings, connected domain is more similar to marked region, concrete, adopt following formula to calculate the length breadth ratio of physical region corresponding to connected domain:
Wherein, r is the length breadth ratio of connected domain, λ
0Be the eigenvalue of maximum of covariance matrix, λ
1Second Largest Eigenvalue for covariance matrix.
Step S220 obtains the pixel coordinate in marked region, produces the attitude of marked region according to this pixel coordinate.
Concrete, in the present embodiment, the attitude of marked region is the attitude vector.As shown in Figure 8, build the 3-D view coordinate system, this coordinate is right-handed coordinate system.In this coordinate system, establish space vector OP, P is at the p that is projected as of plane X OY, and the attitude vector with polar coordinate representation vector OP is [α, θ]
T, α is angle XOp, and namely X-axis is to the Op angle, and span is 0 to 360 degree, and θ is angle pOP, i.e. the angle of OP and XOY plane, span be-90 to spend to 90 and spend.If 2 on the space ray in this coordinate system is A (x1, y1, z1) and B (x2, y2, z2), this attitude of 2 vector [α, θ]
TAvailable following formula is unique to be determined:
In the present embodiment, after extracting marked region, calculate the covariance matrix of the pixel coordinate in marked region, obtain covariance matrix eigenvalue of maximum characteristic of correspondence vector, and this proper vector is converted to the attitude vector.Concrete, establish the attitude vector that obtains and be [dir
x, dir
y, dir
z]
T, wherein, dir
xRepresent 2 distances on the x direction of principal axis, dir
yRepresent 2 distances on the y direction of principal axis, dir
zRepresent 2 distances on the z direction of principal axis.Can think has two points on the ray of this attitude vector description, namely (0,0,0) and
Be that ray triggers from initial point, point to (dir
x, dir
y, dir
z), attitude angle need satisfy above-mentioned formula (1) and (2), makes the x1=0 in above-mentioned formula (1) and (2), y1=0, z1=0, x2=dir
x, y2=dir
y, z2=dir
z, can obtain attitude vector [α, θ]
T
In one embodiment, marked region is a continuum, the process that produces the attitude of marked region according to pixel coordinate is: the covariance matrix that calculates pixel coordinate, obtain covariance matrix eigenvalue of maximum characteristic of correspondence vector, produce the attitude of marked region according to proper vector.As mentioned above, the attitude of this marked region is an attitude vector.
In another embodiment, marked region comprises the first continuum and the second continuum, the detailed process that produces the attitude of marked region according to described pixel coordinate is: calculate the center of gravity of the first continuum and the center of gravity of the second continuum according to pixel coordinate, calculate the attitude of marked region according to the pixel coordinate of the center of gravity of the pixel coordinate of the center of gravity of the first continuum and the second continuum.As shown in Figure 8, in the present embodiment, the pixel coordinate in marked region is three-dimensional coordinate, and is concrete, can produce the attitude of marked region according to the pixel coordinate of the center of gravity of two continuums that calculate, and this attitude is an attitude vector.
In one embodiment, also can comprise before the step of the attitude in identification marking zone: the image that judgement gathers is two dimensional image or the step of 3-D view.Concrete, if the image that gathers is two dimensional image, carry out above-mentioned steps S202 to step S204, if the image that gathers is 3-D view, carry out above-mentioned steps S210 to S220.
As shown in Figure 9, in one embodiment, the detailed process of above-mentioned steps S30 comprises:
Step S302 obtains the attitude of this marked region in current frame image.
As mentioned above, the attitude of obtaining in step S302 can be the attitude (being attitude angle) of the marked region in the two dimensional image of present frame, can be also the attitude (being the attitude vector) of the marked region in the three-dimensional dark image of present frame.In the present embodiment, the mapping relations between attitude and steering order have been preset.This attitude also can be described as absolute attitude.
Step S304, the steering order corresponding with this attitude with the mapping relations generation between steering order according to default attitude.
For example, steering order is left mouse button instruction and right button instruction.Take two dimensional image as example, the span of attitude angle is that-180 degree are to 180 degree.Can preset attitude angle in current frame image in the scope of (a, b), trigger the left button instruction, the attitude angle in current frame image triggers the right button instruction in the scope of (c, d).Wherein, a, b, c, d are predefined angle, satisfy a＜b, c＜d, and the common factor of set [a, b] and set [c, d] is empty.
In addition, in 3-D view, the attitude that identifies comprises two attitude angle, can obtain steering order with one of them attitude angle, also can obtain steering order with two attitude angle.Use Method And Principle and the two dimensional image of one of them attitude angle similar, repeat no more at this.When using two attitude angle, if two attitude angle can be set all in predefined instruction triggers scope the time, just trigger steering order.
As shown in figure 10, in another embodiment, the image that comprises marked region of collection is image sequence, and the detailed process of above-mentioned steps S30 comprises:
Step S310 obtains the relative attitude between the attitude of the attitude of this marked region in current frame image and this marked region in the previous frame image.
In the present embodiment, but the image sequence that Real-time Collection is comprised of a plurality of images that comprise marked region.As mentioned above, the attitude of obtaining in step S310 can be the attitude angle of the marked region in current frame image and previous frame image, can be also the attitude vector of the marked region in current frame image and previous frame image.Relative attitude between attitude in attitude in current frame image and previous frame image is both differences.
Step S320, the steering order corresponding with this relative attitude with the mapping relations generation between steering order according to default relative attitude.
For example, take two dimensional image as example, relative attitude is the relative attitude angle, the attitude angle that can preset current frame image is spent greater than 30 than the attitude angle increase of previous frame, be relative attitude angle when spending greater than 30, trigger the instruction that the roller of mouse rolls counterclockwise, the attitude angle of current frame image reduces when spending greater than 40 than the attitude angle of previous frame, be relative attitude angle when spending less than-40, trigger the instruction that the roller of mouse rolls clockwise.The principle of 3-D view is similar with it, repeats no more at this.
In 3-D view, the attitude that identifies comprises two attitude angle, can obtain steering order with one of them attitude angle, also can obtain steering order with two attitude angle.Use Method And Principle and the two dimensional image of one of them attitude angle similar, repeat no more at this.When using two attitude angle, change and all satisfy when pre-conditioned if two attitude angle can be set, for example first attitude angle changes greater than predefined first threshold, and second attitude angle changes greater than predefined Second Threshold, triggers steering order.
In one embodiment, as shown in figure 11, a kind of system that realizes shortcut function comprises image capture module 10, gesture recognition module 20, instruction generation module 30 and instruction transformation module 40, wherein:
In the present embodiment, marked region is a zone in the image that gathers, and this zone can be formed by interactive device.Concrete, in one embodiment, interactive device can be hand-held device, part or all of hand-held device can be set as color or the shape of appointment, gather the image of hand-held device, this designated color in the hand-held device in image or the part of shape form marked region.In addition, interactive device can also be the hand-held device of tape label, namely attach the mark (as reflectorized material) of designated color or shape on hand-held device, gather the image of hand-held device, on the hand-held device in image, the mark of incidental designated color or shape forms marked region.
In another embodiment, interactive device can also be human body (such as people's face, palm, arm etc.), gathers the image of human body, and the human body in image forms marked region.In addition, interactive device can also be the human body of tape label, namely attaches the mark (as reflectorized material) of designated color or shape on human body, and when gathering the image of human body, this designated color in image or the mark of shape form marked region.
Concrete, the image that collects is processed, extract the marked region in image, then obtain the attitude of marked region according to the pixel coordinate of the pixel in marked region in the image coordinate system that builds.So-called attitude refers to marked region formed posture state in image.Further, in two dimensional image, attitude is marked region and the angle between predeterminated position, the i.e. attitude angle in two dimensional image; In 3-D view, attitude is the vector that marked region in two dimensional image and a plurality of attitude angle between predeterminated position form, i.e. the attitude vector." attitude that marked region produces " said in the present invention, " attitude of marked region " all refers to described attitude, namely the attitude angle of different embodiment and attitude vector.
In the present embodiment, preset the attitude of marked region and the mapping relations between steering order, and these mapping relations are stored in the database (not shown).After identifying the attitude of marked region, the attitude that instruction generation module 30 can be used for identifying according to gesture recognition module 20 is searched the steering order corresponding with attitude from database.
The input instruction is the instruction that operating system can be identified.For example, order line input instruction " dir/p " expression split screen is showed file and the sub-directory under current directory; The current file of choosing of the input instruction " deletion " of right button popup menu expression deletion after the mouse filesselected.
The mapping relations of steering order and input instruction are for presetting, and store with the form of key-value pair.After the steering order that gets generation, instruction transformation module 40 is used for finding the input instruction corresponding with this steering order according to this key-value pair, then sends to instruction process system.For example, shown the attitude of L when user's limb action, when having generated corresponding steering order A001, find corresponding with it input instruction and be " del*.txt " (deleting the file of all expansions txt by name) in default key-value pair, then " del*.txt " sent to instruction process system, instruction process system is carried out the file of all suffix of deletion txt by name by parsing.
The above-mentioned system that realizes shortcut function, go out according to the image recognition that comprises marked region that collects the attitude that marked region produces, and generate steering order corresponding to attitude, then the mapping relations according to default steering order and input instruction convert this steering order to input instruction that instruction process system can be identified.Owing to judging movement locus and the attitude of limb action by the image that obtains limb action, therefore, the user only need to make corresponding limb action just can complete quick input, need to not use keyboard or mouse input, and makes that the user's is more convenient to operate.
In one embodiment, as shown in figure 16, the system that realizes shortcut also comprises mapping configuration module 50, be used for after generating steering order corresponding to attitude, detect whether the configuration-direct input is arranged, when the configuration-direct input having been detected, obtain the input instruction, and generate the mapping relations of steering order and input instruction according to steering order and input instruction; When the configuration-direct input not detected, call instruction modular converter 40.
Further, mapping configuration module 50 can be used for showing the visual configuration interface, obtains the input instruction by the visual configuration interface.
For example, user's limb action has been shown the attitude that opens arms, then generate steering order 002 by instruction generation module 30, mapping configuration module 50 can be used for detecting the user and whether is triggering configuration-direct by pressing button default on telepilot or keyboard before.When mapping configuration module 50 has detected the configuration-direct input, show visual input instruction input frame, obtain the input instruction of character string type, also can obtain the cursor track type input instruction that in input frame, cursor movement forms, then with steering order 002 and the form storage of the input instruction that gets with key-value pair.
In one embodiment, the input instruction is many.Mapping configuration module 50 also is used for inputting instruction and generates the input instruction sequence according to the time sequencing of inputting, and generates steering order and the mapping relations of inputting instruction sequence.
For example, can obtain by the visual configuration interface many input instruction B001, B002, the B003......B00N of user input, and will input instruction and be arranged in the input instruction sequence according to the time sequencing of inputting.Then by steering order and input instruction sequence are generated the mapping relations of steering order and input instruction with the form storage of key-value pair.When configuration finishes, when obtaining the input instruction according to steering order, first obtain the input instruction sequence corresponding with it according to steering order, extract the input instruction and send to instruction process system by the order of inputting from the input instruction sequence, instruction process system namely carries out by the time sequencing of input the input instruction of inputting in instruction sequence.
Further, the mapping configuration module also can be used for the form storage of input instruction sequence with script.
Adopt script storage input instruction sequence, can facilitate the user to preserve and transmission configuration.On the user can upload onto the server the script that configures, and the synchronizeing of version on maintenance and server.When wishing on a plurality of equipment as the user to adopt identical configuration, can directly download this configuration on server, thereby the mapping relations that make the user avoid frequently configuring steering order and input instruction are more convenient to operate.
Further, script also comprises sequence control instruction.Mapping configuration module 50 also is used for obtaining order and describes instruction, describes instruction according to order and will input instruction and be arranged in the input instruction sequence.
Order is described instruction and is comprised that looping discription instruction, branch describe instruction.For example, create fast 10000 files if wish by the limbs attitude, the filename of the file that creates is natural number to be arranged, and can limit the input instruction that creates file with the looping discription instruction, and the file name parameter that each circulation time will create in the input instruction of file adds 1.Added sequentially instruction is described after, the user carries out a plurality of input instructions when the mapping relations of configuration steering order and input instruction according to the order of appointment, facilitate the user to be configured, and simultaneously, has also expanded input instruction corresponding to steering order.
As shown in figure 12, in one embodiment, the image that image capture module 10 collects is two dimensional image, and gesture recognition module 20 comprises the first image processing module 202 and the first attitude generation module 204, wherein:
The first image processing module 202 is used for extracting image and the pixel that default color model mates, and the pixel of obtaining is carried out connected domain detect, and extracts the marked region that detects in the connected domain that obtains.
Concrete, image capture module 10 can be video camera, and its image that collects can be the two-dimensional visible light image.Preferably, also can add infrared fileter before the camera lens of video camera, be used for elimination except other wave band light of infrared band, the image of image capture module 10 collections is the two-dimensional infrared image.In visible images, the object in scene can form the identification of marked region and disturb, and infrared image has been because having filtered out visible light information, disturbs lessly, so the two-dimensional infrared image more is conducive to extract marked region.
Concrete, the first image processing module 202 is used for setting up in advance color model.For example the color of marked region is red, sets up in advance red model, and in this model, the rgb value component of pixel can be between 200 to 255, and G, B component can be close to zero; The first 202 of image processing modules are used for obtaining the pixel that two field picture satisfies the rgb value of this redness model and are red pixel.In addition, when forming marked region by human body in the image that gathers, the first 202 of image processing modules are for obtaining the pixel of image with default complexion model coupling.The first image processing module 202 is used for that also the pixel of obtaining is carried out connected domain and detects, and obtains a plurality of connected domains, if connected domain is the set that individual continuous pixel forms.
In the present embodiment, because the size and shape of marked region should be roughly changeless, the first image processing module 202 is carrying out connected domain when detecting to the pixel of obtaining, can calculate girth and/or the area of all connected domains in the pixel of obtaining.Concrete, the girth of connected domain can be the number of connected domain boundary pixel, and the area of connected domain can be the number of the whole pixels in connected domain.Further, the first image processing module 202 can be used for the girth of the connected domain that will obtain and/or girth and/or the area of area and default marked region compares, and obtains the girth that satisfies default marked region and/or the connected domain of area and is marked region.Preferably, the first image processing module 202 also can be used for girth square with the ratio of area as judgment criterion, this ratio of connected domain satisfies this ratio of default marked region, this connected domain is marked region.
The first attitude generation module 204 is used for obtaining the pixel coordinate of marked region, produces the attitude of marked region according to this pixel coordinate.
In the present embodiment, the attitude that marked region produces is attitude angle.In one embodiment, marked region is a continuum, the first attitude generation module 204 is used for calculating the covariance matrix of pixel coordinate, obtain covariance matrix eigenvalue of maximum characteristic of correspondence vector, produce the attitude of marked region according to proper vector, the attitude of this marked region is an attitude angle.
In another embodiment, marked region comprises the first continuum and the second continuum, the first attitude generation module 204 is used for calculating the center of gravity of the first continuum and the center of gravity of the second continuum according to pixel coordinate, calculates the attitude of marked region according to the pixel coordinate of the center of gravity of the pixel coordinate of the center of gravity of the first continuum and the second continuum.Concrete, calculate the mean value of the whole pixel coordinates in the continuum, resulting pixel coordinate is the center of gravity of continuum.
In another embodiment, the image that collects of image capture module 10 is 3-D view.Concrete, image capture module 10 can adopt traditional stereo visual system (being comprised of two known video camera and related softwares in control position), structured-light system (a right video camera, light source and related software form) or TOF (time of flight, flight time) depth camera to realize collection 3-D view (being the three dimensional depth image).
In the present embodiment, as shown in figure 13, gesture recognition module 20 comprises the second image processing module 210 and the second attitude generation module 220, wherein:
The second image processing module 210 is used for described Image Segmentation Using, extract the connected domain in image, and the property value of calculating connected domain, the property value of connected domain and default marked region property value are compared, and described marked region is the connected domain that meets described default marked region property value.
Concrete, the second image processing module 210 is used for when two adjacent pixel depths of 3-D view differ less than predefined threshold value, for example 5 centimetres, thinks that two pixels are communicated with, whole image is carried out connected domain detect, can obtain comprising a series of connected domains of mark connected domain.
In the present embodiment, the property value of connected domain comprises the size and dimension of connected domain.Concrete, the second image processing module 210 is used for calculating the size/shape of connected domain, compares with the size/shape of mark on interactive device, and the connected domain that obtains meeting the size/shape of mark is the connected domain (marked region) of marked region.Take rectangle marked as example, be to be rectangle in the image that is marked at collection on interactive device, the length of pre-set mark and width, the second 210 of image processing modules are used for calculating length and the width of physical region corresponding to connected domain, length and the width of this length and width and mark are more approaching, and connected domain is more similar to marked region.
Further, the second image processing module 210 is as follows for the process of the length of calculating physical region corresponding to connected domain and width: calculate the covariance matrix of the three-dimensional coordinate of connected domain pixel, adopt following formula to calculate length and the width of physical region corresponding to connected domain:
Wherein, k is predefined coefficient, for example is made as 4, and when λ was the covariance matrix eigenvalue of maximum, l was the length of connected domain, and when λ was the second largest eigenwert of covariance matrix, l was the width of connected domain.
Further, the second image processing module 210 also can be used for presetting the length breadth ratio of rectangle marked, for example length breadth ratio is 2, the length breadth ratio of physical region corresponding to connected domain is more close to the length breadth ratio of the rectangle marked of default settings, connected domain is more similar to marked region, concrete, attribute matching module 234 is used for adopting following formula to calculate the length breadth ratio of physical region corresponding to connected domain:
Wherein, r is the length breadth ratio of connected domain, λ
0Be the eigenvalue of maximum of covariance matrix, λ
1Second Largest Eigenvalue for covariance matrix.
The second attitude generation module 220 is used for obtaining the pixel coordinate of marked region, produces the attitude of marked region according to described pixel coordinate.
In the present embodiment, the attitude of marked region is the attitude vector.In one embodiment, marked region is a continuum, the second attitude generation module 220 is used for calculating the covariance matrix of pixel coordinate, obtains covariance matrix eigenvalue of maximum characteristic of correspondence vector, produces the attitude of marked region according to proper vector.As mentioned above, the attitude of this marked region is an attitude vector.
In another embodiment, marked region comprises the first continuum and the second continuum, the second attitude generation module 220 is used for calculating the center of gravity of the first continuum and the center of gravity of the second continuum according to pixel coordinate, produces the attitude of marked region according to the pixel coordinate of the center of gravity of the pixel coordinate of the center of gravity of the first continuum and the second continuum.In the present embodiment, the pixel coordinate in marked region is three-dimensional coordinate, and is concrete, can produce the attitude of marked region according to the pixel coordinate of the center of gravity of two continuums that calculate, and this attitude is an attitude vector.
In one embodiment, gesture recognition module 20 also comprises the judge module (not shown), and the image that is used for the judgement collection is two dimensional image or 3-D view.Concrete, in the present embodiment, when the image that determines collection when judge module is two dimensional image, the marked region of notifying the first image processing module 202 to extract in two dimensional images, and then produce the attitude of these marked regions by the first attitude generation module 204.When the image that determines collection when judge module is two dimensional image, the marked region of notifying the second image processing module 210 to extract in 3-D views, and then produce the attitude of these marked regions by the second attitude generation module 220.Understandable, in the present embodiment, gesture recognition module 20 comprises judge module (not shown), the first image processing module 202, the first attitude generation module 204, the second image processing module 210 and the second attitude generation module 220 simultaneously.The present embodiment both can by the attitude in two dimensional image identification marking zone, can pass through again the attitude in two dimensional image identification marking zone.
As shown in figure 14, in one embodiment, instruction generation module 30 comprises that the first attitude acquisition module 302 and the first instruction search module 304, wherein:
The first attitude acquisition module 302 is used for obtaining from gesture recognition module 20 attitude of the described marked region current frame image.
Concrete, this attitude can be the attitude angle of the marked region in the two dimensional image of present frame, can be also the attitude vector of the marked region in the three dimensional depth image of present frame.In the present embodiment, the mapping relations between attitude and steering order have been preset.This attitude also can be described as absolute attitude.
The first instruction is searched module 304 and is used for the steering order corresponding with described attitude with the mapping relations generation between steering order according to default attitude.
In the present embodiment, the image that comprises marked region that gathers can be image sequence.The first attitude acquisition module 302 is also for the relative attitude between the attitude of the attitude of obtaining the marked region current frame image from gesture recognition module 20 and the marked region in the previous frame image.The first instruction is searched module 304 and also is used for the steering order corresponding with relative attitude with the mapping relations generation between steering order according to default relative attitude.
In another embodiment, the image that comprises marked region that gathers can be image sequence.As shown in figure 15, instruction generation module 30 comprises that the second attitude acquisition module 310 and the second instruction search module 320, wherein:
The second attitude acquisition module is for the relative attitude between the attitude of the attitude of obtaining the marked region current frame image from gesture recognition module 20 and the marked region in the previous frame image.
The second instruction is searched module 320 and is used for the steering order corresponding with relative attitude with the mapping relations generation between steering order according to default relative attitude.
The above embodiment has only expressed several embodiment of the present invention, and it describes comparatively concrete and detailed, but can not therefore be interpreted as the restriction to the scope of the claims of the present invention.Should be pointed out that for the person of ordinary skill of the art, without departing from the inventive concept of the premise, can also make some distortion and improvement, these all belong to protection scope of the present invention.Therefore, the protection domain of patent of the present invention should be as the criterion with claims.
Claims (14)
1. method that realizes shortcut function comprises the following steps:
Collection comprises the image of marked region;
The attitude in identification marking zone;
Generate steering order corresponding to described attitude;
Obtain the input instruction corresponding with described steering order with the mapping relations of input instruction, and send to instruction process system according to described steering order, default steering order.
2. the method that realizes shortcut function according to claim 1, is characterized in that, steering order corresponding to the described attitude of described generation also comprises afterwards:
Detect whether the configuration-direct input is arranged;
When the configuration-direct input having been detected, obtain the input instruction, and generate described steering order and the mapping relations of inputting instruction according to described steering order and described input instruction;
When the configuration-direct input not detected, obtain the input instruction corresponding with described steering order with the mapping relations of input instruction, and send to instruction process system according to described steering order, default steering order.
3. the method that realizes shortcut function described according to claim 2, is characterized in that, the described step of obtaining the input instruction comprises:
Show the visual configuration interface, obtain the input instruction by described visual configuration interface.
4. the method that realizes shortcut function described according to claim 2, is characterized in that, described input instruction is many;
The described step that generates described steering order and the mapping relations of input instruction according to described steering order and described input instruction comprises:
Described input instruction is generated the input instruction sequence according to the time sequencing of obtaining, and generate the mapping relations of described steering order and described input instruction sequence.
5. the method that realizes shortcut function described according to claim 4, is characterized in that, the described step of obtaining with input instruction corresponding to described steering order according to described steering order, default steering order and the mapping relations of input instruction comprises:
Obtain the input instruction sequence corresponding with described steering order according to described steering order, described steering order with the mapping relations of input instruction sequence, extract the input instruction by the described time sequencing of obtaining from described input instruction sequence.
6. the method that realizes shortcut function described according to claim 5, is characterized in that, also comprises after the described step that will described input instruction generates the input instruction sequence according to the time sequencing of obtaining:
With the form storage of described input instruction sequence with script.
7. the method that realizes shortcut function according to claim 6, is characterized in that, the described step of obtaining with input instruction sequence corresponding to described steering order according to described steering order, described steering order and the mapping relations of input instruction sequence comprises:
Obtain the input instruction sequence corresponding with described steering order according to described steering order, described steering order with the mapping relations of input instruction sequence from script.
8. a system that realizes shortcut function, is characterized in that, comprises with lower module:
Image capture module is used for gathering the image that comprises marked region;
The gesture recognition module is for the attitude in identification marking zone;
The instruction generation module is used for generating steering order corresponding to described attitude;
The instruction transformation module is used for obtaining the input instruction corresponding with described steering order with the mapping relations of input instruction, and sending to instruction process system according to described steering order, default steering order.
9. the system that realizes shortcut function according to claim 8, is characterized in that, also comprises the mapping configuration module, for detection of whether the configuration-direct input is arranged; When the configuration-direct input having been detected, obtain the input instruction, and generate described steering order and the mapping relations of inputting instruction according to described steering order and described input instruction; When the configuration-direct input not detected, the call instruction modular converter.
10. the system that realizes shortcut function described according to claim 9, is characterized in that, described mapping configuration module also is used for showing the visual configuration interface, obtains the input instruction by described visual configuration interface.
11. the system that realizes shortcut function according to claim 9 is characterized in that, described input instruction is many;
Described mapping configuration module also is used for described input instruction is generated the input instruction sequence according to the time sequencing of obtaining, and generates the mapping relations of described steering order and described input instruction sequence.
12. the system that realizes shortcut function according to claim 11, it is characterized in that, described instruction transformation module also is used for obtaining the input instruction sequence corresponding with described steering order according to described steering order, described steering order with the mapping relations of input instruction sequence, extracts the input instruction by the described time sequencing of obtaining from described input instruction sequence.
13. the system that realizes shortcut function according to claim 12 is characterized in that, described mapping configuration module also is used for the form storage of described input instruction sequence with script.
14. the system that realizes shortcut function according to claim 13, it is characterized in that, described instruction transformation module also is used for obtaining with described steering order corresponding input instruction sequence with the mapping relations of input instruction sequence from script according to described steering order, described steering order.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201210032932.5A CN103135758B (en) | 2011-12-02 | 2012-02-14 | Realize the method and system of shortcut function |
Applications Claiming Priority (4)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2011103962353 | 2011-12-02 | ||
| CN201110396235.3 | 2011-12-02 | ||
| CN201110396235 | 2011-12-02 | ||
| CN201210032932.5A CN103135758B (en) | 2011-12-02 | 2012-02-14 | Realize the method and system of shortcut function |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN103135758A true CN103135758A (en) | 2013-06-05 |
| CN103135758B CN103135758B (en) | 2016-09-21 |
Family
ID=48488552
Family Applications (12)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201110451724.4A Active CN103135754B (en) | 2011-12-02 | 2011-12-29 | Adopt interactive device to realize mutual method |
| CN201110453879.1A Active CN103135756B (en) | 2011-12-02 | 2011-12-29 | Generate the method and system of control instruction |
| CN201110451741.8A Active CN103135755B (en) | 2011-12-02 | 2011-12-29 | Interactive system and method |
| CN201210011308.7A Active CN103135881B (en) | 2011-12-02 | 2012-01-13 | Display control method and system |
| CN201210011346.2A Active CN103135882B (en) | 2011-12-02 | 2012-01-13 | Control the method and system that window picture shows |
| CN201210023419XA Pending CN103139508A (en) | 2011-12-02 | 2012-02-02 | Method and system for controlling display of television pictures |
| CN201210024389.4A Active CN103127717B (en) | 2011-12-02 | 2012-02-03 | The method and system of control operation game |
| CN201210024483.XA Active CN103135883B (en) | 2011-12-02 | 2012-02-03 | Control the method and system of window |
| CN201210025300.6A Active CN103135453B (en) | 2011-12-02 | 2012-02-06 | Control method and system of household appliances |
| CN201210031595.8A Active CN103136986B (en) | 2011-12-02 | 2012-02-13 | Sign Language Recognition Method and system |
| CN201210032932.5A Active CN103135758B (en) | 2011-12-02 | 2012-02-14 | Realize the method and system of shortcut function |
| CN201210032934.4A Active CN103135759B (en) | 2011-12-02 | 2012-02-14 | Control method for playing multimedia and system |
Family Applications Before (10)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201110451724.4A Active CN103135754B (en) | 2011-12-02 | 2011-12-29 | Adopt interactive device to realize mutual method |
| CN201110453879.1A Active CN103135756B (en) | 2011-12-02 | 2011-12-29 | Generate the method and system of control instruction |
| CN201110451741.8A Active CN103135755B (en) | 2011-12-02 | 2011-12-29 | Interactive system and method |
| CN201210011308.7A Active CN103135881B (en) | 2011-12-02 | 2012-01-13 | Display control method and system |
| CN201210011346.2A Active CN103135882B (en) | 2011-12-02 | 2012-01-13 | Control the method and system that window picture shows |
| CN201210023419XA Pending CN103139508A (en) | 2011-12-02 | 2012-02-02 | Method and system for controlling display of television pictures |
| CN201210024389.4A Active CN103127717B (en) | 2011-12-02 | 2012-02-03 | The method and system of control operation game |
| CN201210024483.XA Active CN103135883B (en) | 2011-12-02 | 2012-02-03 | Control the method and system of window |
| CN201210025300.6A Active CN103135453B (en) | 2011-12-02 | 2012-02-06 | Control method and system of household appliances |
| CN201210031595.8A Active CN103136986B (en) | 2011-12-02 | 2012-02-13 | Sign Language Recognition Method and system |
Family Applications After (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201210032934.4A Active CN103135759B (en) | 2011-12-02 | 2012-02-14 | Control method for playing multimedia and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (12) | CN103135754B (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107548317A (en) * | 2015-04-29 | 2018-01-05 | 微软技术许可有限责任公司 | Remapped via the game console function of external accessory |
| CN111354072A (en) * | 2018-12-24 | 2020-06-30 | 山卫科技股份有限公司 | System and method for measuring scoliosis |
| CN113326849A (en) * | 2021-07-20 | 2021-08-31 | 广东魅视科技股份有限公司 | Visual data acquisition method and system |
Families Citing this family (39)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104349197B (en) * | 2013-08-09 | 2019-07-26 | 联想(北京)有限公司 | A kind of data processing method and device |
| JP5411385B1 (en) * | 2013-08-12 | 2014-02-12 | 株式会社 ディー・エヌ・エー | Server and method for providing game |
| CN104801042A (en) * | 2014-01-23 | 2015-07-29 | 鈊象电子股份有限公司 | Method for switching game screens based on player's hand waving range |
| CN103810922B (en) * | 2014-01-29 | 2016-03-23 | 上海天昊信息技术有限公司 | Sign language interpretation system |
| CN103902045A (en) * | 2014-04-09 | 2014-07-02 | 深圳市中兴移动通信有限公司 | Method and device for operating wallpaper via non-contact postures |
| CN105094785A (en) * | 2014-05-20 | 2015-11-25 | 腾讯科技（深圳）有限公司 | Method and device for generating color matching file |
| CN104391573B (en) * | 2014-11-10 | 2017-05-03 | 北京华如科技股份有限公司 | Method and device for recognizing throwing action based on single attitude sensor |
| CN104460988B (en) * | 2014-11-11 | 2017-12-22 | 陈琦 | A kind of input control method of smart mobile phone virtual reality device |
| KR101608172B1 (en) | 2014-12-22 | 2016-03-31 | 주식회사 넥슨코리아 | Device and method to control object |
| CN106139590B (en) * | 2015-04-15 | 2019-12-03 | 乐线韩国股份有限公司 | The method and apparatus of control object |
| CN105204354A (en) * | 2015-09-09 | 2015-12-30 | 北京百度网讯科技有限公司 | Smart home device control method and device |
| WO2017057107A1 (en) * | 2015-09-28 | 2017-04-06 | 日本電気株式会社 | Input device, input method, and program |
| CN105892638A (en) * | 2015-12-01 | 2016-08-24 | 乐视致新电子科技（天津）有限公司 | Virtual reality interaction method, device and system |
| CN106896732B (en) * | 2015-12-18 | 2020-02-04 | 美的集团股份有限公司 | Display method and device of household appliance |
| CN105592367A (en) * | 2015-12-23 | 2016-05-18 | 青岛海信电器股份有限公司 | Image display parameter adjusting method and system |
| JP6370820B2 (en) * | 2016-02-05 | 2018-08-08 | 株式会社バンダイナムコエンターテインメント | Image generation system, game device, and program. |
| CN105760106B (en) * | 2016-03-08 | 2019-01-15 | 网易（杭州）网络有限公司 | A kind of smart home device exchange method and device |
| CN105930050B (en) * | 2016-04-13 | 2018-01-26 | 腾讯科技（深圳）有限公司 | Behavior determines method and device |
| CN106682593A (en) * | 2016-12-12 | 2017-05-17 | 山东师范大学 | Method and system for sign language conference based on gesture recognition |
| WO2018120657A1 (en) * | 2016-12-27 | 2018-07-05 | 华为技术有限公司 | Method and device for sharing virtual reality data |
| CN108668042B (en) * | 2017-03-30 | 2021-01-15 | 富士施乐实业发展（中国）有限公司 | Compound machine system |
| CN109558000B (en) | 2017-09-26 | 2021-01-22 | 京东方科技集团股份有限公司 | Man-machine interaction method and electronic equipment |
| CN107831996B (en) * | 2017-10-11 | 2021-02-19 | Oppo广东移动通信有限公司 | Face recognition starting method and related product |
| CN107861682A (en) * | 2017-11-03 | 2018-03-30 | 网易（杭州）网络有限公司 | The control method for movement and device of virtual objects |
| CN108228251B (en) * | 2017-11-23 | 2021-08-27 | 腾讯科技（上海）有限公司 | Method and device for controlling target object in game application |
| CN108036479A (en) * | 2017-12-01 | 2018-05-15 | 广东美的制冷设备有限公司 | Control method, system, vision controller and the storage medium of air conditioner |
| CN110007748B (en) * | 2018-01-05 | 2021-02-19 | Oppo广东移动通信有限公司 | Terminal control method, processing device, storage medium and terminal |
| WO2019153971A1 (en) * | 2018-02-06 | 2019-08-15 | 广东虚拟现实科技有限公司 | Visual interaction apparatus and marker |
| CN108765299B (en) * | 2018-04-26 | 2022-08-16 | 广州视源电子科技股份有限公司 | Three-dimensional graphic marking system and method |
| CN108693781A (en) * | 2018-07-31 | 2018-10-23 | 湖南机电职业技术学院 | Intelligent home control system |
| JP7262976B2 (en) * | 2018-11-02 | 2023-04-24 | キヤノン株式会社 | Information processing device, information processing method and program |
| CN109711349B (en) * | 2018-12-28 | 2022-06-28 | 百度在线网络技术（北京）有限公司 | Method and device for generating control instruction |
| CN109816650B (en) * | 2019-01-24 | 2022-11-25 | 强联智创（北京）科技有限公司 | Target area identification method and system based on two-dimensional DSA image |
| CN111665727A (en) * | 2019-03-06 | 2020-09-15 | 北京京东尚科信息技术有限公司 | Method and device for controlling household equipment and household equipment control system |
| CN111803930A (en) * | 2020-07-20 | 2020-10-23 | 网易（杭州）网络有限公司 | Multi-platform interaction method and device and electronic equipment |
| CN115623254A (en) * | 2021-07-15 | 2023-01-17 | 北京字跳网络技术有限公司 | Video effect adding method, device, equipment and storage medium |
| CN113499585A (en) * | 2021-08-09 | 2021-10-15 | 网易（杭州）网络有限公司 | In-game interaction method and device, electronic equipment and storage medium |
| CN113822186A (en) * | 2021-09-10 | 2021-12-21 | 阿里巴巴达摩院(杭州)科技有限公司 | Sign language translation, customer service, communication method, device and readable medium |
| CN113822187A (en) * | 2021-09-10 | 2021-12-21 | 阿里巴巴达摩院(杭州)科技有限公司 | Sign language translation, customer service, communication method, device and readable medium |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20080089587A1 (en) * | 2006-10-11 | 2008-04-17 | Samsung Electronics Co.; Ltd | Hand gesture recognition input system and method for a mobile phone |
| CN101636711A (en) * | 2007-01-30 | 2010-01-27 | 苹果公司 | Gesturing with a multipoint sensing device |
| CN102067067A (en) * | 2008-06-24 | 2011-05-18 | 韩国电子通信研究院 | Registration method of reference gesture data, driving method of mobile terminal, and mobile terminal thereof |
| CN102262438A (en) * | 2010-05-18 | 2011-11-30 | 微软公司 | Gestures and gesture recognition for manipulating a user-interface |
Family Cites Families (31)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5594469A (en) * | 1995-02-21 | 1997-01-14 | Mitsubishi Electric Information Technology Center America Inc. | Hand gesture machine control system |
| JPH0918708A (en) * | 1995-06-30 | 1997-01-17 | Omron Corp | Image processing method, image input device, controller, image output device and image processing system using the method |
| KR19990011180A (en) * | 1997-07-22 | 1999-02-18 | 구자홍 | How to select menu using image recognition |
| US8062126B2 (en) * | 2004-01-16 | 2011-11-22 | Sony Computer Entertainment Inc. | System and method for interfacing with a computer program |
| US10279254B2 (en) * | 2005-10-26 | 2019-05-07 | Sony Interactive Entertainment Inc. | Controller having visually trackable object for interfacing with a gaming system |
| WO2005031552A2 (en) * | 2003-09-30 | 2005-04-07 | Koninklijke Philips Electronics, N.V. | Gesture to define location, size, and/or content of content window on a display |
| US7379563B2 (en) * | 2004-04-15 | 2008-05-27 | Gesturetek, Inc. | Tracking bimanual movements |
| JP2006068315A (en) * | 2004-09-02 | 2006-03-16 | Sega Corp | Pause detection program, video game device, pause detection method, and computer-readable recording medium recorded with program |
| CN100345085C (en) * | 2004-12-30 | 2007-10-24 | 中国科学院自动化研究所 | Method for controlling electronic game scene and role based on poses and voices of player |
| US8726194B2 (en) * | 2007-07-27 | 2014-05-13 | Qualcomm Incorporated | Item selection using enhanced control |
| CN101388138B (en) * | 2007-09-12 | 2011-06-29 | 原相科技股份有限公司 | Interaction image system, interaction apparatus and operation method thereof |
| CN101398896B (en) * | 2007-09-28 | 2012-10-17 | 三星电子株式会社 | Device and method for extracting color characteristic with strong discernment for image forming apparatus |
| JP4938617B2 (en) * | 2007-10-18 | 2012-05-23 | 幸輝郎 村井 | Object operating device and method for specifying marker from digital image frame data |
| CN101483005A (en) * | 2008-01-07 | 2009-07-15 | 致伸科技股份有限公司 | Remote control device for multimedia file playing |
| JP5174492B2 (en) * | 2008-03-05 | 2013-04-03 | 公立大学法人首都大学東京 | Image recognition apparatus, image recognition method, image recognition program, gesture motion recognition system, gesture motion recognition method, and gesture motion recognition program |
| CN101551700B (en) * | 2008-03-31 | 2012-03-28 | 联想(北京)有限公司 | Electronic game input device, electronic game machine and electronic game input method |
| EP2266016A4 (en) * | 2008-04-02 | 2014-10-29 | Oblong Ind Inc | Gesture based control using three-dimensional information extracted over an extended depth of field |
| CN101504728B (en) * | 2008-10-10 | 2013-01-23 | 深圳泰山在线科技有限公司 | Remote control system and method of electronic equipment |
| CN101729808B (en) * | 2008-10-14 | 2012-03-28 | Tcl集团股份有限公司 | Remote control method for television and system for remotely controlling television by same |
| CN101465116B (en) * | 2009-01-07 | 2013-12-11 | 北京中星微电子有限公司 | Display equipment and control method thereof |
| CN101504586A (en) * | 2009-03-25 | 2009-08-12 | 中国科学院软件研究所 | Instruction method based on stroke tail gesture |
| CN101527092A (en) * | 2009-04-08 | 2009-09-09 | 西安理工大学 | Computer assisted hand language communication method under special session context |
| CN101539994B (en) * | 2009-04-16 | 2012-07-04 | 西安交通大学 | Mutually translating system and method of sign language and speech |
| CN101763515B (en) * | 2009-09-23 | 2012-03-21 | 中国科学院自动化研究所 | Real-time gesture interaction method based on computer vision |
| CN101673094A (en) * | 2009-09-23 | 2010-03-17 | 曾昭兴 | Control device of home appliance and control method thereof |
| US20110151974A1 (en) * | 2009-12-18 | 2011-06-23 | Microsoft Corporation | Gesture style recognition and reward |
| CN101799717A (en) * | 2010-03-05 | 2010-08-11 | 天津大学 | Man-machine interaction method based on hand action catch |
| CN101833653A (en) * | 2010-04-02 | 2010-09-15 | 上海交通大学 | Figure identification method in low-resolution video |
| CN201750431U (en) * | 2010-07-02 | 2011-02-16 | 厦门万安智能股份有限公司 | Smart home centralized control device |
| CN102179048A (en) * | 2011-02-28 | 2011-09-14 | 武汉市高德电气有限公司 | Method for implementing realistic game based on movement decomposition and behavior analysis |
| CN102226880A (en) * | 2011-06-03 | 2011-10-26 | 北京新岸线网络技术有限公司 | Somatosensory operation method and system based on virtual reality |
-
2011
- 2011-12-29 CN CN201110451724.4A patent/CN103135754B/en active Active
- 2011-12-29 CN CN201110453879.1A patent/CN103135756B/en active Active
- 2011-12-29 CN CN201110451741.8A patent/CN103135755B/en active Active
-
2012
- 2012-01-13 CN CN201210011308.7A patent/CN103135881B/en active Active
- 2012-01-13 CN CN201210011346.2A patent/CN103135882B/en active Active
- 2012-02-02 CN CN201210023419XA patent/CN103139508A/en active Pending
- 2012-02-03 CN CN201210024389.4A patent/CN103127717B/en active Active
- 2012-02-03 CN CN201210024483.XA patent/CN103135883B/en active Active
- 2012-02-06 CN CN201210025300.6A patent/CN103135453B/en active Active
- 2012-02-13 CN CN201210031595.8A patent/CN103136986B/en active Active
- 2012-02-14 CN CN201210032932.5A patent/CN103135758B/en active Active
- 2012-02-14 CN CN201210032934.4A patent/CN103135759B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20080089587A1 (en) * | 2006-10-11 | 2008-04-17 | Samsung Electronics Co.; Ltd | Hand gesture recognition input system and method for a mobile phone |
| CN101636711A (en) * | 2007-01-30 | 2010-01-27 | 苹果公司 | Gesturing with a multipoint sensing device |
| CN102067067A (en) * | 2008-06-24 | 2011-05-18 | 韩国电子通信研究院 | Registration method of reference gesture data, driving method of mobile terminal, and mobile terminal thereof |
| CN102262438A (en) * | 2010-05-18 | 2011-11-30 | 微软公司 | Gestures and gesture recognition for manipulating a user-interface |
Non-Patent Citations (3)
| Title |
|---|
| 李瑞峰 等: "基于深度图像和表观特征的手势识别", 《华中科技大学学报(自然科学报)》, vol. 39, 30 November 2011 (2011-11-30) * |
| 狄海进: "基于三维视觉的手势跟踪及人机交互中的应用", 《中国优秀硕士学位论文全文数据库》, no. 11, 15 November 2011 (2011-11-15) * |
| 邱耀德: "以單一攝影機追蹤手部與模擬多點觸控螢幕", 《台湾博硕士论文知识加值系统》, 31 December 2009 (2009-12-31) * |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107548317A (en) * | 2015-04-29 | 2018-01-05 | 微软技术许可有限责任公司 | Remapped via the game console function of external accessory |
| CN107548317B (en) * | 2015-04-29 | 2020-10-23 | 微软技术许可有限责任公司 | Game controller and method of controlling operation of game controller via external accessory |
| CN111354072A (en) * | 2018-12-24 | 2020-06-30 | 山卫科技股份有限公司 | System and method for measuring scoliosis |
| CN111354072B (en) * | 2018-12-24 | 2024-01-16 | 山卫科技股份有限公司 | Spinal lateral curvature measuring system and method |
| CN113326849A (en) * | 2021-07-20 | 2021-08-31 | 广东魅视科技股份有限公司 | Visual data acquisition method and system |
| CN113326849B (en) * | 2021-07-20 | 2022-01-11 | 广东魅视科技股份有限公司 | Visual data acquisition method and system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN103135453B (en) | 2015-05-13 |
| CN103135882B (en) | 2016-08-03 |
| CN103135756B (en) | 2016-05-11 |
| CN103135758B (en) | 2016-09-21 |
| CN103135881B (en) | 2016-12-14 |
| CN103135883B (en) | 2016-07-06 |
| CN103135453A (en) | 2013-06-05 |
| CN103135882A (en) | 2013-06-05 |
| CN103135755A (en) | 2013-06-05 |
| CN103127717A (en) | 2013-06-05 |
| CN103127717B (en) | 2016-02-10 |
| CN103136986B (en) | 2015-10-28 |
| CN103139508A (en) | 2013-06-05 |
| CN103135759A (en) | 2013-06-05 |
| CN103135755B (en) | 2016-04-06 |
| CN103135881A (en) | 2013-06-05 |
| CN103135759B (en) | 2016-03-09 |
| CN103135754A (en) | 2013-06-05 |
| CN103136986A (en) | 2013-06-05 |
| CN103135754B (en) | 2016-05-11 |
| CN103135756A (en) | 2013-06-05 |
| CN103135883A (en) | 2013-06-05 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN103135758A (en) | Method and system for achieving shortcut function | |
| US11030237B2 (en) | Method and apparatus for identifying input features for later recognition | |
| CN105807926B (en) | A kind of unmanned plane man-machine interaction method based on three-dimensional continuous dynamic hand gesture recognition | |
| US20180186452A1 (en) | Unmanned Aerial Vehicle Interactive Apparatus and Method Based on Deep Learning Posture Estimation | |
| US9734393B2 (en) | Gesture-based control system | |
| CN102184021B (en) | Television man-machine interaction method based on handwriting input and fingertip mouse | |
| CN108509026B (en) | Remote maintenance support system and method based on enhanced interaction mode | |
| CN103049761A (en) | Sign language recognition method and system based on sign language gloves | |
| CN102722249A (en) | Manipulating method, manipulating device and electronic device | |
| CN107992792A (en) | A kind of aerial handwritten Chinese character recognition system and method based on acceleration transducer | |
| CN105160323A (en) | Gesture identification method | |
| CN103995595A (en) | Game somatosensory control method based on hand gestures | |
| CN113672099A (en) | Electronic equipment and interaction method thereof | |
| CN105302294A (en) | Interactive virtual reality presentation device | |
| KR101512239B1 (en) | System and method for transfering content among devices using touch command and unusual touch | |
| KR20110097504A (en) | User motion perception method and apparatus | |
| CN112488059B (en) | Spatial gesture control method based on deep learning model cascade | |
| Choondal et al. | Design and implementation of a natural user interface using hand gesture recognition method | |
| KR20160023417A (en) | Non-contact multi touch recognition method and system using color image analysis | |
| Quraishi et al. | A novel human hand finger gesture recognition using machine learning | |
| CN109218833A (en) | The method and system that control television image is shown | |
| Dudhapachare et al. | Voice Guided, Gesture Controlled Virtual Mouse | |
| CN106484087A (en) | A kind of Portable telemetry body-sensing input method and device | |
| CN114627313A (en) | Distribution task control method, robot, and storage medium | |
| Nan et al. | Finger Recognition and Tracking Algorithm Based on Single Optical Camera |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information |
Address after: 518000 Shenzhen, Nanshan District Province, science and technology, South Road, the building of the big square, building 02, room 4, Applicant after: SHENZHEN TAISHAN SPORTS TECHNOLOGY CORP., LTD. Address before: 518000 Shenzhen, Nanshan District Province, science and technology, South Road, the building of the big square, building 02, room 4, Applicant before: Shenzhen Tol Technology Co., Ltd. |
|
| COR | Change of bibliographic data | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| CP01 | Change in the name or title of a patent holder | ||
| CP01 | Change in the name or title of a patent holder |
Address after: 518000 room 02, 4th floor, Fangda building, Keji South 12th Road, Nanshan District, Shenzhen City, Guangdong Province Patentee after: Shenzhen Taishan Sports Technology Co.,Ltd. Address before: 518000 room 02, 4th floor, Fangda building, Keji South 12th Road, Nanshan District, Shenzhen City, Guangdong Province Patentee before: SHENZHEN TAISHAN SPORTS TECHNOLOGY Corp.,Ltd. |