CN1315024C - Video recognition input system - Google Patents

Abstract

The present invention relates to a video recognition input system. The present invention provides a video recognition input system which is composed of two camera heads and a digital computer and overcomes the defects of complex equipment, larger volume, low recognition rate and higher cost of the prior art. The two camera heads respectively arranged on a horizontal position and a vertical position are utilized for collecting image sequence manually moved, and finally, information manually keyed in is obtained by recognition of an image processing system of the digital computer. The present invention can realize input operation without a keyboard and a mouse, and has the higher input recognition rate. Due to the only need of the two ordinary camera heads, the present invention has the advantages of simple equipment, low cost and convenient maintenance and update.

Description

A kind of input method of video recognition input system

(1) technical field

The present invention relates to a kind of input method, the input method of especially a kind of on-keyboard, no mouse input system.

(2) background technology

At present, mouse and keyboard are generally adopted in the input of computing machine, because the existence of keyboard and mouse makes the input equipment of computing machine seem huge and inconvenient mobile, portable computer equally also can't be done smaller and more exquisitely owing to the existence of keyboard.If simply dwindle the size of keyboard, too small keyboard is not suitable for the handled easily of finger again.And the mode that adopts nib directly to contact is imported on miniature keyboard, makes input speed greatly reduce especially.

This difficult problem is perplexing all mobile-phone manufacturers equally.At present, mobile phone is just making every effort to develop to smaller and more exquisite, more portable direction, but owing to omit the existence of the keyboard that does not fall, makes the development of this direction that certain limitation always be arranged.

In the instruction input system of vehicle, instrument, robot, machining center and the weaponry of intelligence, also need a kind of reliable and stable, miniature input system so that the weight saving of equipment, machine-building be connected difficulty and reduce.

In order to address this problem, application has been developed and entered to the phonetic entry recognition system, but the discrimination of phonetic entry is still lower, and user's the token sound degree and the quiet degree of environment for use are had higher requirement.Also have some to adopt touch screen and stylus, all need specialized apparatus, increased the complexity and the deployment cost of equipment as the method for input medium, and the renewal of equipment after being unfavorable for.

In November, 2002, disclosed publication number was the patent of CN1378171, the computer input that one cover is made of image sensing apparatus and induction information extraction element has been described, can obtain hand or assigned object shape information by image sensing apparatus in the space, the image information feature extraction element extracts the characteristic information of this spatial shape and imports main frame into, show on computer display screen and sell or the spatial shape of assigned object that simultaneous computer produces computer-managed instruction according to the characteristic information that is imported into.But this system needs specific image sensing apparatus and induction information extraction element, the equipment complexity, and cost is higher.Though the patent that publication number in 2003 is CN03136144.7 is replenished, and can change the dsp processor function of Flame Image Process into.

(3) summary of the invention

Can only discern static gesture in order to overcome above-mentioned prior art, the deficiency that amount of input information is less, the invention provides the input method of the video recognition input system of a kind of on-keyboard, no mouse, can utilize common apparatus effectively to improve the input discrimination, and reduce cost.

The present invention includes two cameras and a digital machine, two cameras are placed on horizontal and vertical position respectively, the image sequence that the staff that image processing step identification camera by digital machine collects moves finally draws the information that staff is keyed in.

For Recognition and Acquisition to image sequence and obtain corresponding ASCII character, described input system is carried out following steps:

1) image capturing system that constitutes with two cameras that are placed on horizontal and vertical position is respectively finished finger with dynamic staff and is moved and convert the single frames digital picture to.

2) by image processing system image is handled, obtained the two-dimensional array data that palm connects the point coordinate on the appearance profile line of pointing at last.Determine the finger center point coordinate according to data, judge it is keystroke state or rolling mouse state according to palm outline line array.

3) finish pattern-recognition and tracking by the target determining program to image:

A) finish tracking and location to object in the image, and determine in view of the above one with the central plane of ten fingers apart from plane, the plane of sum minimum as dummy keyboard or mouse place.

B) motion of tracing object is judged, judged the key assignments that triggers;

C) action carried out of simulated object, and return corresponding ASCII character value.

4) the ASCII character value that draws in the target determining program is turned back to operating system, finish the operation of simulation imput device.

The present invention can realize the input operation of on-keyboard, no mouse, has higher input discrimination, and owing to only need two common cameras, equipment is simple, and is with low cost, is convenient to maintenance update.

(4) description of drawings

Accompanying drawing 1 is a system flowchart of the present invention

Accompanying drawing 2 is image acquisition step process flow diagrams of the present invention

Accompanying drawing 3 is image processing step process flow diagrams of the present invention

Accompanying drawing 4A and accompanying drawing 4B are simulating keyboard status target determining step process flow diagrams of the present invention

Accompanying drawing 5 is analog mouse status target determining step process flow diagrams of the present invention

(5) embodiment

The present invention is one and integrates the real-time input method that image acquisition, Flame Image Process, target are judged.

Referring to accompanying drawing 1, the present invention is divided into image acquisition step (Figure 101), image processing step (Figure 102) and target determining step (Figure 103) three parts:

1, image acquisition step: referring to accompanying drawing 2, this step is mainly finished the image that the avi file circulation of USB camera collection is turned to single frames, and the back is with the array output of a certain specific Integer N frame/second.Its detailed process is as follows:

At first the image sequence that moves of the finger of gathering staff by two general cameras forms the AVI data, by loading this AVI data streaming file, realize the single framesization (Figure 20 3) of AVI data stream, after with continuous images grouping (Figure 20 4) with a certain specific Integer N frame/time form array to pass to image processing step and target determining step (Figure 20 5), after treating that at last whole process is finished first round Flame Image Process, system deletes the image (Figure 20 6) after handling automatically.

2, image processing step:, mainly finish identification for the finger center referring to accompanying drawing 3.

The functional steps and the algorithm thereof of specific implementation are as follows:

By calling the Bitmap class: at first realize the gray scale processing, make the rgb value of every bit equal the arithmetic mean of original bitmap R value, G value and B value (Figure 30 1).

Next smoothing processing, we just once handle image roughly here, and fundamental purpose is the noise of removing in the image.Use common neighbours territory method, even the gray-scale value of coordinate [i] [j] point equals itself and [i-1] [j], [i+1] [j], [i] [j-1], the arithmetic mean (Figure 30 2) of the gray-scale value of these four some pixels of [i] [j+1].

Be that iteration is asked image optimum partitioning algorithm (Figure 30 3) again:

A) obtain minimum and maximum gradation value Z in the image _lAnd Z _k, make the threshold values initial value

$T^0=\frac{Z_1+Z_k}{2}$

B) according to threshold values T ^kImage segmentation is become target and background two parts, obtain two-part average gray value Z _OAnd Z _B:

$Z_O=\frac{\underset{z(i,j)<T^k}{\mathrm{\&Sigma;}}Z(i,j)\&times;N(i,j)}{\underset{z(i,j)<T^k}{\mathrm{\&Sigma;}}N(i,j)}$ $Z_B=\frac{\underset{z(i,j)>T^k}{\mathrm{\&Sigma;}}Z(i,j)\&times;N(i,j)}{\underset{z(i,j)<T^k}{\mathrm{\&Sigma;}}N(i,j)}$

Z in the formula (i, j) be on the image (i, gray-scale value j), N (i, j) be (i, weight coefficient j),

General N (i, j)=1.0

C) obtain new threshold values: $T^{K+1}=\frac{Z_O+Z_B}{2}$

D) if T ^K=T ^K+1, then finish, otherwise K ← K+1 forwards step b to.

It again binaryzation.(Figure 30 4)

It is as follows that the transforming function transformation function of gray scale threshold values conversion is expressed formula:

$f\left(x\right)=\left\{\begin{array}{cc}0& X<T\\ 255& X\&GreaterEqual;T\end{array}\right.$

It again isolated point, because this removes isolated point, be to determine the finger central point for next step high-quality binary image is provided, so adopt eight classical neighborhood methods, its algorithm is: with the rgb value and eight adjacent points: [i-1] [j] in the zone of [i] [j], [i+1] [j], [i] [j-+1], [i] [j+1], [i-1] [j-1], [i+1] [j-1], [i-1] [j+1], [i+1] [j+1] compares, and area size is elected 3 * 3 pixel region as, if the rgb value of [i] [j] and eight points does not on every side wait, then with [i] [j] some inverse (Figure 30 5).

Gather all point coordinate formation arrays (Figure 30 6) on the hand-type outline line at last, by processing (Figure 30 7) to this array, judge this moment hand-type and be in and knock keyboard state (Figure 30 9a) or rolling mouse state (Figure 30 9b), call corresponding processing procedure subsequently.

3. knock keyboard target determining step:, mainly finish judgement for the action of knocking keyboard state referring to accompanying drawing 4A and accompanying drawing 4B.

For knocking the keyboard input state, utilize native system software the image of camera collection to be carried out pattern-recognition in computer-internal, just accurately the position of the contact point (hereinafter to be referred as " finger ") of finger below and dummy keyboard is also judged in the location.According to the moving situation of finger, the pursuit movement track, analyzing specifically is which finger is in the state of moving downward, and provides the position of concrete button, returns the ASCII character value of the finger button that strikes.

By the interface of this image processing step and operating system, the SDK interface of promptly former keyboard and operating system imports these ASCII character values into operating system realizes concrete content of operation at last.

The detailed algorithm of the target determining step that knocks keyboard shown in accompanying drawing 4A and the accompanying drawing 4B is as follows:

1. in the time of) in finger enters the camera collection zone, the initialization finger position is reference plane (Figure 40 2), judge whether in one plane finger has a few (Figure 40 3), if not in one plane, these 10 discrete o'clock to a plane fitting, make these 10 points to this plane apart from sum minimum (Figure 40 4).By horizontal camera and spatial value (X, Y, Z) (Figure 40 5) of vertical camera collection to finger position this moment.

2.) forefinger, middle finger, little finger of toe and the third finger with the right-hand man is that benchmark is set up the keyboard coordinate system, position when the null position of this keyboard coordinate system enters for the first time pickup area, initialization finger with finger is as the criterion, and with the difference of the position that enters each time difference.(as Figure 40 6a)

3.) draw one " on-screen keyboard " in the screen lower left corner, concrete form is similar to " on-screen keyboard " in the Windows annex aid.After this loading routine starts, show which kind of keyboard on the screen, then in the space, just produce corresponding dummy keyboard, keyboard plane and finger planes overlapping.The keyboard size is identical with true keyboard, is the equal proportion of on-screen keyboard affine (Figure 40 6b).

4.) set up a database, deposit database in respect to the coordinate at keyboard zero point, during regional under finger drops on certain key, just show on virtual on-screen keyboard that this key is a grey for the stupefied crossing point of each key on the keyboard for dummy keyboard.(Figure 40 7a) judges in the finger motion process, and (X, Y during the spatial value Z value of planimetric position, think that then object key is hit when Z) the Z value in is less than or equal to initialization when the spatial value of finger position.When this key is hit, the effect (Figure 40 7b) that this key is hit with the keypad sounds and the deep form simulation of gray scale of simulating.

5.) " on-screen keyboard " drawn in the screen lower left corner gone up the little hand of drawing a dummy activity.The position of little hand be finger position spatial value (X, Y, in Z) (X, Y) value, array Array[10] [N] can show the continuous coordinate of the position of pointing in the N frame picture, wherein N is a certain specific integer (Figure 40 8a).The ASCII character value of the pseudo operation response in the said process is returned by the SDK interface of keyboard and system in the back, finishes the simulation of whole keystroke action.(Figure 40 8b)

4. the target determining step of rolling mouse state:, mainly finish judgement for the action of rolling mouse state referring to accompanying drawing 5.

For the rolling mouse input state, utilize native system software the image of camera collection to be carried out pattern-recognition, just accurate location and judge the position of " finger " in computer-internal.Moving situation according to finger analyzes finger and whether is in left click, right click and double-click state, and on the screen of computing machine, the tracing display movement locus, and return the coordinate figure at corresponding system parameter values and given finger center.

The detailed algorithm of the rolling mouse target determining step shown in the accompanying drawing 5 is as follows:

1. in the time of) in finger enters vertical camera collection zone, the given position (Figure 50 2) of mouse graphic presentation at screen hinted obliquely at in initialization middle fingertip position.

2.) by analyzing to the data message (Figure 50 3a) of vertical camera collection with to the data message (Figure 50 3b) of horizontal camera collection, with middle finger and forefinger finger tip is tracking target, and the data of the middle fingertip that arrives with vertical camera collection are as mouse moving information (Figure 50 4a); With horizontal camera collection to forefinger finger tip data judge whether to finish click, double click procedure (Figure 50 4c); In conjunction with level and vertical camera collection to forefinger and the data of the middle fingertip process (Figure 50 4b) that can judge left click and right click.

3.) above-mentioned mouse action process is carried out overall treatment (Figure 50 5), draw a virtual mouse, the motion conditions of simulation middle fingertip, go out the different-effect (Figure 50 6a) of left click, right click and double-click with different speech simulations, by the SDK interface of mouse and system, the coordinate figure (Figure 50 6b) that returns corresponding system parameter values of above-mentioned mouse action and given finger center is finished the simulation of whole mouse action action.

4.), can define voluntarily according to said method for other the mouse button and the special operational of roller.

Claims (1)

1, a kind of input method of video recognition input system, comprise two cameras and a digital machine, utilize two cameras that are placed on horizontal and vertical position respectively to gather the image sequence that staff moves, image processing system identification by digital machine, finally draw the information that staff is keyed in, it is characterized in that: for Recognition and Acquisition to image sequence and obtain corresponding ASCII character, described input system is carried out following steps:

1) image capturing system that constitutes with two cameras that are placed on horizontal and vertical position is respectively finished the digital picture that dynamic finger motion is converted to single frames;

2) by image processing program image is handled, obtain the two-dimensional array data that palm connects the point coordinate on the appearance profile line of pointing at last, determine the finger center point coordinate according to data, judge it is keystroke state or rolling mouse state according to palm outline line array;

3) finish pattern-recognition and tracking by the target determining program to image:

A) finish tracking and location to object in the image, and determine in view of the above one with the center position of ten fingers apart from plane, the plane of sum minimum as dummy keyboard or mouse place;

B) motion of tracing object is judged, judged the key assignments that triggers;

C) action carried out of simulated object, and return corresponding ASCII character value;

4) the ASCII character value that draws in the target determining program is turned back to operating system, finish the operation of simulation imput device.

Images