CN101359250A - Virtual keyboard system - Google Patents

Abstract

The invention discloses a keyboard without an entity shape. The keyboard is composed of a lens, an analog-to-digital converter and a processing unit; the lens and the analog-to-digital converter are used to collect image signals and convert the image signals into digital image signals; through the comparison between the digital image signals collected at the adjacent time slices, the processing unit identifies the input intention expressed by the changing images and converts the input intention into an input event according to the internal comparison table. The keyboard is not restricted from shape and can be used in any environment; besides, the keyboard can be customized with input content, and has good portability and high universality.

Description

A kind of virtual keyboard system

Technical field

The present invention relates to the application electric technology field, relate in particular to the application process of photoelectric technology and image processing techniques.

Background technology

Keyboard (Keyboard) is a kind of electron device that is widely used, as the most widely used a kind of electron device, common keyboard generally consists of the following components: 1, a plurality of pressure-sensitive switching devices, and its number depends on the mode of the number and the coding of input element; 2, selectable process unit; 3, output interface.When using keyboard, the press...with one's finger switching device of pressure pressure sensitivity of user makes this switching device form pulse electrical signal; And output interface is used for converting pulse electrical signal to external system (for example main frame) required signal; Processing unit is optionally, if input element is very simple, then can not have processing unit, otherwise needs processing unit that the pulse electrical signal that the pressure sensitive switch device forms is transformed into required input element.

For an example of simple keyboard: the keyboard on the regular handset is made up of the two-layer film that is printed on carbon film up and down.Be printed on 0-9, *, # on the film of upper strata, dial key, on-hook key totally 14 input characters, each position that is printed on input character scribbles carbon film at the lower surface of film.Lower floor's film then is the film of whole piece upper surface conduction.15 leads of two films and the common mistakes of the disposable plates of mobile phone link to each other, and wherein 14 leads are received respectively on the carbon film at 14 input character lower surfaces of upper strata film place, and the 15th lead is ground wire, receives on lower floor's film conductive layer.Two-layer film is owing to the elasticity of film can not contact at ordinary times, when press...withing one's finger, the user impresses the position that input character is arranged, the upper strata film can be pressed and touch lower floor's film, because carbon film contact conduction, make corresponding to the lead and the ground wire of this input character and connect, the disposable plates of mobile phone just knows the user is by which key.This keyboard does not have processing unit, and output interface also is simplified to for each input element to be had only the connection of lead and interrupt two kinds of situations.

Again for the example of sophisticated keypads: the keyboard of common computer is by forming with lower member: 1) 84 or 104 pressure-sensitive normal open switch, 2) sweep circuit, 3 formed of bilevel lead in length and breadth) keyboard processing unit, 4) the 6 core keyboard output interfaces that link to each other with machine calculation machine host.On 104 switch surfaces, be printed on alphabetical A-Z, digital 0-9 and some function keys such as shift SHIFT, control character key CTRL or the like.When the user wishes input alphabet a, the user presses the switch that is printed on alphabetical A on the switch surface, can cause the some points in the sweep circuit that bilevel lead in length and breadth forms to be in on-state like this, the keyboard processing unit is pressed according to longitudinal and transverse two position judgment letter A place switch of connecting, processing unit forms input character " a ", at last, the 6 core keyboard output interfaces of this input character " a " by linking to each other with main frame send to main frame in the serial code mode with 82 long scale codings.

But there is following shortcoming in above-described keyboard: (1) must be according to the switching device of the content production fixed qty of being imported, and keyboard is the input content customization, mutually can not be general.The required device of complicated more input content is many more, cost is also high more.For example mobile phone only needs input digit, *, #, dials and these 14 elements of on-hook, and the keyboard of mobile phone just must have 14 switching devices; Computing machine need be imported upper and lower case letter, numeral, punctuation mark and control character, but the device cost of making more than 250 switch is high and use inconvenience, therefore formulates input rule (can use the capital and small letter switch to switch, be used in combination shift SHIFT, operating key CTRL and letter and numeral) by utilization switch has been tapered to 84 to 104.The keyboard that goes out for mobile phones design and the keyboard that goes out for Computer Design differ each other too big can not be general; (2) owing to importing the reason that content defines switching device number and use habit, the position that keyboard is arranged out these switching devices has also formed the parts of fixed size and shape, forms the requirement that keyboard is placed and has limited the degree of freedom of the shaped design of product.For example cell phone keyboard is a 4-5 centimeter length, 3-4 centimetre wide input area, and the design of mobile phone must be satisfied the requirement of this input area, so the keyboard input plane of mobile phone becomes a difficulty that is difficult to overcome of mobile phone shaped design; The defeated dish of computing machine then is one 30 centimeter length, 15 centimetres of wide input areas, causes computing machine where no matter to be placed on, and all needs a special zone of placing keyboard.

Except above-mentioned conventional keyboard, developed again afterwards and a kind of universal keyboard with touch screen, it is to be applied to that fixing input content customizes, the pattern of switching device fixed amount that universal keyboard with touch screen is different from conventional keyboard.Touch-screen generally has several class, every kind of class has versatility to a certain degree, for example, the touch-screen that provides for PDA provides 320*240 to possess pressure-sensitive point, and certain or some what contents of some representative input that specifically touch in this 320*240 the point are handled by software.Can on screen, demonstrate earlier the pattern of a keyboard such as PDA, the user on the keyboard that shows by the zone of correspondence.Use the shortcoming of keyboard with touch screen to be that touch-screen must use together in conjunction with display screen; Another shortcoming reason that then to be touch-screen require owing to transparency requires high to material, the material that constitutes touch-screen does not at present in use stop to lose efficacy in the relatively shorter time through regular meeting under the situation of physics contact, brings big inconvenience to application.

In addition, all be to contact by physics to realize input process with upper keyboard, the user can't leave keyboard input, and the user has in use been caused inconvenience.

Summary of the invention

Because common keyboard shape and structure that the realization technology of existing keyboard causes are fixed, the keyboard between the different system is not general; Perhaps the strict and inefficacy easily of restriction is installed, and solves the problem that in use can't leave keyboard to the user, the invention provides a kind of technical scheme that realizes dummy keyboard as keyboard with touch screen.

Technical scheme of the present invention is: realize a kind of keyboard that does not have physical form.It is made up of camera lens, analog to digital converter, processing unit, camera lens and analog to digital converter are used for picking up image signal and convert picture signal to digital signal, processing unit compares by the image digital signal that adjacent timeslice is obtained, and identifies the represented input wish of modified-image and converts incoming event to according to the table of comparisons of inside.

With respect to prior art, the invention has the beneficial effects as follows: at first, there is not the keyboard of physical form in the present invention, therefore solved fixed in shape, not general and inflexible shortcoming, comprised that applicable to conduct cell phone keyboard, main frame keyboard, sub-district doorbell keyboard, automatic vending machine select the scope of application of nearly all conventional keyboard such as goods keyboard; The optional output interface converting unit of the present invention has simultaneously guaranteed that also replacing keyboard need not to change to the other system of original connection conventional keyboard afterwards, and is therefore all very little to the retrofit work amount and the technical difficulty of original other system.Simultaneously, the present invention has also solved tactile keyboard and must use and exist physics to contact the shortcoming that easily lost efficacy in conjunction with display screen, and keyboard of the present invention need not the physics contact, do not have a loss, and life of product is long; And because there is not physics contact, the user does not require the distance of user and keyboard when operating input, while so the user can lie on a bed that operation keyboard is imported or move at treadmill and to use.

Description of drawings

Fig. 1 is virtual keyboard system and the simple structure block diagram that comprises the input and output signal;

Fig. 2 is the effect synoptic diagram of the employed background of prototype system of the present invention after finishing;

Embodiment

The present invention is further described in more detail below in conjunction with drawings and the specific embodiments.

Fig. 1 is the virtual keyboard system structural representation, and it is made up of several parts such as camera lens S1, analog to digital converter S2, processing unit S3.

Camera lens S1 that uses in prototype system of the present invention and analog to digital converter S2 are the camera of a personal computer that camera lens S1 and analog to digital converter S2 are fitted together with 350,000 pixel USB interface, camera lens on this camera partly is the monolithic convex lens, poly-2 millimeters of projecting to camera inside of the ambient light congruence that camera can be faced are taken advantage of in 2 millimeters the zone, (CCD is analog-to-digital one type an of photoelectricity to have placed a slice analog to digital converter CCD in this zone, CCD is that English Charge Coupled Device is the abbreviation of charge-coupled image sensor), the light that camera can be received converts digital image information to; The processing unit S3 that uses in the prototype system of the present invention is a common personal computer, and operation in the above has the dummy keyboard processing unit software that oneself is developed.

Prototype system of the present invention also needs following cooperation before use: the environment that (1) is bright; The plane of (2) pure colors as a setting, 40 centimetres of square white tabletop of taking advantage of 40 centimetres have been used in this example, take advantage of 40 centimetre desktop to be divided into 4 with light color pencil fine rule with 40 centimetres at desktop and take advantage of 16 square area of 4, and put on 0-9, *, #, rollback, off-hook from left to right, from top to bottom, finish, redial with the light color pencil, finish effect and see also Fig. 2; (3) camera fixed support, being used for directly over the center position of white tabletop zoning half meter fixes, the digital image information that camera just in time can be full of the position of desktop zoning fully collect and extra-regional part not in the digital image information scope (because the image that camera is got is a rectangle, so also can enter the camera collection scope a bit in addition in that square region is overseas, the lower left corner of just in time aiming at the camera collection scope according to square area gets final product); (4) the circular plate mat of black 10 cm diameters is one, be used to be placed in the variation that forms image on the white tabletop diverse location, move for convenience, fixed the thin wire of one 50 centimeter length on the plate mat with adhesive tape, but hand take thin wire with plate mat be placed into the arbitrary region of desktop and camera collection that hand can not occur to image in; (5) another debugging software prints the information of dummy keyboard processing unit software output at any time, is used to the effect of checking the present invention to export; (6) motion stopwatches that have millisecond to show, the speed that is used to control input is harmonized into both errors with the clock of processing unit S3 in advance and is no more than 100 milliseconds so that carries out synchronously with the unit interval of dummy keyboard processing unit S3 before operation.

Prototype system of the present invention also need be provided with following parameter: (1) image acquisition unit duration: 1000ms (millisecond) before using on the dummy keyboard processing unit software as processing unit S3; (2) image pixel acquisition range: (0,0)-(479,479); (3) image allows jitter range: 0%; (4) anamorphose scope: 0%; (5) image threshold: 0.6; (6) trigger mode: 0 (normal bright pattern); (7) coordinate elemental range: [0,0]-[3,3] (to the right toward the X-axis positive dirction, upwards toward the Y-axis positive dirction); (8) coordinate element transformation matrix: (0,0)-＞' * ', (0,1)-＞' 0 ', (0,2)-＞' # ', (0,3)-＞' Talk ', (1,0)-＞' 7 ', (1,1)-＞' 8 ', (1,2)-＞' 9 ', (1,3)-＞' End ', (2,0)-＞' 4 ', (2,1)-＞' 5 ', (2,2)-＞' 6 ', (2,3)-＞' Redial ', (3,0)-＞' 1 ', (3,1)-＞' 2 ', (3,2)-＞' 3 ', (3,3)-＞' ^H ' (^H is a control character, and function is character of rollback).

When using prototype system of the present invention, the hand-held thin wire that is fixed on the plate mat of user when the motion stopwatch just shows new one second, is put into plate mat some zones, ready-portioned 16 zones of desktop immediately, and when arriving in next one second, move to another zone again or remain unchanged.On another computing machine, can see this moment and can print a content p.s..

For the ease of the present invention is further understood, describe the present invention below in conjunction with the specific embodiment of the present invention.

(101) at first, when beginning in the 0th second, diffuse reflection can take place in the light that shines on the white tabletop, some of them light arrives the camera lens S1 place of camera and projects on the CCD S2, formed one 640 digital image information of taking advantage of 480 pixels, this digital image information is received by the dummy keyboard handling procedure on the personal computer S3 then, the dummy keyboard handling procedure first step checks that at first whether clock information is one second beginning, determine be one second at first, the dummy keyboard handling procedure just reads in digital image information and begins further processing, second step of handling is according to the parameter " image pixel acquisition range " (0 that is provided with, 0)-(479,479) take advantage of 480 image to cut out foursquare part with 640; The 3rd step of handling calculates the scope of each coordinate element corresponding to real image according to " coordinate elemental range " [0,0]-[3,3] are set, and formula is (479+1)/(3+1)=120, and promptly per 120 take advantage of 120 pixels corresponding to a coordinate element; The 4th step of handling is that the brightness with all pixels in each coordinate element adds up, be about to image from the lower left corner (0,0) to the upper right corner (119,119) all pixel intensity accumulated values are put in the coordinate element [0,0], and image is from the lower left corner (0,120) to the upper right corner (119,239) all pixel intensity accumulated values are put in the coordinate element [0,1], and the rest may be inferred; The 5th step of handling is according to being provided with " image threshold " 0.6, and all brightness accumulated values take advantage of 120 to take advantage of 0.6 coordinate element to be set to 0 less than 120, and all brightness accumulated values take advantage of 120 to take advantage of 0.6 coordinate element to be set to 1 greater than 120.Because all be white portion this moment, the value result of all coordinate elements is 1.

(102) because be to read for the first time the coordinate element, handling procedure with [0,0]-[3,3] value of totally 16 coordinate elements save as currency.

(103) the 1st second when beginning plate mat is moved to a certain zone of desktop for example during zone " 1 " as the user, diffuse reflection can take place in light equally that shine on white tabletop and the black plate mat, same, light arrives the camera lens S1 place of camera and projects on the CCD S2, formed one 640 digital image information of taking advantage of 480 pixels, this digital image information is received by the dummy keyboard handling procedure on the personal computer S3 then, the dummy keyboard handling procedure according to (101) same five the step treatment steps information is saved in the coordinate element, but different with step (101) is, coordinate element [3,0] because places the relation of black plate mat, the brightness accumulated value has reduced, so when the 5th step handled, coordinate element [3,0] value has become 0, and the value of other coordinate elements still all is 1.

(104) the dummy keyboard handling procedure is with [0,0]-[3,3] currency preserved of totally 16 coordinate elements and (102) step compares, determined coordinate element [3,0] value becomes 0 from 1, this moment, the dummy keyboard handling procedure was further according to " trigger mode " that be provided with: the value that 0 (normal bright pattern) judged coordinate element [3,0] becomes 0 from 1 effective variation has taken place.

(105) judge finish after, the dummy keyboard handling procedure abandons the currency of original preservation, with [0,0]-[3,3] value of totally 16 coordinate elements save as new currency.

(106) result that changes of [3,0] coordinate element value of obtaining according to " coordinate element transformation matrix " and (104) step judgement that is provided with of dummy keyboard handling procedure, finding coordinate element [3,0] corresponding characters is ' 1 '.

(107) the dummy keyboard handling procedure is searched the character ' 1 ' that obtains with (106) step and is outputed to debugged program.

(108) debugged program reads input signal and printout, has shown character ' 1 ' on screen.

(109) similar with (103) step, in the zero hour next second, the user moves to for example zone " rollback ", a certain zone of desktop with plate mat, diffuse reflection can take place in light equally that shine on white tabletop and the black plate mat, same, light arrives the camera lens S1 place of camera and projects on the CCD S2, formed one 640 digital image information of taking advantage of 480 pixels, this digital image information is received by the dummy keyboard handling procedure on the personal computer S3 then, and the dummy keyboard handling procedure is saved in information in the coordinate element according to four same step treatment steps, and still different with step (103) is, coordinate element [3,0] because of the removed relation of black plate mat, the brightness accumulated value has increased, and coordinate element [3,3] because place the relation of black plate mat, the brightness accumulated value has reduced, so when the 5th step handled, coordinate element [3,0] value has become 1, the value of coordinate element [3,3] has become 0, and the value of other coordinate elements still all is 1.

(110) similar with (104) step, the dummy keyboard handling procedure is with [0,0]-[3,3] currency preserved of totally 16 coordinate elements and (102) step compares, determined coordinate element [3,0] value becomes 1 and coordinate element [3 from 0,3] value becomes 0 from 1, this moment, the dummy keyboard handling procedure was further according to " trigger mode " that be provided with: 0 (normal bright pattern) judged coordinate element [3,0] value becomes 1 invalid being left in the basket of variation of taking place from 0, effective variation has taken place and the value of coordinate element [3,3] becomes 0 from 1.

(11) judge finish after, the dummy keyboard handling procedure abandons the currency of original preservation, with [0,0]-[3,3] value of totally 16 coordinate elements save as new currency.

(112) the dummy keyboard handling procedure is according to " the coordinate element transformation matrix " that be provided with, and finding coordinate element [3,3] corresponding characters is ' ^H '.

(113) the dummy keyboard handling procedure outputs to debugged program with character ' ^H '.

(114) debugged program reads input signal and printout, output character on screen ' ^H ', and this character is a control character, the effect of seeing is that cursor has moved lattice forward, and has removed the character ' 1 ' of (108) step demonstration.

(115) user's repeating step (109) is to (114), but when repeating to do step (109), can plate mat be placed on arbitrary position in 16 zones according to user's idea, place each time, can see at debugged program and to have exported character, and the content that the zone indicated that the content of character and plate mat are placed is identical.

More than be exactly all processes and the step introduction of prototype system of the present invention.In order to reduce the cost of checking, form the present stage electronic market existing device that all hardware equipment of prototype system of the present invention all is to use, having only dummy keyboard processing unit software is to develop voluntarily.Succeeding in developing of prototype system confirmed that the present invention is feasible technically, and novel keyboard can be used fully and give an account of continue middle identical materials, principle and step and repeat system of the present invention.

In prototype system of the present invention, because the processing unit S3 that uses is a common personal computer, volume is bigger, carry inconvenience, if make according to prototype system fully, practicality of the present invention is very poor, therefore after verifying that with prototype system the present invention is feasible, the inventor again design and assembly No. 1, prototype system modified of the present invention, camera lens S1 that uses and analog to digital converter S2 are the camera of a personal computer that camera lens S1 and analog to digital converter S2 are fitted together with 350,000 pixel blue tooth interfaces, simultaneously in order to verify versatility of the present invention, increased the output interface converting unit, USB-bluetooth converter has been installed on processing unit S3, and come reception information to test with the hand-held PDA (personal digital assistant) that supports bluetooth, through overtesting, camera and hand-held PDA that the user can carry blue tooth interface easily place and use system of the present invention between the room, and it is motionless to be placed on original position as the personal computer of processing unit S3.

Further improve and be embodied in No. 2, prototype system modified of the present invention, No. 1, the hardware that No. 2, modified and modified are identical, difference is to improve as the program above the personal computer of processing unit S3, new program has been used image XOR computing method, 640 take advantage of the image of 480 pixels and 640 brightness values of taking advantage of the image of 480 pixels to put one by one according to the principle of same coordinate that receive last second subtract each other with what receive, like this when twice image of the same area do not change, accumulated value as a result after subtracting each other is very little, and twice image is when changing (plate mat is put into or taken away), accumulated value as a result after subtracting each other is very big, use the program after improving like this to need not as a setting with white tabletop, but can use background arbitrarily, such as wall, the floor, sheet, can, usable range enlarged greatly.

Similarly improvement also is embodied in No. 3, prototype system modified of the present invention, No. 1, the hardware that No. 3, modified and modified are identical, difference is further to be improved on the basis of No. 2, modified as the program above the personal computer of processing unit S3, new program has been used the image-region matching algorithm, according to the value that " image permission jitter range " is set, by skew value from small to large, earlier from left to right, again from top to bottom, to receive successively 640 take advantage of the image of 480 pixels and receive last second 640 take advantage of 480 pixels to calculate intersection area, the brightness value that image in will occuring simultaneously is again put one by one subtracts each other, and the subtraction value result who is had a few in the intersection area added up, and get the result of summation minimum among a plurality of results, image lateral excursion and the vertical misalignment of correspondence are exactly the off-set value of two width of cloth images as a result for these.As long as present image is carried out translation according to image lateral excursion value and vertical misalignment value, just present image can be reduced into the image when skew not taking place.It is fixing on the wall or on the table to use program user after improving like this to need not camera, but can be worn on the head, and in background arbitrarily, such as wall, floor, sheet, desktop etc., can use, and has further enlarged usable range and practicality.

Other improvement thinking also has by the method for simplifying background and reduces requirement to processing unit S3, because Flame Image Process comprises coupling, rotation, operations such as translation all are the very big algorithms of operand, so by the simplification prospect, for example use borders as a setting such as pure color ruddiness, use 16 high-brightness laser points rather than 16 zones to judge in addition, can make the dummy keyboard handling procedure when the image of reduction distortion and skew, and judge that the operand when whether image changes reduces a lot, even can use the very high single-chip microcomputer of integrated level to realize program function fully.System after improving like this, camera lens S1, analog to digital converter S2, processing unit S3 can be integrated into a very little degree fully, and increase output interface converting unit such as blue tooth interface, cumulative volume is little of being installed on the carry-on articles such as eye glass frame, watch chain, hair clip, necklace, make keyboard break away from the shape restriction fully, become new device personalized and that everyone is happy to use.

The above only is the simplest several embodiment of the present invention; should be pointed out that for those skilled in the art, under the prerequisite that does not break away from the principle of the invention; can also make some improvements and modifications, these improvements and modifications also should be considered as protection scope of the present invention.

Claims (10)

1, a kind of keyboard that does not have solid shape is characterized in that, comprises step:

(A) described system is made up of camera lens, analog to digital converter, processing unit;

(B) camera lens and analog to digital converter convert the electromagnetic wave that receives in the camera lens range of receiving to data image signal;

(C) processing unit receives the data image signal that camera lens and analog to digital converter obtain with the unit interval, and the changing unit of two width of cloth images judgement image by the more adjacent time;

(D) processing unit is searched the position of described internal system preservation and the table of comparisons of incoming event according to the image change part, converts the position to incoming event.

2, system according to claim 1 is characterized in that, is not included in the described system when carrying out input operation to carry out the parts that physics contacts with the user;

3, system according to claim 1 is characterized in that, uses the electromagnetic wave of various wavelength such as the visible light can be converted to data image signal by camera lens and on analog to digital converter, infrared ray, ultraviolet ray among the described step B; Electromagnetic wave is converted to data image signal by the photoelectricity analog to digital converter;

4, system according to claim 1, it is characterized in that, processing unit is the data image signal that interval acquiring camera lens and analog to digital converter obtain with the unit interval among the described step C, and contrast adjacent consecutive image of resulting time, judge the positional information whether the place part has taken place between the consecutive image to obtain changing when variation and image change;

5, system according to claim 1, its feature also is, between adjacent two width of cloth images, allow image range that image range that camera lens receives with respect to camera lens or camera lens receives with respect to camera lens generation less offset among the described step C, and be also to allow to take place less offset between inner each zone of image in the camera lens range of receiving, processing unit is eliminated the position recovering of the bias effect of camera lens and image with front and back two width of cloth images by image being carried out maximum match.And the offset ranges that camera lens allows is configurable to adapt to from static state to dynamic different environment for use;

6, system according to claim 1, its feature also is, processing speed and accuracy requirement in order to reduce processing unit in the described step C, can use the high-contrast mark in the camera lens range of receiving is the scope of image, processing unit can be calibrated the skew that camera lens takes place by checking the high-contrast gauge point; And the foundation that can only judge as variation in the image with the variation of high-contrast gauge point;

7, system according to claim 1, its feature also is, there are a lot of interfere informations inequality by the digital image information precision after the analog to digital conversion, among the described step C by adopting a span to judge, the variation that only ought exceed span just is considered to change in the image, and this span can dispose;

8, system according to claim 1, its feature are that also the table of comparisons of position and incoming event can dispose among the described step D;

9, system according to claim 1, its feature also is, three parts of described system, can combine physically, also can not combine and link to each other by radio communication device, camera lens, analog to digital converter are combined as light, the portable parts of weight, and powerful, the non-portable device of processing unit functions of use is realized;

10, system according to claim 1, its feature also is, when the incoming event signal format of processing unit output can not directly be used by other equipment, can increase input data-switching that the output interface converting unit forms processing unit and become required interface shape to offer other equipment or parts use.

Images