CN103443744A

CN103443744A - Dynamically located onscreen keyboard

Info

Publication number: CN103443744A
Application number: CN2011800642205A
Authority: CN
Inventors: R·J·马斯登; S·霍勒
Original assignee: Cleankeys Inc
Current assignee: Apple Inc
Priority date: 2010-11-30
Filing date: 2011-11-30
Publication date: 2013-12-11
Anticipated expiration: 2031-11-30
Also published as: JP6208718B2; WO2012075197A3; JP2014514785A; JP5782133B2; KR101578769B1; JP2015232889A; CN103443744B; WO2012075197A2; KR20140116785A; WO2012075199A2; CN106201324A; CN106201324B; WO2012075199A3; EP2646894A2; EP2646893A2

Abstract

A touch-sensitive display surface having touch-capacitive and vibration sensors. This surface allows the user to rest their fingers on the keys of an onscreen keyboard and type as they would on a regular keyboard. As the user places their fingers on the touch screen, the system relocates the onscreen keyboard to the location where the fingers are resting. The touch sensors report the signal strength level of each key touched to a processor, but no keystroke is issued by the processor until a corresponding "tap" (i.e., vibration) is detected. When a tap is detected, the processor references the status of the touch capacitance sensors before, during, and/or immediately after the moment in time the tap occurred.; The size, position, and orientation of the onscreen keyboard keys are dynamically set as determined by the user initiating a home-row definition event by resting their fingers momentarily on a virtual home-row.

Description

The on-screen keyboard of Kinematic Positioning

Technical field

The present invention relates to allow the user that their hand or finger are stopped to the level and smooth touch sensitive surface that causes from the teeth outwards and not event to activate.More particularly, touch-surface is the dynamic display presented for the on-screen keyboard of input text and order.

Background technology

Origin as the contemporary keyboard of the main method for being input to machine from people's text and data traces back to early stage typewriter of 19th century.Along with the development of computing machine, it is progress of nature that typewriter keyboard is suitable for use as for the main method of input text and data.Although the keyboard of also realizing on computer keyboard subsequently on typewriter develops into electric and finally develops into electronic type from mechanical type,, the size of keyboard self, position and mechanical essence remain unchanged basically.

Computing machine develops into from " desk-top " configuration the more portable configuration that is called " on knee ", " notebook ", " net originally " or " portable ".These laptop computers generally have the mechanical keyboard that one turns to the part of device.Such integrated keyboard have size and sensuously with the similar advantage of stand-alone keypad generally be combined with desk-top computer.But, comprise that keyboard causes portable computer to have two parts: display and keyboard.Most portable computer model adds " clam shell " design of the demonstration part that has the keyboard portion that forms pedestal and form lid.Therefore, on portable computer, existing keyboard to cause it is roughly its twice of the size of needs in addition.

The portable computing of the new model of " flat board " computing machine, appearred being commonly referred to as in the past 10 years.Such portable computing does not generally have integrated keyboard, and only depends on the touch as the Main Means of people-computer interface.Many people think, the standard mode that " touch-surface " that dull and stereotyped, final and daily life become one will become people and " computing machine " in the future and connect each other.

Although the touch center calculation of this new model has many advantages,, an obvious shortcoming is to lack keyboard.Although the external physical keyboard generally can be connected with touch screen computer,, it usually makes the purpose of device fall through and negates its advantage that is better than conventional laptop computer.

Along with calculation element, to the development of the user interface based on touching, the organic growth of keyboard thought must carry it into the real world of Computer display.

Auer etc. have described the system that produces suitable control signal in the touch of one or more images of the touch-sensitive screen display simulating keyboard of computing machine and response simulation key in U.S. Patent No. 4725694.In the improvement of later stage of this concept, with on other application that floats on operation on computers rather than occupy the image of mode display keyboard of the private part of screen.The user by guide the cursor pointer thereon or directly by use, point or stylus by the touch-screen membrane keyboard come with should " on-screen keyboard " or " dummy keyboard " reciprocation.

The on-screen keyboard of describing such as Auer is mainly used in lacking the device of QWERTY keyboard, to such an extent as to such as some public information call box and PDA(Personal Digital Assistant), smart phone, board and too little other handheld computer that can not hold physical keyboard.Individual with the physics challenge that can't use conventional electricapparatus keyboard also often uses on-screen keyboard.

Less touch panel device such as PDA and smart phone does not have enough screen sizes to allow the people to key on on-screen keyboard by the conventional method of using many finger touch to key in.As a result, a lot of inventions seek to provide the substituting text entry method of physical space that need to be less than conventional keyboard layout.

Grover etc. have described by distributing a plurality of letters to reduce the system of the quantity of the different key needed on each key in U.S. Patent No. 5818437.This allows key still less and therefore takies space on screen still less.Other aims at similarly the size that reduces on-screen keyboard and/or makes more easily the invention of input text on less screen comprise: Lee, U.S. Patent No. 6292179; Kaehler, U.S. Patent No. 5128672; Vargas, U.S. Patent No. 5748512; Niemeier, U.S. Patent No. 5574482; Van Kleeck, U.S. Patent No. 6008799; And Perlin, U.S. Patent No. 6031525.

Although these inventions have for key in the different benefit of text on little on-screen keyboard, the speed that they can not be suitable with the standard with keying on conventional keyboard " ten fingers " is keyed in text.

In order to improve typing speed, Robinson etc. have described when the disambiguation algorithm is selected each letter of word on the key of user in on-screen keyboard in U.S. Patent No. 7277088 the not system so accurately of user that allows.Allow so accurately not cause the user to key in quickly.

Kushler etc. have described and have allowed to rest on that the one hand caused on the key of first letter of the word that representative wishes refers to that (or stylus) text is keyed in and then in the system with sliding between the key at the respectively letter subsequently of word when touch-surface contact by the user in U.S. Patent No. 7098896.This has and need not promote and rest on the benefit on on-screen keyboard for each letter, accelerates thus text and keys in.The disambiguation algorithm allows the user inaccurate when selecting each letter, thus further pick up speed.

Swype as the commercialization version of the technology of being described by Kushler etc.

be used to set the world record of keying in the soonest on smart mobile phone.The individual who breaks a record is with the phrase of the speed input regulation of 61 words of per minute.Although this speed is uncommon,, suppose that it enters based on singlehanded digital, it still lags far behind the possible prestissimo that uses ten digitals to enter.

Other method is to use sound recognition system to pass through speech sounding input text.Although this technology recently be improved significantly,, even it 100% is worked exactly, the text by the input of speech sounding is not often user's desirable (such as in the needs privacy or while considering other people in the range of audibility) yet.And, therefore, still need to key in by the keyboard mode of some types the substituting mode of text.

Therefore, for the larger touch-screen that can hold ten digitals and enter, wish to find for keying in the mode faster of the text that closer mates the key entry stylus of grasping on conventional keyboard.In doing so, there are three main challenges: at first, overcome the needed relatively many display spaces of 10 finger on-screen keyboard.Secondly, overcome lacking of tactile feedback common in mechanical keyboard.The 3rd, allow the user that their finger is rested on " original position row " position on on-screen keyboard as conventional electricapparatus keyboard.

Marsden etc. in U.S. Patent application No.2009/0073128 by allowing the user that their finger is rested on touch sensitive surface and pressing and overcome this problem by the key that uses the touch of working relatively and vibration transducer to detect intention.But the method supposes that keyboard key is in fixing position, and therefore on the dynamic display of mancarried device, occupy obvious space.And, because the position of key is fixed, therefore, the user must carefully see that their finger tapping is on correct position.The tactile markings such as recess of positioning key helps the user in the situation that do not see and feel key.But it is unpractical placing tactile markings on touch panel device.

It is the concept of the key of user's finger of putting and stopping them when preparing to key in that conventional electricapparatus keyboard is used " original position row " for a long time.This concept is particular importance for having grasped the user that 10 digitals of not seeing key enter.By putting in original position row (comprising use special " mark " found on some key on row in position), the user knows their finger is moved to where to key in letter, symbol, numeral or the function of wishing.This allows the user promptly to key in the situation of the finger of not seeing them, and can be absorbed in the text of their creation.

With previous generation (at that time, key in course and only offer the people who wishes to seek the professionalism position), compare, computing machine, Email and text message are at the number percent that has produced higher " touching the typist " prevailing of society.In fact, often teach this keyboard input technical ability now in child's early education problem.10 fingers (or " touch ") are keyed in the faster and more reliable known method be still for creative text.

Summary of the invention

The invention provides and allow the user to stay on the key of the on-screen keyboard of touch-sensitive screen display by their finger and the system and method for the position of Dynamic Definition on-screen keyboard, orientation, shape and size.The user needn't note their finger is placed on key and (generally can requires the tactile markings on described key), the position that system has dynamically stopped the position of on-screen keyboard to user's finger.

In one aspect of the invention, process definition " original position row definition event ", it is that the system that causes of being carried out by the user redefines the operation of the original position row of positioning screen upper keyboard.This position of operation Dynamic Establishing based on the user.

In another aspect of this invention, original position row definition event is defined as the user and all four fingers of two hands is rested on to time durations default on touch sensitive surface (for example, 1 second) simultaneously.

In another aspect of this invention, original position row definition event is defined as the user and double-clicks all four fingers of two hands and then after knocking for the second time, make them stop from the teeth outwards on touch sensitive surface.

In still another aspect of the invention, original position row definition event is defined as the user all four fingers of two hands is rested on touch sensitive surface simultaneously and then press momently them downwards.

These operations (following other operation) are started with the finger to the system indicating user in original position row stop place by the user.Therefore then system of the present invention make the on-screen keyboard orientation.Note, the key on the original position row need to be on continuous line (being like this on most electricapparatus key).But the position definition of being gone in the definition event in position by eight fingers of the user by the touch sensor sensing in the position of each key on the original position row, then be extrapolated for the key that is not " the original position row stops key ".By this way, the original position row can be along two independent lines, and one for a hand position, or, even can form two curves.

Note, finger and user that the method requires system differentiation user of the present invention to put down and stop them on touch-sensitive display surfaces will key in by knocking virtual key.At Marsden, this method has been described in U.S. Patent application No.2009/0073128.

Once original position row definition event occurs, and system just provides feedback to the user in many ways.In one aspect of the invention, system is by causing on-screen keyboard virtual feedback to occur providing under user's finger.In another aspect of this invention, system provides the hint that can listen.In still another aspect of the invention, system causes the of short duration vibration of touch-screen.

In one aspect of the invention, according to user's preference, when key entry occurs, it is visible continuously that on-screen keyboard keeps.As an alternative, on-screen keyboard becomes transparent after capable definition event in position.In another aspect of this invention, it is translucent that on-screen keyboard becomes, and to allow the user, by keyboard, sees the content on following screen.

In still another aspect of the invention, when the user keys in, on-screen keyboard visible and invisible between the circulation.Whenever the user knocks " hide " on-screen keyboard, hidden taking off after the of short duration appearance of on-screen keyboard the time quantum that can set the user.

In still another aspect of the invention, only some key becomes visible after each thump.The temporary transient visible key that becomes is those keys that most probable is followed the text input order that is close to front (word database in being stored in system is determined).

In still another aspect of the invention, when finger stays in user on the original position line position and presses down surface with the finger of their stop, it is temporary transient visible that on-screen keyboard will become.

In still another aspect of the invention, carry out on the edge of the shell in the touch sensitive region outside as the user such as two and knock or during triple predetermined operation of knocking, it is visible that on-screen keyboard becomes.

In one aspect of the invention, the original position row stops four eight keys that finger stops that key is defined as each hand.In still another aspect of the invention, comparable eight keys of stop key are few, to allow the user, can not use eight all fingers.

In still another aspect of the invention, system makes the key of intention indefinite according to specific finger along the movement that is intended to direction.For example, the user promotes their nameless and moves down slightly it and knock.The user may also not have mobile enough far away to arrive the virtual location of adjacent key, but their intention is to select it obviously, and reason is that they move definable threshold distance and are knocked along the direction of adjacent key from their stop place.Even may not occur knocking on adjacent key at this example, system also will be selected it.

In still another aspect of the invention, the probability of each key that the text order adjustment of system based on before being close to selected.This probability is combined with the beating position algorithm of describing in paragraph in front, with the key of determining that user's most probable knocks.

In still another aspect of the invention, when the user keys on on-screen keyboard, system is automatically tackled " user's drift ".In the situation that do not benefit from the sense of touch of each key, the user is the slight finger that moves them when they key at an easy rate.The physical location tracking behavior that system is knocked by relatively being intended to Jian center and user.If consistent drift detected on the space of continuous key events, system makes the position excursion of key therefore to adapt to drift so.And, be not to make the user be concerned about where key is in, but system moves to the position of the finger of consumer positioning by key.

If user drift is too far to arrive the dispersal point of touch sensitive region, so system with can listen, vision and/or vibrate tip-off they.

In another aspect of this invention, method and system monitor on the surface in portable computing but not the user in the border of touch sensor knock.For example, but the edge of the shell of user's knocking device with the indication space bar, activate.About other the event of knocking, system makes signal correction from touch sensor and vibration transducer to determine beating position.When not the existing of signal being detected by touch sensor, system is " knock outside " (that is, the lip-deep of device knocks, but in the outside, border of touch sensor) by event recognition.The position that knock according to them on shell outside produces unique vibrational waveform.The characteristic of these waveforms is stored in database, and is used to identify uniquely the general position that knock outside.Once knocking to be identified, outside just can be assigned to keypad function (such as space or backspace).

The accompanying drawing explanation

Below, describe preferred and alternate example of the present invention in detail with reference to following accompanying drawing:

Fig. 1 means the block diagram of the example system formed according to embodiments of the invention;

Fig. 2 A～2F means by the process flow diagram of the exemplary process of the execution of the system shown in Fig. 1;

Fig. 3 A is the schematic diagram with board device of the flat surfaces dummy keyboard formed according to embodiments of the invention;

Fig. 3 B and Fig. 3 C illustrate the keyboard display formed according to embodiments of the invention.

Embodiment

Fig. 1 means the block diagram of the exemplary means 100 of the self-adaptation on-screen keyboard user interface for the digital alphabet input is provided.Device 100 comprises the processor to CPU() 110 provide one or more touch sensors 120 of input.When surface is touched, touch sensor 120 is to processor 110 notice contact events.In one embodiment, touch sensor 120 or processor 110 comprise explanation by the original signal of touch sensor 120 generations and pass through the known communication protocol of use by visible FPDP and transmit the hardware control of information to processor 110.Device 100 comprises when being knocked on surface with the similar mode of touch sensor 120 the one or more vibration transducers 130 to processor 110 transmission signals.The signal of processor 110 based on receiving from sensor 120,130 is created in display 140(touch-surface) keyboard image that presents.Loudspeaker 150 also is coupled with processor 110, makes any suitable audible signal for example, as instructing (, rub-out signal) to be delivered to the user.Vib. 155 also is coupled with processor 110, for example, so that suitable touch feedback (, rub-out signal) to be provided to the user.Processor 110 and storer 160 data communication, but the combination that this storer 160 comprises temporary transient and/or permanent storage and read-only and write store (random access memory or RAM), ROM (read-only memory) (ROM) but, such as the write non-volatile memory of FLASH storer, hard drive, floppy disk etc.Storer 160 comprise comprise such as operating system 171, self-adaptation on-screen keyboard (" OKS ") component software 172 and any other all programs of application program 173 and the program storage 170 of software.Storer 160 also comprises the data-carrier store 180 of the record 182 that comprises numerical data base 181, user option and preference and any other data 183 that received by any key element of installing 100.

Once the signal based on from sensor 120,130 detects original position by processor 110 and acts part, processor 110 is just located the virtual screen upper keyboard on display 140 below user's finger.When the user keys in, processor 110 monitors user's the position of finger and the beating position that each key activates consistently, and position, orientation and the size of each key (with whole keyboard) are adjusted, and to guarantee on-screen keyboard, is positioned at the position that the user keys in.By this way, can tackle user's " drift " or make the frame out original position of upper keyboard of their finger.If the user drifts about too far to arrive the edge in touch sensor zone along a direction, processor 110 outputs can be listened and/or tactile alert so.

At any time, the user can manually redistribute by initialization original position row definition event (describing) position of on-screen keyboard in the above.

In one embodiment, the key (F key and J key on typical english keyboard) that makes their forefinger be positioned to be commonly referred to as " home key " as the user, while going up, provides tactile feedback by Vib. 155.In one embodiment, by use the vibration of slightly different frequency for left and right, when the user makes their finger stay on key, send the vibration of moment.By this way, when user's selection processor 110, when dynamically not changing the position of on-screen keyboard, the user can select their hand is retracted to fixing original position line position.In another embodiment, the intensity of these vibrations can change according to the finger position of the home key of the original position row with respect to fixing.

Device 100 allows users in the situation that do not see that their finger or dummy keyboard key in.Thereby keyboard does not need visible at any time.This allows the purposes of valuable screen space for other.

In one embodiment, between one or more in following state of the visual appearance of keyboard, change: visible, part is visible, invisible and translucent.Make their finger in the situation that while not keying in the threshold time amount that stop can set, full keyboard visually occurs when original position row definition event occurring or as the user.When the user begins typing, keyboard is hidden to be taken off for invisible, until the user carries out including but not limited to original position row definition event, suspends any of keying in, pressing in other a large amount of operation of unique identification gesture of four fingers or some simultaneously.In another embodiment, keyboard is not hidden to be taken off for fully invisible, but becomes translucent, and therefore, where the user still can perceive key, but also can see in on-screen keyboard " below " screen on content.

In one embodiment, the key that keyboard temporary transient " lighting " knocks and apart from proportional translucent mode hard-pressed bale, enclosing the key of the key knocked or making it visible with the key to knocking.This illustrates the zone of knocking of keyboard at short notice.

In one embodiment, keyboard become " part " visible, make key and this probability with the selected probability of the highest next one light pro rata.As long as the user knocks key, it is visible or partly visible that the key that may follow just becomes.The more selecteed key of possibility is more visible, and vice versa.By this way, keyboard " is lighted " path of user to most probable next key.

In one embodiment, knocking gesture (such as rapid continuous dual knocking or triple knocking) by user's execution on the outer rim of the shell that surrounds touch sensitive surface makes on-screen keyboard temporarily visible.

Can in user interface program, by Preferences, by the user, be selected the various patterns of the visual performance of on-screen keyboard.

Fig. 2 A～2F means the exemplary process of being carried out by device 100.Fig. 2 A～2F will describe the whole of software of the present invention fully in detail, but for illustrative purposes.

The processing 200 that instruction based on being provided by OSK component software 172 is carried out by processor 100 is provided Fig. 2 A.In piece 206, when at first starting to process 200, such as the various system variables of quantity, drift distance threshold value and the key threshold value of the minimum residence time, finger touch threshold value, be initialised.In piece 208, process 200 waits and the notice of contact in the zone of touch-screen, occurred.Then, in piece 210, the one or more signal based on from sensor 120,130, occur that the original position row detects.Describing in more detail the original position row in Fig. 2 B detects.In piece 212, determine the position of the key of dummy keyboard that will be shown based on sensor signal.Describing in more detail key position in Fig. 2 C determines.Then, in piece 216, process key and activate (more details are referring to Fig. 2 D and Fig. 2 E).In piece 218, the finger drift based on the sensor signal detection user.The finger drift is described in more detail in Fig. 2 F.Then, in piece 220, based in piece 210～218, carry out determine at least one, present dummy keyboard on display 140.When the user removes their eight fingers and then contacts with touch-screen, process 200 repetitions.

Fig. 2 B means original position row Check processing 210.In determining piece 234, process 210 and determine whether the user makes their finger stay in time quantum minimum on touch-screen (that is, the minimum threshold value that stops).In determining piece 236, process 210 and determine whether the finger of right quantity stays on touch-surface, start thus original position row definition event.If do not meet the condition in

piece

234 or 236, process so 210 in the situation that do not change the position of on-screen keyboard and exit.

When meeting the T/A that stops the finger requirement, process 110 and determine the positions that stop finger, see piece 240.Then determine KeySpaceIndex(or " KSI " in piece 242) value.KSI is used to as the size of user's finger and interval customization on-screen keyboard.

KSI can change between different original position row definition events, even for same user.In one embodiment, all four fingers of each hand stay on touch-surface to start original position row definition event.In this case, provide KSI by following formula:

KSI=(on average stops the key interval)/(modelling nominal spacing)=[(a+b+c)/3]/A=(a+b+c)/3A

Here,

Modelling nominal range (being generally 19mm) between the A=key

A=stops key 1 and stops the measuring distance between key 2

B=stops key 2 and stops the distance between key 3

C=stops key 3 and stops the distance between key 4

If use the stop that is less than four to point to start original position row definition event (being defined in one group of user preference in being stored in database), can therefore adjust the KSI formula so.Use KSI in processing subsequently.

The data model of standard screen upper keyboard is stored in the storer of system.In this data model, the on-screen keyboard layout is divided into two parts: the key of generally keying in the right hand and the key of generally keying in left hand.And each key stops key relevant (being defined as " the relevant key that stops ") with the original position row that most probable is keyed in the finger stop of this specific key.The position of each key is defined as the relative measurement of the stop key relevant from it in data model.

The example formula that is used for the position of definite each key is:

Key(x′，y′)=KeyModel（x*KSI，y*KSI）

Here,

The x distance at the center to relevant stop key (RRK) of the storage of x=nominal

The y distance at the DaoRRK center of the storage of y=nominal

The key position of the modification of two or more keys may be overlapping.If this occurs, reduce so the size of overlapping key, until eliminate overlapping.

Stop for each orientation that key is determined the X-Y axle individually.For each in the sector, left and right, curve is suitable for the key of the stop in this sector.Then the X-Y axle of each key is oriented to tangent line (for the x axle) and the quadrature tangent line (for the y axle) of the supercentral curve of this key.

Fig. 2 C means to distribute key position to process 212.For each key of keyboard, re-treatment 212.In piece 252, with respect to its relevant key position that stops of form [RestingKey, Δ x, Δ y], from the pre-stored position of database 181 each keys of retrieval.For example, represent that the key of letter " R " is generally letter " F " with stop key L1() relevant, and be positioned at the upper left side of L1.Therefore, its data group is [L1 ,-5,19] (take millimeter measured for unit).The database 181 of data from to(for) each key retrieval of similar.In piece 254, by being multiplied by KSI from the skew of database retrieval, for each key, calculate cenotype to skew.In piece 258, then by the skew by new, be added on the absolute position of the relevant stop key of determining in piece 254, determine the absolute coordinates of each key.In determining piece 260, process 212 and test to observe whether any key is overlapping, if so, adjust so their size and position in piece 262, any overlapping to eliminate.Then, process 212 and return to processing 200.

Fig. 2 D means to process key actuation process 216, and actual key events is determined and exports thus.Whether process 216 has occurred starting during the decision piece 270 of event is knocked in effective touch in test.As at Marsden etc., in U.S. Patent application series No.2009/0073128, more completely explain like that, by the correlationship between touch sensor 120 and vibration transducer 130, determine it.In piece 272, by the utility key scoring algorithm, be the Candidate Key scoring.Then key with the highest mark is output in piece 274, and processes 216 and return.

Fig. 2 E means from the processing of the key scoring algorithm of the piece 272 of Fig. 2 D.In piece 280, the signal received by touch sensor 120 and vibration transducer 130 is correlated with, and take and determines where occur that the user's knocks and will be close to adjacent key definition as " Candidate Key ".By considering tapping areas key (not being to adjust the key knocked to occur) on every side, processor 110 reply users key in the uncertainty of pattern.In piece 282, process 272 and test to observe the user whether from stopping finger that key moves them to key in.Note, in typical key entry pattern, even 10 finger touch typists stop four all fingers also non-constantly at any time.Therefore, in order on effective key, to key in the variation that the stop key occurs, not condition precedent.But, as explaining in piece 284, if (or being Candidate Key self as it) do not appear changing near the state of stop key Candidate Key, can obtain Useful Information from this variation so.In piece 284, between near the stop key that the detected state calculated changes knocking and the position of knocking, calculate dummy line in piece 280.Dummy line is extended beating position.In piece 284, determine that projection line passes or the key of process, and therefore processor 110 increases the mark of these keys.By this way, even beating position directly on key, do not occur, relatively moving of the direction of the key of hope is also relevant to this key.In piece 288, with the language data in being stored in data-carrier store 181, to compare, processor 110 is considered word and the character of the front of key entry.This comprises the common known disambiguation method to predicting in prediction and word between statistical frequency, partial match estimation, word such as letter.Distribute suitable scoring for each Candidate Key.In piece 290, the Candidate Key of the highest score of the highest calculating probability of determining that intention with representative of consumer selects, and process 272 and return.

Drift Check processing 218 when Fig. 2 F means to move inadvertently their hand (or " drift ") as the user for adaptation when keying in.Process 218 in piece 300 the current center of the intention key of actual beating position and demonstration, and be Δ X and Δ Y by the Difference Storage of X coordinate and Y coordinate.In piece 302, these differences are added on the cusum from the thump of front.In determining piece 304, processor 110 is tested the variable whether accumulated deficiency of direction arbitrarily surpass the pre-stored that is called " DriftThreshold " (limiting from user preference or the default data that is stored in database 182).If surpass threshold value, processor 110 moves all Δ s and the mean value of Δ Y from last position definition event by the position of whole keyboard in piece 308 so.If accumulated deficiency is no more than DriftThreshold for whole keyboard, carry out so the similar calculating of the key of single selection in piece 316.In determining piece 318, whether the accumulated deficiency of processor 110 these single keys of test surpasses user-defined key threshold value after piece 316, and, if so, adjust so its position in piece 320.To the current location of relevant key, compare, the key threshold value is the permissible error amount of the position of knocking.When surpassing the key threshold value, mobile relevant key.After piece 308, decision in if block 318 is No, perhaps, thus after piece 320 in piece 310, whether still processor 110 tests any and any other overlapping and whole keyboard of key in reposition whether in the border in touch sensor so.If have any conflict for test arbitrarily, proofread and correct them with " best-fit " algorithm so in piece 312, and then exit.And, if do not find conflict, process so 218 and return.

Even method of the present invention allows the user to key in the sightless situation of on-screen keyboard, when also still existing the user to wish to observe key.For example, if they do not know which key is relevant to the character of hope, or in the situation that some character be positioned on independent numeral and/or symbol layer.Where other user can not memorize mechanically key entry in by know each character from memory.For these situations and the reason that comes across other, it is very important visually on the screen of device, presenting on-screen keyboard.

According to the user's who stores preference, it is visible that on-screen keyboard can keep continuously when being keyed in.As an alternative, on-screen keyboard becomes transparent after capable definition event in position.In one embodiment, it is translucent that on-screen keyboard becomes, and to allow the user, by keyboard, sees the content on following screen.

Be set as in sightless situation at keyboard, can on whole screen, show other content.Can have other the user interface element such as button, these user interface elements look like movable and are positioned at below invisible on-screen keyboard.In this case, install the user input that this element is pointed in 100 interceptions, and cause on-screen keyboard to become visible, thus reminding user it in fact exist.Then the user can select " packing up " keyboard by the corresponding key of pressing on keyboard.Note, pack up keyboard and make them invisible different.Pack up keyboard mean together " minimizing " it so that it frames out, this is the general reality of touch panel device.

In one embodiment, on-screen keyboard when the user keys in visible and invisible between the circulation.Whenever the user knocks " hide " on-screen keyboard, hidden taking off after the of short duration appearance of on-screen keyboard the time quantum that can set the user.

In one embodiment, only some key becomes visible after each thump.The temporary transient visible key that becomes is those keys that most probable is followed the text input order that is close to front (word database in being stored in system is determined).

In one embodiment, when finger stays in the variation of user based on by touch sensor 120 sensings on the original position line position and presses down surface with the finger of their stop, it is temporary transient visible that on-screen keyboard will become.

In one embodiment, carry out on the edge of the shell in the touch sensitive region outside as the user such as two and knock or during triple predetermined operation of knocking, it is visible that on-screen keyboard becomes.

When existing generally visually when inserting carat text of (or similarly indication) representative and insert condition (by operating system 171 indications), if being set as appearance, on-screen keyboard generally will occur.

In one embodiment, when the user stays on F and J original position line unit by their finger location, by tactile feedback (such as the vibration caused on the touch-screen) tactile markings that simulation is generally used on these keys is provided.By this way, the user can be chosen in upper being set up of same screen for keyboard and keep static, only by touching (not seeing), finds again the correct position of their hand.

In order to increase the precision of keyboard, use statistical language model.Produce uncertain key selection if touch/knock event, call the key of statistical model to provide user's most probable to wish by processor 110 so.

Should " eliminating uncertain " different from other method of text input system for other, reason be in the present invention, must carry out hastily the permanent decision about the key of hope.Do not exist and can show the suffix description that selected ci poem is selected and the output of revising to the user.In fact, whenever the user knocks key, just must make decision and must activate to destination application (that is, text is keyed in program) send key.

Can use several statistical analysis techniques: part is mated letter prediction, the prediction of current word, next word prediction and is combined next word and predict.Explain them in following part.

Pass through partial match estimation

The known algorithm that is initially useful in this case data compression invention is by partial match estimation (or PPM).When being applied to keyboard, the given character string occurred (thering is length k), the PPM algorithm is used to predict most probable next character.Value computing time and resource growth index with k.Therefore, preferably use minimum and produce the uncertain k as a result of acceptable elimination value.

As an example, make k=2.Two characters of the front of having keyed in are recollected in processing of the present invention, and the probability relatively of the database by next character of most probable that will key in then.For example, the following letter representation with underscore is used to predict the letter of next most probable letter:

An

A n

An? e

An? ex

An?e xa

An?ex am

An?exa mp

An?exam pl

An?examp le

For the sum of possible key A, the needed data of this algorithm are stored as:

A ^k+1

For typical on-screen keyboard, this processing consumes the data that are less than 1MB.

Build statistical model (although thering is less value for k) for each language; For the language with common root, this table can be similar.When the user keys in text, model also dynamically updates.By this way, system grasp user keys in pattern and predicts more accurately in time them.

Form with the specific dictionary of language by the configuration of operating system control panel provides language variant.Control panel is identified active user's language from the system place, and selects suitable prediction dictionary.By " systray " application of the continuous operation that also provides neologisms identification and common word to use scoring, inquiry dictionary are provided.

In one embodiment, the database that uses the everyday words in language to form, activate elimination with the key by intention uncertain.Algorithm is more such letter of keying in and word database only far awayly, and most probable next letter of coupling prediction in based on database then.

For example, for example the user has keyed in " Hel ".Possible coupling in word database is:

Hello（50）

Help（20）

Hell（15）

Helicopter（10）

Hellacious（5）

The numerical value representative of each word outside is normalized to 100 their use " frequency ".(for convenient, the sum frequency in this example adds up 100; But not generally this situation).

The candidate letter that most probable is followed " Hel " is:

L(70)-probability of adding for word " Hello ", " Hell " and " Hellacious "

P（20）

I（20）

This example is useful especially, and reason is that alphabetical L, P and I approach mutually.The user for example may even probably knock, on the position of adjacent indefinitely several keys (, I, O, P or L).By increasing Word prediction, select obviously to obtain clear and definite; In the present example, obvious most probable next letter is " L ".

Note, it is different from the conventional algorithm for on-screen keyboard that this of Word prediction algorithm realized, reason is that it is not real Word prediction system: it is the alphabetical prognoses system of using word database.

In one embodiment, the key that uses word to select further to define most probable.In simple Word prediction, there do not is the linguistic context of the first letter of clearly current word; It is fully uncertain.(for the second letter of word, the uncertain slight reduction of this elimination, for the remainder of word, reduce similarly).By considering the word of keying in previously being close to current word, can obviously reduce the uncertain character of front several letters of word; This is called as " next Word prediction ".

For example, if the word of just having keyed in is " Cleankeys ", next the general word be stored in so in database may be:

Keyboard（80）

Inc.（20）

Is（20）

Will（15）

Makes（10）

Touch（5）

If the user knocks indefinitely for the beginning of next word between I and K key, next Word prediction algorithm can help to eliminate uncertain (in this case, " K " will win) so.

Logic can be indicated the concept of the word that can key in previously for the k keyed in previously word enforcement consideration.For example, for k=2, system can be stored the database that has next word (or lower word) of the 2nd degree for each word in database.In other words, the word of the first two word to determine that most probable is followed recollected in combination.But, about space and computing power, this becomes dumb very soon.The so many combinations of storage are unpractiaca, neither be very useful, and reason is that most these combinations never occur.

But, have considerable important exception: the word with next very many word candidates.For the part of the statement that is called conjunction and article, it is this situation.

Seven the most frequently used conjunctions in English are:

and,but,or,for,yet,so,nor。

Article in English is:

the,a,an。

By becoming privileged this 10 words, system enhancement the first letter prediction.

Consider phrase: kick the_

Due to most likely next word candidates of article " the " of each noun in database, therefore, seldom there is purposes to depart from next Word prediction algorithm.But, if keep the linguistic context of article " the " " kick " before, obtain so under abundanter the selected ci poem that places an order and select.Effectively, new " word " is stored in the database that is called " kick_the ".This new entity has next following word candidates:

Ball（50）

Bucket（20）

Habit（15）

Can（10）

Tire（5）

Therefore, can predict assuredly that most probable next letter of following phrase " kick_the_ " is letter " B ".

The word that any and conjunction or article combination are found and these parts of language combine to form new word entities.

Obvious difference between letter-by-letter prognoses system described herein and the prognoses system based on word is the ability of the prediction of each letter of dynamically educating again.For example, if conjecture is wrong for specific key and word that wish becomes clear subsequently, algorithm is abandoned the selection that it carries out for incorrect letter so, and the target word based on newly definite is for remaining alphabetical applied forcasting.

For example,

Along with the word development, mean that initial letter " B " should be " H " (these letters is adjacent one another are on the qwerty keyboard layout, and are easy to mistake with other).But, not entrust in this first letter fully and only consider the word started with " B ", system is still considered other candidate when prediction the second letter.Therefore, B, H and G are regarded as the first letter of key subsequently.By this way, mistake is not propagated, and the user only needs once to proofread and correct rather than may carry out correction many times.

Therefore, for each new key of keying in, adjacent key and other uncertain candidate are regarded as possibility when determining subsequently alphabetical.

When mistake and user's backspace occurring and proofreading and correct it, system can be adjusted data feedback in algorithm and therefore.

For example, the user keys in the key of the centre of keyboard indefinitely, and scoring algorithm to indicate possible candidate be " H ", " J " and " N "; The mark of these three letters falls in acceptable scope, and, obtain best mark.In the present example, for example algorithm is returned as most probable candidate by letter " J ", so it is the key of keyboard output.And then after it, key in clearly<backspace of user > and " H ", so error recovery.

This information is fed back to scoring algorithm, and which subalgorithm this scoring algorithm is watched and obtained " H " that proportion by subtraction " J " is high when the indefinite key of initial key entry.The weighting of these algorithms increases, and therefore, if same indefinite input occurs again, letter " H " can be selected so.By this way, directly based on the user, proofread and correct the feedback circulation is provided.

Certainly, the user may self key in the mistake of the result that is not algorithm; It is correctly exported, and the user keys in.Therefore, necessary careful when determining whether to answer initialization user correction feedback circulation time.Its general only key under discussion occurs when indefinite.

The user can set option and can allow keyboard to send backspace and allow new letter to proofread and correct the word of apparent error.In above example, when fallout predictor determines that only logic word is selected to be " habit ", keyboard can send backspace, and " b " become to " h ", and again sends letter (the even whole word of possibility) subsequently.

Because too many factor causes the clear and definite of key, therefore, add all algorithms to the qualifications for being elected of key potentially.The method is called as scoring; All algorithms are weighted and then are added in together.Weighting dynamically changes, evaluation algorithms is adjusted to user's key entry pattern and environment.

Fig. 3 A mean representative at it schematic diagram towards the typical hand-held flat computer 350 of the touch-sensitive display 352 that there are design and use according to an embodiment of the invention on front surface and keyboard 354.Keyboard 354, when being used according to the present invention, produces the text of the text insertion position 360 that outputs to text display area 358.Term in the application's book " keyboard " refers to touching and knocking any keyboard of realizing on sensing surface, is included in the keyboard presented on touch-sensitive display.The alphabetic(al) letter of each language that keyboard 354 expressions are selected on each single key by the user, these letters are arranged and are configured with the standard " QWERTY " of finding on most keyboard.

In one embodiment, change orientation, position and the size (and each single key) of keyboard according to user's input behavior self application ground.When the user makes their finger stay on touch-surface 352 in some way, system moves to keyboard 354 by stopping the definite position of finger.When the user will drive key on keyboard 354, they are by the finger that promotes them key that knocks surface 352 " knocking " hope by the power do not perceiveed.The user who occurs on the zone 362,364 of 352 outsides, touch sensor zone knocks by vibration transducer and detects, and also can be assigned to the keypad function such as the backspace bar.

Not having touch sensor signal is in fact the signal with null value, and, when to knock (or vibration) when relevant, sensor can be used to identify uniquely beating position.In one embodiment, such as the vibration signal of the specific region of 352 outsides, touch sensor zone of indication on zone 362,364, be unique, and by system storage in database.When in conjunction with the event of knocking while not existing of touch signal occurring, the vibration characteristics that systematic comparison is knocked be stored in those in storehouse, with the position of determining that outside is knocked.In one embodiment, bottom outer border area 362 is assigned to space function, and right part outer border area 364 is assigned to back space function.

Fig. 3 B is the schematic diagram that represents exemplary virtual screen upper keyboard 370.Keyboard 370 is divided into two halves.Half is 372 relevant to user's right-hand man with half 374(of the right side on the left side).Two independent Ban districts are not aimed at mutually.General eight keys that stop 378 of user are according to generally for key, using which finger for example, by mark black (, L1 represents the forefinger of left hand, and L4 represents the little finger of toe of left hand, etc.).All other ex situ line units mean by meaning which finger is generally used for by using conventional touch to key in the label that technical key enters this key.But it should be noted that and have many key entry patterns that do not utilize the finger shown in Fig. 3 B to place, and those labels that comprise here are only for illustrative purposes.

Half means that all keys aim at along horizontal line the left side of keyboard 372, as they on conventional electricapparatus keyboard.In half embodiment shown in 374 of the right side, the original position line unit disperses along circular arc, with the normal stop place of four fingers coordinating better the user.The relative position that the ex situ line unit stops key according to them and original position row similarly disperses.And in one embodiment, the size of each key also can change (probability is higher, and key is larger) according to the statistical probability of user's options button.

Fig. 3 C represents according to embodiments of the invention the schematic diagram with the virtual screen upper keyboard 384 of certain angular orientation.The user can make any position that their hand 390 is wished at them stay on the touch sensitive surface 392 of typical hand-held flat computer 394 with being orientated.In this case, hand with normal condition, compare further separate and take install 394 straight flange as benchmark with certain angular orientation.The user starts the operation of indication " original position row definition event ", and this operation is including but not limited to following operation: stop all eight and point the times that of short duration user can limit; Simultaneously twoly on surface 392 knock eight all fingers and then make them stay on surface 392; Perhaps at eight fingers, stay on surface 392 and press down eight all fingers simultaneously.In another embodiment, do not need eight all fingers to start original position row definition event.For example, if someone does not have middle finger, so can be only by three finger startup original position row definition events on hand.Here, the user makes their hand 290 stay on flat computer 394 with certain angle, causes thus the processor of computing machine 394 produce and show virtual screen upper keyboard 384 with certain angle.

Require the embodiments of the invention of proprietary entitlement or privilege to be defined as follows:

Claims

1. a device comprises:

Display;

With display-coupled a plurality of touch sensors, a plurality of touch sensors are configured to contact the generation sensing signal based on sensing with user display;

A plurality of movable sensors, a plurality of movable sensors are configured to the generation of vibration movable signal of the shell based on sensing;

The processor of communicating by letter with display, a plurality of touch sensor and a plurality of motion sensor signals, wherein, processor is configured at least one generation in the movable signal of sensing signal based on producing or generation and presents the image of the keyboard with a plurality of keys on display; With

Be configured to hold the shell of display, a plurality of touch sensor, a plurality of movable sensor and processor.

2. according to the device of claim 1, wherein, processor is configured to the position that sensing signal based on producing is determined the keyboard image on display.

3. according to the device of claim 1, wherein, processor is configured to exist original position row definition event to determine the position of the keyboard image on display based on determining.

4. according to the device of claim 3, wherein, when the sensing signal of two or more generations is confirmed as when movable for the scheduled volume time, processor is determined the original position row definition event that exists.

5. according to the device of claim 4, wherein, processor is configured to:

Determining of the position of two or more sensing signals based on producing, determine the position of the original position line unit of keyboard image; With

At least one definite position based in the original position line unit, determine the position of the ex situ line unit of keyboard image.

6. according to the device of claim 4, wherein, processor is configured to:

Determining of the position of two or more sensing signals based on producing, determine the size of the original position line unit of keyboard image; With

At least one definite position based in the original position line unit, determine the size of the ex situ line unit of keyboard image.

7. according to the device of claim 4, wherein, processor is configured to:

Determining of the position of two or more sensing signals based on producing, determine the orientation of the original position line unit of keyboard image; With

At least one definite position based in the original position line unit, determine the orientation of the ex situ line unit of keyboard image.

8. according to the device of claim 3, wherein, shell also comprises the vibrating device of the vibration that is configured to produce one or more frequencies, and wherein, processor is configured to cause vibrating device to activate under predetermined frequency based on original position row definition event.

9. according to the device of claim 3, wherein, shell also comprises the vibrating device of the vibration that is configured to produce one or more frequencies, and wherein, processor is configured to:

Place the keyboard presented in the still-mode of operation;

Determine the position of at least one user's finger based on sensor signal; With

When definite position of at least one user finger, when at least one home key is in threshold distance, cause vibrating device to produce the vibration of preset frequency.

10. according to the device of claim 9, wherein, vibrating device is configured to point based at least one user the intensity of the distance change vibration of corresponding home key.

11., according to the device of claim 3, wherein, shell also comprises the audio devices of the sound signal that is configured to produce one or more frequencies, wherein, processor is configured to:

Place the keyboard presented in the still-mode of operation;

When definite position of at least one user finger, when at least one home key is in threshold distance, cause audio devices to produce the sound signal of preset frequency.

12., according to the device of claim 11, wherein, audio devices is configured to point based at least one user the intensity of the distance change sound signal of corresponding home key.

13., according to the device of claim 1, wherein, processor is configured to:

Property ground receives the sensing signal relevant with the Continuous Contact of display to user's finger during this time;

During determine receiving, whether sensing signal indicates from the generation at keyboard image and the drift of the position of the sensing signal of use presenting; With

Drift based at least one key of indication, at least one key of mobile keyboard image on display.

14. according to the device of claim 13, also comprise output unit, wherein, processor is configured to:

During determining, whether the drift of the sensing signal indicating user finger contact of property reception is in the threshold distance at the edge in display; With

If the user's finger contacts drift is confirmed as in threshold distance, to the output unit output signal.

15., according to the device of claim 1, wherein, processor is configured to:

Sensing signal based on producing and the movable signal of generation, sensing user is keyed in operation; With

When not sensing the user for predetermined time amount and key in operation, keyboard image is changed into at least one state in translucent or invisible mode.

16., according to the device of claim 15, wherein, after at least one state that makes keyboard image in translucent or invisible mode, processor is configured to that to cause keyboard image to show as when sensed at least still less transparent when the user keys in operation.

17., according to the device of claim 1, wherein, processor is configured to:

Activate based on one or more previous keys the key of determining that at least one next most probable activates; With

Show uniquely at least one next most probable key of determining.

18., according to the device of claim 1, wherein, processor is configured to:

Sensing signal based on producing is determined one or more user's fingers relatively moving from the original position line unit; With

Movable signal based on producing and definite relatively moving, produce key and activate event.

19., according to the device of claim 1, wherein, processor is configured to:

At least a portion of sensing signal based on producing and the movable signal of generation produces one or more Candidate Keys; With

By eliminate the indefinite property of the one or more Candidate Keys that produce with Statistical Probabilistic Models, produce key and activate event.

20., according to the device of claim 19, wherein, processor is configured to:

Determine the size value of at least one key based on Statistical Probabilistic Models and at least one previous key activation event; With

Definite size value based at least one key, change keyboard image.

21., according to the device of claim 1, wherein, processor is configured to:

The operation of first user based on sensing causes the keyboard image that presents invisible in active state; With

The second user operation based on sensing causes the keyboard image that presents invisible in inactive state.

22. according to the device of claim 1, wherein, at least one movable signal of generation is relevant to the position with respect to shell,

Wherein, processor is configured to the location recognition function based on respect to shell when producing at least one movable signal and not producing sensing signal.