CN103649876B - Performing actions on a computing device using a contextual keyboard - Google Patents

Performing actions on a computing device using a contextual keyboard Download PDF

Info

Publication number
CN103649876B
CN103649876B CN201180065367.6A CN201180065367A CN103649876B CN 103649876 B CN103649876 B CN 103649876B CN 201180065367 A CN201180065367 A CN 201180065367A CN 103649876 B CN103649876 B CN 103649876B
Authority
CN
China
Prior art keywords
described
keyboard
input
text
operation
Prior art date
Application number
CN201180065367.6A
Other languages
Chinese (zh)
Other versions
CN103649876A (en
Inventor
克利福德·A·库什勒
简·曼宁
迈克·麦克谢里
亚伦·希迪
Original Assignee
纽昂斯通信有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority to US41582810P priority Critical
Priority to US61/415,828 priority
Application filed by 纽昂斯通信有限公司 filed Critical 纽昂斯通信有限公司
Priority to PCT/US2011/061530 priority patent/WO2012068544A2/en
Publication of CN103649876A publication Critical patent/CN103649876A/en
Application granted granted Critical
Publication of CN103649876B publication Critical patent/CN103649876B/en

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0487Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
    • G06F3/0488Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures
    • G06F3/04886Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures by partitioning the screen or tablet into independently controllable areas, e.g. virtual keyboards, menus
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0487Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
    • G06F3/0488Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures
    • G06F3/04883Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures for entering handwritten data, e.g. gestures, text

Abstract

A system and method for performing actions via a keyboard application of a computing device is described. In some examples, a contextual keyboard facilitates the entry of text and the performance of actions across applications supported by a device. For example, the contextual keyboard may receive a word macro and/or a gesture, and determine that the received word macro and/or gesture is a command to perform one or more actions. Example actions to be performed include searching for information, storing information, posting information, sending information, enhancing information, and so on.

Description

Execute operation using contextual keyboard on the computing device

Technical field

The present invention relates to computing device, more specifically but be not limited to execute behaviour on the computing device using contextual keyboard Make.

Background technology

The origin of the main method as the input text from people to machine for the modern keyboard, can trace back to early stage in 19th century Typewriter.With the development of computer, this is a natural evolutionary process to adapt to typewriter keyboard for as input The main method of text.For a skilled typist, keyboard remains as and inputs text into computer or other data processing The feasible the fastest mode of equipment.

Efforts be made so that computer is less and easily portable therewith, physical keyboard has become great majority restriction computing device can With one of little important limiting factor to what degree:The physical size of the finger of people is not that computer designers can change 's.As a result, the computing device of some portable applications, such as mobile device and tablet PC, it is designed without Physical keyboard, and touch screen or dummy keyboard are used as the primary interface between equipment and user.

Although can be realized by touch screen, dummy keyboard often is considered only as receiving one kind of text input from user Mechanism.That is although it is useful that many innovations have transmitted keyboard for modern comfort, but typical dummy keyboard is still Run as its older generation, only play the function of receiving text input from user.It is similarly to physical keyboard or early stage typewrites Machine makes the thing that user can do on equipment, receives text input from user.

It is therefore desirable to have a system, which overcome the problems referred to above, and extra benefit is provided.In general, existing Or correlation system in embodiments herein and its purpose of restriction of correlation is illustrative, rather than exclusiveness. Based on detailed description below, the restriction in system that other had both been deposited or existing is aobvious for those skilled in the art And be clear to.

Content of the invention

According to an aspect of the invention, it is provided a kind of 1. keyboard systems of mobile device, including:Keyboard components, quilt The touch screen being configured to pass described mobile device assumes dummy keyboard to user;Gesture receiving unit, is configured to institute State keyboard components and receive stroke gesture input;Context component is configured to identify the contextual information describing context, institute State stroke gesture input to be received in described context;Interpretation component, be configured to based on receive stroke gesture input and The contextual information of identification, without extra user input, determines that the stroke gesture input of described reception is for described The order of keyboard system, or to the input related text input of text-string, wherein, described keyboard system and described movement The other application of equipment is different;Text input assembly, is configured to described dummy keyboard reception and is determined as literary composition The stroke gesture input of this input, and the text input based on described reception updates text-string;And operating assembly, joined It is set to the stroke gesture input receiving the order being determined as described keyboard system;Stroke gesture input quilt in described reception Send to before another active application, intercept the stroke gesture input of described reception;And execution one or more with pass through The operation that the described order that described dummy keyboard receives is associated, wherein said operation is not the function of the key of described keyboard.

According to another aspect of the present invention, there is provided a kind of method by computing device, it provides context key Disk, including:Stroke gesture input is received by the keyboard being shown by described computing device, the keyboard of wherein said display is configured For receiving text input;Identification describes the contextual information of context, described stroke gesture input quilt in described context Receive;Based on the contextual information of the stroke gesture input receiving and identification, without extra user input, determine described The stroke gesture input receiving whether include with by described contextual keyboard pending operate related order, wherein, described Contextual keyboard is different from the other application of mobile device;And wherein said operation is not the work(of the key of described keyboard Energy;When the stroke gesture input of described reception is confirmed as not including the order for described keyboard applications, then:Connect described in reception The stroke gesture input received is as the input being used to help text input;And when the stroke gesture input of described reception is determined It is including the order for described keyboard applications, then:The stroke gesture input of described reception be sent to another active should With before, intercept the stroke gesture input of described reception, thus described stroke gesture input is not received as text input;And The seed object of the display that detection operation can be carried out thereon;And based on a determination that order and detection seed object, hold The related operation of row.

According to another aspect of the present invention, there is provided a kind of method of the keyboard applications program performing by mobile device, Methods described includes:Stroke gesture input is received by the dummy keyboard of described keyboard applications, wherein said dummy keyboard is joined It is set to reception text input;Identification describes the contextual information of context, and described stroke gesture input is in described context Received;Based on the contextual information of the stroke gesture input receiving and identification, without extra user input, determine institute State reception stroke gesture input whether include to by the pending related order of operation of described keyboard applications, wherein, described Keyboard applications are different from the other application of described mobile device;And wherein said operation is not the work(of the key of described keyboard Energy;When the stroke gesture input of described reception is confirmed as not including the order for described keyboard applications, then:Connect described in reception The stroke gesture input received is as the input being used to help text input;And when the stroke gesture input of described reception is determined It is including the order for described keyboard applications, then:The stroke gesture input of described reception be sent to another active should With before, intercept the stroke gesture input of described reception, thus described stroke gesture input is not received as text input;And Based on a determination that order execute described associated operation.

Brief description

Fig. 1 is the block diagram of the assembly that mobile device or other suitable computing device are described.

Fig. 2 is the block diagram of the assembly of keyboard applications program that computing device is described.

Fig. 3 is the block diagram of the assembly of the operation module that keyboard applications program is described.

Fig. 4 be by keyboard applications program performing operate routine flow chart.

Fig. 5 is the flow chart for the execution routine of operation related to seed for the explanation.

Whether Fig. 6 is explanation is the stream of the routine of a part of order or text-string for determining the text of input Cheng Tu.

Fig. 7 is the flow chart of the routine illustrating to operate by keyboard applications program performing based on the text being transfused to.

Fig. 8 is explanation for executing the flow process of the routine of the operation related to the gesture receiving by keyboard applications program Figure.

Fig. 9 is that explanation is used for sending information to destination application, to respond the gesture being received by keyboard applications program The flow chart of routine.

Figure 10 A-10D is the screenshot capture illustrating when starting the step being performed during information search.

Figure 11 A-11C is the screenshot capture illustrating when starting the step being performed during information search.

Figure 12 A-12B is that the screen that the step being performed when insertion link or other information are to application program is described cuts Figure.

Figure 13 A-13B is to illustrate that the screen of the step being performed cuts when insertion link or other information are to application program Figure.

Figure 14 A-14D is to illustrate that the screen of the step being performed cuts when inserting map or other information to application program Figure.

Figure 15 A-15B is the screenshot capture of the step illustrating, when storage information in data base, to be performed.

Figure 16 A-16B is the screenshot capture of the step illustrating, when storage information in data base, to be performed.

Figure 17 A-17B is the screenshot capture of the step illustrating, when releasing news to website or other positions, to be performed.

Figure 18 A-18B is the screenshot capture of the step illustrating, when releasing news to website or other positions, to be performed.

Specific embodiment

General introduction

For the system and method by the keyboard applications program performing operation of computing device, such as mobile device or flat board sets Standby, it is described.In certain embodiments, system passes through the keyboard showing on the touchscreen or other have the shown of device program Dummy keyboard receives input, determine that the input that receives is associated with operation to be performed, and execute and/or start-up operation Performance.

Now, the relevant various embodiments of system will be illustrated.Explained below provides thoroughly for the embodiment of system The thorough detail understanding and enable explanation.It will be understood by those skilled in the art, however, that this system can also without these details It is carried out.In other embodiments, known 26S Proteasome Structure and Function is not illustrated or is described in detail, to avoid system is implemented Unnecessarily the obscuring of description of example.

Its objective is the term used in the description that is provided below, to be explained with its widest reasonable manner, Even if it is used together by the detailed description of some specific embodiments together with system.Some terms are possibly even emphasised, but It is that the term that any intention is explained with any ways to restrain is defined as here detailed description portion to be disclosedly and specifically Point.

Suitable system

Fig. 1 illustrates the block diagram of computing device 100, is able to carry out for example in computing device supported keyboard applications program The operation of " contextual keyboard ".Computing device 100 can be mobile device, smart phone, panel computer, network book, mobile GPS Navigator, remote control equipment, landline telephone or communication control console or device, surface or desktop computer, projector, image projection instrument, Desk computer, electronic reader, ATM, automatic vending machine or there is the keyboard that the user interface of equipment presents, for example virtual Or be based on keyboard with touch screen, any other equipment.Computing device 100 includes various hardware and/or component software, is configured to Based on the input being received by the keyboard of computing device 100 to execute operation.For example, computing device 100 includes various man machine interfaces Assembly 110, apparatus assembly 120 and memorizer 130.

Computing device 100 includes touch screen 112 or other provide input directly to the input module of processor 122, such as when touching When screen is touched, notifier processes device 122 contacts the input of event.Touch screen potentially includes hardware control or and hardware control Communicated, such as touch screen drives, it is explained the primary signal receiving from touch screen and sends and the event that contacts (for example, button Or button instruction, the XY coordinate of contact point (as the finger on touch screen, touch pad or drawing board or hand-written pen touch)), gesture, Tap path, user's request presses physics or virtual key, point to input equipment current location, contact area, pressure, continue Time, etc. related information to processor 122.For example, hardware control can send information, and this information is set with various instructions Standby, including mouse, trace ball, stick or simulation rod, indicating arm or projection mouse, idler wheel mouse, foot mouse, palm mouse, Light pen, light gun, position rifle, laser pointer, gyroscope or gyro mouse, accelerometer, eye move follow-up mechanism, video tracking sets Standby, writing pencil, etc. associated.Processor 122 is communicated with hardware control or driver, this driver and display 114 phase The association information (for example, the letter of the contact key keyboard of display on) related to show the event of the contact with detecting.Display Device 114 can be integrated into computing device 100, can be a part for touch screen 112, or can be independent setting Standby.Example display 114 includes touch-screen display, flat faced display, cathode ray tube, electronic ink display, wear-type Display, liquid crystal display, light emitting diode indicator, plasma panel displays, electroluminescent display, vacuum fluorescence show Show device, digital projector, laser-projector, HUD, etc..Equipment 100 can include other interface modules 116, this The speaker of sample provides suitable audible signal to assist the keyboard of user's navigation display or the assembly of other display.

Computing device 100 may include various apparatus assemblies 124, and such as (for example, GPS or other positions determine sensing to sensor Device, motion sensor, gyroscope, optical sensor, etc.), moveable storage device (for example, SIM), camera and others Video capture device, mike and other audio collecting devices, communication equipment (for example, bluetooth equipment, radio, sky Line), etc..

Processor 122 can be communicated with the data in the memorizer 130 of the equipment of being stored in 100 or application program, storage Device 130 potentially includes the combination of interim and/or permanent storage, read-only and writable memory (random access memory or RAM), only Read memorizer (ROM), writeable nonvolatile memory such as FLASH memory, hard disk drive, floppy disk, the group based on SIM Part, etc..Memory assembly includes various program assemblies or module, such as operating system 132 and various application program 134, As downloaded to the application program of mobile device 100.For example, memorizer 130 can be with locally stored application program to equipment, and it is setting Standby above carry out permanent operation (for example, the keyboard applications program of dummy keyboard be provided, text message transmission application program, etc.), And downloaded and released by equipment by user application program (for example, with social network sites, game, etc. related application Program).

For example, memorizer 130 can storage keyboard application program, such as contextual keyboard 150.Contextual keyboard 150, it is permissible Including gesture database 152, in certain embodiments, this gesture database provides the work(more powerful than QWERTY keyboard application program Can, e.g., make keyboard execute the function of operation, provide information to the function of other application programs, etc..Relevant contextual keyboard 150 and its further detail below of function will be described herein.

Storage assembly 150 also includes various data storage components, the data base 136 of such user data or other data Storehouse 139 is that the application program performed by equipment 100 provides and/or storage information.For example, customer data base 136 can store with The information that the user of equipment 100 is associated, such as associated person information, subscriber information, record information, use information, tendency information, User preference information, etc..

Generally, in other factors, equipment 100 can store or comprise any and all assembly, module, or ought execute operation When, the data file being required or being used by equipment 100 and/or contextual keyboard 150.

Fig. 1 and brief, the general description of the discussion in this suitable computing environment of offer.In the computing environment, Contextual keyboard can be performed.Although optional, each side of this system are logical in computer executable instructions It is described in normal context, such as by general computer, for example, mobile device, server computer or personal computer are held The routine of row.Various equivalent modifications will be understood that, this system can be communicated with other, data processing, or computer system Configuration is implemented, including:Internet equipment, handheld device (includes tablet PC and/or personal digital assistant (PDA)), various The honeycomb of form or mobile phone, multicomputer system, based on microprocessor or programmable consumption electronic product, machine top Box, network PC, minicomputer, mainframe computer, etc..It is true that term " computer ", " main frame " and " master computer " and " mobile device " and " mobile phone " is herein typically used interchangeably, and can refer to above-mentioned any equipment and system, and Any data processor.

The each side of system can be embodied in specific purposes computing device or data processor, and this data processor is entered Row special programming, configuration or construction are to execute one or more of computer executable instructions explained herein in detail.System Each side can carry out in a distributed computing environment, wherein task or module are executed by remote processing devices, and by communication Network connection, such as LAN (LAN), wide area network (WAN), or the Internet.In a distributed computing environment, program module can position In local and remote memory storage device.

The each side of system can be stored on computer-readable medium or be distributed (example on a computer-readable medium As, physics and/or tangible computer-readable recording medium).Computer-readable medium includes magnetically or optically readable computer CD, The chip (for example, EEPROM semiconductor chip) of hardwired or pre-programmed, nanotechnology memory, biological memory, or other Data storage medium.It is true that other data of computer executed instructions, data structure, screen display, and system aspects can (example on the transmitting signal within a period of time, propagation medium being distributed in by the Internet or other networks (inclusion wireless network) As, electromagnetic wave, sound wave etc.), or they can also be arranged on any analog or digital network (packet switch, circuit switching, Or other schemes) on.One skilled in the relevant art will recognize that partial system is located on server computer, and corresponding portion Divide and be located on client computer, such as mobile or portable set.Therefore although some hardware platforms are described in this, but it is The each side of system are equally applicable to the node on network.In another embodiment, mobile device or portable set can represent Server section, and server can represent client part.

Contextual keyboard

As described herein, in certain embodiments, contextual keyboard or contextual keyboard application program make mobile device User executes operation by keyboard, and this keyboard presents to user by computing device touch screen or other users interface.That is, one In a little examples, contextual keyboard is the keyboard applications program of computing device, and computing device is capable of identify that the text or its receiving It inputs (for example, gesture) and orders as processing, and detects that those orders are performed the application program of the computing device being based on Object in environment.Therefore, contextual keyboard can perform dual-use function, promote from user the first function of text input and The second function based on the execution operation being inputted by the order of keyboard (for example, word grand or gesture).

Keyboard is the useful place adding function.Under normal circumstances, keyboard is software layer, when using computing device and it Various application program when, software layer is usual or always addressable.Therefore, other functions or increase and key are added in keyboard The related other functions of disk will provide many benefits, easily or simply navigating, increasing between the application program on such as equipment Strong user interface capabilities and other benefits.For example, keyboard can allow users to access theirs as information exchange media On equipment or the data of position that communicates of their equipment, and hand over the application program that runs on equipment or other programs Change information.Each assembly of the contextual keyboard that can perform such technology will be discussed now.

Block diagram shown in Fig. 2 illustrates the assembly of the keyboard applications program 150 of computing device.Keyboard applications program 150 has When be also referred to as " keyboard of context ", including input module 210, be configured to receives input and generate the layer of text-string Or assembly, and operation module 220, it is configured to receives input and execute or arrange to execute related to the input being received The layer of individual or multiple operation or assembly.In some cases, keyboard applications program 150 includes data base 152, for example, be used for storing The information associated with the operation of gesture, exercisable word (for example, word is grand), execution and/or the data base 152 of other information.

In certain embodiments, input module 210 includes the assembly of conventional keypad application program, such as can provide text input Function, thump function, percussion, gesture, and/or contact movement function, or any other assistance is from the text based of user The assembly of the function of reception of input.Input module 210 can make computing device 100 show keyboard by touch screen 120, and via The shown keyboard receives input being presented by touch screen 120.Keyboard can be physical keyboard or dummy keyboard, for example any The keyboard of coining on the keyboard that presents on the keyboard that is implemented on Touch sensitive surface upper keyboard, touch-sensitive display, Touch sensitive surface, etc. Deng.Keyboard that exemplary keyboard includes showing on a monitor, the keyboard touching screen display, light are thrown on plane or curved surface Keyboard or have the electronic variable being integrated in key key symbol keyboard, etc..In some cases, keyboard is probably " empty Intend " touch, such as by screen or the projection of certain instruction equipment or gesture identification control.With regard to provided by input module 210 The further details of suitable text input application program can in commonly assigned United States Patent (USP) 7,542,029, announce in On June 2nd, 2009, entitled " SYSTEM AND METHOD FOR A USER INTERFACE FOR TEXT EDITING AND Find in MENU SELECTION ", this patent is completely herein incorporated with way of reference.

In some instances, when the user is entering text, the text input application program being provided by input module 210 is permissible The identification of execution input text and/or disambiguation technology.For example, when the continuous path on user tracking dummy keyboard, input Module 210 can provide and receive the text input application program that text is inputted.Text input application program in United States Patent (USP) 7, 098,896, announce in August in 2006 29 days, entitled " SYSTEM AND METHOD FOR CONTINUOUS STROKE It is described in more detail this patent in WORD-BASED TEXT INPUT " to be completely herein incorporated with way of reference.

In some instances, operation module 220 includes being configured to identify that the input receiving to execute operation as order Assembly, and be configured to execute or lead to execute the assembly of operation, not as being usually added to text-string Text input.Fig. 3 system illustrates the block diagram of the assembly of operation module 220 of keyboard applications program.

Operation module 220 includes being configured to the defeated of the input that reception, monitoring and/or examination are received by keyboard applications program Enter assembly 310, be configured to determine that the operation of the one or more operations being associated with the input being received determine assembly 320, Be configured to identify and/or determine with the context component 330 of the related context of input receiving, be configured to execute or Lead to the operating characteristics assembly 340 of one or more operations determined by execution, its storage letter related to the input receiving Breath and/or the data base 350 of operation to be performed and/or other are easy to the operating assembly of operation module 220, for example should with keyboard The assembly being communicated with the other assemblies of the input module 210 of program 150, or other assemblies.With regard to input module 210, behaviour The further detail below making the operation of each assembly in module 220 and/or module will be entered with respect to various routines described herein Row discusses.

In certain embodiments, input module 210 can provide information to operation module 220, for example, advantageous as with life The associated input of order identifies to execute operation.For example, when entering text, input module 210 can be using identification and/or disambiguation Technology carrys out the intention of identifying user.This technology can text be output to activity application program as common text it Before, the text of identification division input is as possible order, and provides this type of information to operation module 220.Therefore, in text quilt Before being sent to the application program of activity, operation module 220 can intercept text to execute some other operations to produce other Benefit.

As described herein, in certain embodiments, various forms and executable operation that contextual keyboard 150 will input Associated.For example, contextual keyboard 150, by the operation of operation module determine assembly 320 can identify one or more by " word is grand " in the word character string of user input.Word is grand to be and orders related text-string, such as on-unit.? In some cases, word is grand is the text-string (for example, " EMZ ") unlikely being received by keyboard.In some cases, word is grand It is the similar common word of user language or the text-string of word (for example, " email ").Under normal circumstances, word is grand will be one Quantitative text input is received by keyboard or selects to be received after or during the period.When determining that whether input is that word is grand or common literary composition This when, operation module 220 can be by the use of the text input selected as " seed ".The use of text input is herein carried out It is more fully described.

As another example, contextual keyboard 150, assembly 320 is determined by the operation of operation module 220, can recognize that One or more " gestures " being received from user by keyboard applications program 150.Gesture be finger (or stylus) pass through screen and/ Or on keyboard input data interacting with application program, and/or the movement indicating various other user views.For example, gesture It is probably the track followed the trail of on the virtual keyboard or by dummy keyboard, the shape drawn on the virtual keyboard or by dummy keyboard Shape, and/or on the virtual keyboard or by other movements of dummy keyboard.In some cases, keyboard applications program 150 can connect Receive gesture and without any contact between user and keyboard.For example, the gesture of reception is probably the gyroscope inspection in equipment 100 The movement of the pointer in certain motion of the equipment 100 measuring, or the virtual display that detects of the photographing unit of equipment 100.

Operation module 220 can store the information that and/or gesture grand with the various words in data base is associated, such as grand/handss Gesture data base 350 or gesture database 152.The information of storage potentially include with the executable grand related information of word operating and The information of gesture correlation of executable operation and the setting (for example, the preference information of the accounts information of user or user) of user Related information, etc..These information can make operation module 220 input plain text to setting by keyboard applications program 150 Determined based on the order receiving during standby 100 and execute operation, and produce other benefits.

Fig. 4 system illustrates the flow chart of routine 400 operating by keyboard applications program performing.In step 410, calculating sets Standby contextual keyboard passes through the keyboard receives input of equipment.For example, the input module 310 monitoring context of operation module 220 The contact event of keyboard, and receive the input being associated with the event that contacts.The input receiving can be that word is grand, gesture, or its The form of his input.

For example, contextual keyboard makes user execute gesture and tells what operation is contextual keyboard will execute, and execution supplies user The gesture of the menu of optional operation to be performed, execution gesture simultaneously selects using keyboard executed what operation, and/or use Keyboard keys in " word is grand " etc..

At step 420, contextual keyboard identifies that one or more operation to be performed are associated with the input being received 's.For example, the operation of operation module 220 determines that assembly 320 identifies and/or determines that received input is stored in data base In 350, and it is associated with action to be executed.Identified operation may be sent out with search information, link information, storage information Cloth information, starts application program, etc. associated.

In some cases, contextual keyboard can associate the operation of execution to the input receiving, and the input receiving is right It is specific in application program or other computing environment, the user in other computing environment is just working on equipment or using.? That is, some words are grand and/or gesture is probably that application program is specific, are considered as only that word is grand or handss in this application program Gesture.For example, text editor application can support the handss related to the operation executing printing or convert documents are PDF version Gesture, and other application programs may not support such gesture.

In some cases, contextual keyboard can identify the related operation of inputs that are two or more and receiving.Example As " email " word in response to receiving is grand, and operation determines that assembly 320 can identify and send the personal electric with user The first related operation of the Email of email account, and there is the electronics postal of the electronic mail account of user company with transmission The second related operation of part.In order to solve ambiguity, in other factors, contextual keyboard may occur in which to the options menu of user, This menu is associated with available operation to be executed, or can select most possibly will execute based on various contextual informations Operation, including action to be executed.For example, contextual keyboard can make conjecture, the choosing of the passing history based on user Select, and decide whether to make conjecture or prompting user's one option of selection, help conjecture (for example, input tool using context The meaning that number " calling " may be guessed to the left side is " this number of taking action-dial ", and indicates there is " I is ready " " calling " be possibly intended as content to the left side), etc..

In step 430, contextual keyboard execution or start (or the selecting) being identified and be associated with the input receiving Operation.For example, the operating characteristics assembly 340 of operation module 220 executes or leads to execute and determines that assembly 320 is identified by operation Operation.Execute this operation and may include and lead to application program launching, such as start short message application program, the text of transmission input is to net Stand, such as to social network sites, storage information in the local database, such as the text of storage input is to the contact data of user Storehouse, strengthens the word of input, such as adds link, map, image and/or other contexts to message, etc..

That is, in certain embodiments, contextual keyboard is implemented to lead to application program in the operation of computing device Between in system or environment execution operation.For example, the operation being performed can start in keyboard applications program and can lead in fortune Application program launching in row time system simultaneously executes one or more tasks, such as by internet access website, passes through honeycomb Communication channel sends message etc..The exemplary operation that can be performed includes:

Search engine is sent a query to by the related application running on the computing device;

Released news to website (for example, to social networkies, online by the related application running on the computing device Community, microblogging etc.);

Storage information in Local or Remote data base;

Send message or other information to addressee;

Retrieval content the content update text with retrieving, as text in the e-mail;Or

Interactive with website or other network sites, such as place or more new order, buy article, etc..

Therefore, in certain embodiments, when some words are grand or gesture passes through the shown keyboard of computing device from user When receiving, contextual keyboard is conducive to the performance operating, such as by the application program being provided by equipment or by by equipment The operation of the application program execution providing.

As described herein, in certain embodiments, user input word is grand or gesture is so that contextual keyboard executes and common The operation that text is associated, such as by the text input of contextual keyboard, or the text presenting to user by application program.Example As, user may want to for previously passed contextual keyboard be input to equipment text a part or as by equipment should The page that presented with program or the text of a part for document a part of executing operation.This part text can be used as upper and lower " seed " of civilian keyboard, contextual keyboard can be determined what operation of execution using seed, grand or connect to respond the word that receives The gesture receiving.

Many is used by the functions that contextual keyboard executes or relies on seed.Seed can be implicitly or explicitly.Explicitly Seed is probably the visible text wholly or largely on screen, or the highlighted visible text on screen or screen Otherwise selected visible text part on curtain.Implicit expression seed be probably any in current context available its The Email Information of the information of his type, such as author, the telephone number in message registration, geographical position etc..

In some cases, user explicitly indicates that a part for text can be construed to seed.For example, user can highlight A part for text, instruction text wants contextual keyboard to use as seed.Other are explicitly indicated and may include in a part of text Place or its neighbouring execution gesture, including the textual portions in word is grand, etc..

In some cases, determine whether a part for text or other information are used as seed in contextual keyboard.This Kind determine be probably implicit expression determine because contextual keyboard based on the possible some information of context cues or context as kind Child-operation.For example, a part of of text can include address, and contextual keyboard can determine that address is to take into account when that determination is held What operation of row, with the implicit expression seed of the gesture in response to receiving.Implicit expression seed potentially includes the text of people, place or things Instruction, the text calling the turn in foreign language, the text in the data base of user, the unknown text of user, etc..

As another example, contextual keyboard is contemplated that other information as seed, and this seed helps be determined holding The operation of row is in response to the order receiving.For example, contextual keyboard is contemplated that contextual information, such as identification computing device position Information, the information being associated with the application program being being currently used, the phase with the application content being presently using The information of association, the information that the operation with former execution is associated, the information being associated with the hobby of user, etc..

The flow chart that Fig. 5 is performed for the routine 500 of the operation related to seed.In step 510, contextual keyboard By the keyboard receives input of equipment.The input receiving can be that word is grand, gesture, or the form of other inputs.

In step 520, contextual keyboard determines whether received input is related to seed information.For example, operate The context component 330 of module 220 can be determined whether seed-bearing explicit representation (of the text for example, highlighting Point), the implicit representation (telephone number that the application program for example, being used on the computing device is comprised) of seed, or it Combination.

In step 530, the seed information based on the input receiving and correlation for the contextual keyboard, identification and/or determination One or more operation to be performed.For example, to determine that assembly 320 identifies two or more for the operation of operation module 220 Executable operation (for example, search information in contact database storage information) (for example, is searched for the gesture receiving The gesture of content) it is associated, and the behaviour Ji Yu seed information (telephone number for example, highlighting in an envelope Email) Elect one of operation (for example, storing information in contact database).

In step 540, contextual keyboard execution or start identified related to the input being received and seed information Operation.For example, the operating characteristics assembly 340 of the execution of operation module 220 executes or leads to execute and determines assembly 320 by operation The operation (telephone number for example, highlighting in contact data library storage) of identification.

Therefore, in certain embodiments, contextual keyboard is determined and will be executed based on the input receiving and seed information Operation, such as explicitly indicate that the information as seed information, be defined as the information of seed information with containing by hidden, etc..Therefore, exist In other factors, contextual keyboard is potentially based on explicitly or implicitly context and adapts to operation to be performed, this context and user How related using his/her computing device.

In some instances, when whether the input that determination receives is processed or the bar as text as order When mesh is processed, contextual keyboard can be utilized seed information explicitly or implicitly.Fig. 6 is a flow chart, and it illustrates to be used for Whether the word determining input is the routine of a part for order or text-string.

In step 610, contextual keyboard passes through the keyboard receives input of computing device.For example, contextual keyboard, passes through Input module 310 receives input of operation module 220, it is grand that input includes possible word, for example, " send ".

In step 620, the seed information that contextual keyboard reception is associated with the input receiving.For example, context Keyboard, identifies various possible seeds by the context component 330 of operation module 220, and for example other are by the word of user input, The position of described computing device, by the content of the currently viewing webpage of user, etc..

In act 630, contextual keyboard determines whether received input is to be associated with operation to be performed.That is, Whether contextual keyboard is that word is grand or plain text using seed information to determine the text of input.Contextual keyboard, by behaviour The operation making module 220 determines assembly 320, can determine that the text of input is to be associated with operation to be performed based on seed information , and routine 600 enters step 650, and otherwise routine 600 enters step 640.

In step 640, the keyboard of context, determine received input not grand to word related on the basis of, input The input being received is as plain text.For example, contextual keyboard determines that the input " transmission " receiving is not that a word is grand, because It is a part for text-string for it, for example, write " must not forget to send card to your mother " of calendar item by user, And input this character string to calendar.

In step 650, the keyboard of context, is determining that received input is actually grand with a word, execution The operation being associated with input.For example, " tonight, match was shown in!" text-string be transfused to after, contextual keyboard determine The input " transmission " receiving is received, and text-string is sent to the user of friend as message.

Therefore, in other factors, contextual keyboard makes user explicitly indicate the operation of seed or intention, or context Keyboard can automatically determine the meaning of user by determining the dependency between the word and the seed explicitly or implicitly of presence of input Figure, this explicitly or implicitly seed be used input word as order suitable target.

For example, the property according to order, contextual keyboard can determine that the current seed in processing environment.? In some cases, contextual keyboard can identify two or more seeds of the possible target as the input receiving, And present and/or show identified seed in order to determined by seed user selection.Contextual keyboard can be various Mode assumes list, such as from the most clearly to the menu of least clearly tactic possible seed, based on possible seed The menu of the operation that may take, etc..

Therefore, system can be grand based on the target seed pointed out by the input receiving, such as gesture or word, is provided with and is beneficial to The keyboard of the context of operating characteristics.The list of exemplary operation is presented herein below, this list can by contextual keyboard receive defeated Enter the target seed execution specified:

Execute the on-line search of intraseminal content, such as in the content search of search engine, social network sites, etc.;

It is created to the link of website, this website is related to the content of seed and inserts the link in message;

With the fills message of seed, such as email message or text message, breath of concurrently delivering letters;

Dial the telephone number being associated with the content of seed;

Using the related individual of payment application Zhi Fuyu seed or enterprise;

Obtain the map of the position being associated with the content of seed;

The content of storage seed, such as name and telephone number, in the data base of computing device;

The information being associated with the content of seed is retrieved from the data base of computing device, etc..

Certainly, those of ordinary skill in the art will be recognized that system described herein can execute other and operate and use Other instructions are as seed.

Word is grand

As described herein, in certain embodiments, contextual keyboard is by the use of the grand order as execution operation of word.Fig. 7 is One flow chart, illustrates based on the routine 700 executing operation by the keyboard applications program text being transfused to.In step 710 In, contextual keyboard receives text-string by the keyboard of computing device.Text-string can be that word is grand, one or more Word or phrase that the operation executing with by contextual keyboard and/or starting is associated.

In a step 715, contextual keyboard alternatively determines that the text-string receiving is associated with seed information.Example As, as described herein, contextual keyboard can determine that selected text a part or include seed content instruction text A part.

In step 720, contextual keyboard identification and/or the one or more text words to be executed and being received of determination Symbol string correlation (and, alternatively, related to the seed information being received) operation.Connect when determining that operation to be performed responds During the text-string receiving, contextual keyboard can check form, list or other, seeds grand with word, and/or executable behaviour Make related data structure.The example that following table (table 1) illustrates the data structure being utilized by contextual keyboard is assorted to determine execution Operation:

Word is grand The operation of execution? Seed? Send Assume the menu of messages option No Search Content searched for by search engine It is Preserve Information is preserved in data base It is Search Search within a mapping application It is Flat (user) Send text message to employee It is Lucy (user) Send Email to daughter It is

Table 1 lists different entries, and each entry stores, operation to be performed grand with word and determines that whether input text is Whether the grand seed of word is necessary related information.For example, " send " and be considered by contextual keyboard text and " assume message The word that the operation of the menu of option " is related is grand, regardless of whether there are seed.Another example is, the text input of " putting down " The word that related to the operation of " sending text message to employee " user in the presence of seed creates is considered by contextual keyboard Grand.Certainly, other words that this table can include not being specifically shown in table 1 are grand or operation.

In step 740, the identified operation of contextual keyboard execution.For example, when name " plug Pu Bulate " is dashed forward When going out to show, in response to receiving the text-string of " search ", contextual keyboard determines using table 1 text-string is and execution The word that the operation of search is related is grand, and executes the search of the information with regard to " plug Pu Bulate " in search engine.

Therefore, in certain embodiments, contextual keyboard determines that the plain text being received by keyboard can be carried out operating Order, the text that such as computing device shows be associated operation.Contextual keyboard is contemplated that such text is that word is grand, and Carry out corresponding operating.

Gesture

As described herein, in certain embodiments, contextual keyboard is by the use of gesture as command-execution operation.Fig. 8 is one Individual flow chart, illustrates the routine 800 of the operation related for the gesture that executes to receive by keyboard applications program.

In step 810, contextual keyboard receives gesture by the keyboard of computing device.Gesture can be certain on keyboard A little motions, such as the motion of the home key from keyboard to the second key, through the path of some keys on keyboard, shape, numeral, handss Write characters, etc..

In step 815, contextual keyboard alternatively determines that the gesture receiving is associated with seed information.For example, such as Described herein, contextual keyboard can determine that the part selecting text, or the part including the text of content instruction of seed.

In step 820, contextual keyboard identification and/or determine one or more to be executed with the gesture receiving The operation that (and, be alternatively associated with the seed information being received) is associated.Connect when determining that operation to be performed responds During the gesture receiving, contextual keyboard can check form, list or other, gesture, seeds grand with word, and/or executable behaviour Make related data structure.The example that following table (table 2) illustrates the data structure being utilized by contextual keyboard is assorted to determine execution Operation:

Gesture? The operation of execution? Seed? Frame Assume messages option menu No Underscore It is inserted into the link of content It is Question mark Search content It is The start key of T-W key Transmission pushes away special (Tweet, Twitter) It is The start key of I key Insertion image No Hand-written " C " It is added to contacts list It is

Table 2 includes various entries, and each entry storage is relevant to gesture, operation to be performed, and determines that gesture is No it is whether the seed of order is necessary information.For example, the gesture of underscore by contextual keyboard be considered with seed in The gesture of the operation correlation of " being inserted into the link of content " of content (content for example highlighting).As another example, The gesture of the tracking to " I " key for the start key from keyboard is considered to be associated with the operation of " insertion image " by contextual keyboard , in spite of there is seed.Certainly, other words that this table can include not being specifically shown in table 2 are grand or operation.

In step 840, the identified operation of contextual keyboard execution.For example, in response to receiving in an envelope Email In name on hand-written " C " gesture, using table 2, contextual keyboard determines that gesture related is added content to operation Contacts list, and the Email from the contacts list of user adds name and other relevant informations.

Therefore, in certain embodiments, contextual keyboard can perform operation in response to the gesture of keyboard reception so that using Textual entry can seamlessly be moved to other application programs from keyboard in family, such as messaging application, and other benefits.

In some instances, in other factors, contextual keyboard can be utilized and/or adopts gesture function to provide information To destination application, such as provide to another application program in relevant user and/or user's navigation from an application program The metadata held or other information.Fig. 9 is a flow chart, illustrates to be used for sending information to destination application to respond by key The routine 900 of the gesture that disk application program receives.

In step 910, contextual keyboard receives gesture by the keyboard of computing device.As described herein, gesture is permissible It is some motions on keyboard, such as the motion of the start key from keyboard to the second key, through the road of some keys on keyboard Footpath, shape, numeral, hand-written character, etc..

In step 920, the keyboard of context can recognize that the application program being associated with the gesture receiving.For example, on By the operation of operation module 220, hereafter keyboard, determines that assembly 320 can determine that received gesture and user want that browse answers It is associated with program.

In step 930, contextual keyboard determines that information is transferred to identified application program.For example, operation module 220 context component 330 can determine that with the currently used information being associated of application program by be transferred to identified with The application program that this gesture is associated.

In step 940, information determined by contextual keyboard startup transfers to identified application program.For example, grasp Make the application program that the operating characteristics assembly 340 of module 220 is extremely identified in transinformation (for example with regard to the metadata of user) When, guiding user is to the application program being identified.Such information transfer can use new application program to lift Consumer's Experience, because New application program receives the information of identification context by gesture, and user reaches application program in this context, and produces Other benefits.

As an example, user writes the mail with regard to Los Angeles football club one milky way to friend.User wishes Link is sent in Email.User highlights word " milky way " and makes the question mark gesture related to search information.Context Keyboard passes through gesture by search word " milky way " together with other information (the such as identification " football " in e-mail body and " admission ticket " Information and represent user be football player information) transfer to search engine.There is the search of search word and additional information Engine can carry out more targeted search for and send the result about football club's milky way, and " milky way " will not be sent Other iteration result, the such as related result of astronomy or the smart mobile phone being named as " milky way ".

Although there have been described herein various gesture, it will be appreciated by those of ordinary skill in the art that contextual keyboard can recognize that Almost all of motion on or near keyboard is as the gesture being potentially associated with operation to be performed.In some examples In, user can create the gesture being utilized by contextual keyboard 1.The gesture that the gesture that user creates provides before may replacing, Or one group of gesture being provided by computing device may be provided.

In some instances, gesture is specific for the current type of application program being used by user, device polling, use Family, etc..That is, one group of gesture may be based on application program in use, based on device type, based on user preferences etc. And change or be modified.For example, the application program (such as e-book application program) being mainly used in the text of reading can be provided that The extra gesture related to the comment on the various articles in the book that user is reading.

As another example, because smart mobile phone screen size is little, the application program of operation on smart mobile phone The limited one group gesture related to starting other application programs can be enabled, and tablet device is because screen size is big, The same application domain run thereon can enable and start a complete set of gesture that other application programs are associated.

Therefore, in other factors, increase the function of keyboard, such as by contextual keyboard, make user execute them The operation of the content in application program using, and application program need not be left or grasp between various different application programs Vertical, this is time-consuming and poorly efficient.Keyboard, the nowhere application program of not receives input, may be used as hinge, wherein user is in place Execute operation on the multiple application programs on equipment, provide powerful and effective mechanism to these users.By these mechanism, User can work, plays, link up, or otherwise using the function that the many of their equipment is different.

The setting of contextual keyboard and function

As described herein, contextual keyboard 150 executes many different operations, such as insertion link, sends message, updates Contacts list, etc..The experience beneficial in order to be supplied to user, contextual keyboard may include how regulation contextual keyboard is held The various settings of row various operations described herein.Setting can be stored in various and contextual keyboard or support context key The data base that the equipment of disk is associated, such as data base 152 or 350, various to be provided on how to execute to contextual keyboard The information of operation.For example, contextual keyboard can determine how insertion link using setting, how to assume image, etc..

In some instances, setting can be user-defined.That is, the character of text that user can define String, when on keyboard input when, lead to contextual keyboard come to execute the search operation being associated with selected text or other Operation.Exemplary setting, is user-defined in some cases, including:

Define how to provide link, for example, HTML assumes link, shorten service using URL, etc.;

Define how to fill customer data base, such as use telephone number, e-mail address etc.;

Definition using what search engine or is selecting to be provided what search engine in menu;

Define when to provide options menu, when two or more operations are all associated with the gesture receiving;

Define when to allow contextual keyboard to select suitable operation, when two or more operations all with receive When gesture is associated;Or

Define the specific gesture of user, the order that word is grand and other are based on input, etc..

Therefore, when determining whether gesture is associated with operation to be performed, when selecting suitably to operate to execute, And/or in the input that execution operation response receives, contextual keyboard is using setting.

The process being executed by contextual keyboard

As discussed in this article, contextual keyboard can be with user friendly various tasks, and such as search content is published to society Friendship website, storing contact information, etc..The exemplary use example of the following embodiment of scene description contextual keyboard.

Scene #1- searches for information

In some instances, contextual keyboard allows users to using gesture search for and/or obtain read with them Read or input related information content.Figure 10 A-10D system screenshot capture, illustrates when starting information search using context key The step being performed of disk.Figure 10 A shows the screenshot capture 1000 of the email message being sent to user.This Email Message includes the text 1005 in source body.When reading message, user wants to know about more east about " bulgoki " West.Figure 10 B display screen sectional drawing 1010, wherein " bulgoki " are highlighted explicit 1012.Once highlighted, user passes through Draw the gesture that arrow inputs arrow 1014 on the touchscreen.Contextual keyboard is although the not screenshot capture 1020 in Figure 10 C Shown in, menu 1025 on the screen of startupoptions, wherein user may want to search for more information with regard to " bulgoki ". Once option is selected, such as " Google " option, contextual keyboard starts (upper with regard to " bulgoki " on Google website Hereafter keyboard is considered a seed) information search, as shown in the screenshot capture 1030 described by Figure 10 D.Screenshot capture The 1030 various Search Results 1035 of display.Therefore, in certain embodiments, contextual keyboard allows users to execute gesture, with When read emails or alternative document understanding more contents about Email, and other benefits.

In some instances, contextual keyboard using word grand allow users to search for and/or retrieve related they read The information of the content read or input.Figure 11 A-11C system screenshot capture, illustrates when starting information search using contextual keyboard The step being performed.Figure 11 A shows the screenshot capture 1100 of the email message being sent to user.This email message Including the text 1105 in mail body.When reading message part, user wants to know about more things about " bulgoki ". Figure 11 B display screen sectional drawing 1110, wherein " bulgoki " are highlighted explicit 1112.Once highlighted, user is in display Input text-string 1114 " Google " on keyboard 1116.Contextual keyboard determines that input text is that word is grand and in Google net Start the search with regard to " bulgoki " information, as shown in the screenshot capture 1120 described by Figure 11 C, search shoots institute on standing Depiction.Screenshot capture 1120 shows various Search Results 1125.Therefore, in certain embodiments, contextual keyboard makes to use Family can provide word grand on the keyboard of display, simultaneously the alternative document of read emails or search information, and other good Place.

Certainly, when being prompted search information, contextual keyboard can provide various different options.Although in the example shown Googe is illustrated it is also possible to using other search engines, such as other universal search engines (for example, Bing, Yahoo, etc.), Consumer reviews website (as Yelp), online retailer (such as amazon.com), both content aggregators, blog, social network sites, etc. Deng.

Contextual keyboard can be such as intraseminal interior based on one or more of many factors determination and/or the Recommended option Hold type (for example, when content instruction food type, such as epicurious.com is selected for food blog), the going through of relevant user History information (for example, user always selects google.com), and positional information (for example, when user is in New York, nyimes.com quilt Choose), application environment (for example, when user is on facebook.com, search carried out in Facebook) and/or Other seeds or non-seed information.In some cases, the keyboard of context may provide user definable option.Example As the menu 1025 of Figure 10 C includes option " adding new ", and that is, user can input required website, scans for wherein.

Scene #2- strengthens text

In some instances, contextual keyboard allows users to strengthen in they are reading or inputting using gesture Hold.Figure 12 A-12B system screenshot capture, illustrates the step being performed when insertion link or other information are in application program Suddenly.Figure 12 A depicts the screenshot capture 1200 of the text message being imported in the text message applications program being shown by equipment. Text message includes the content 1202 being transfused to and highlighted content 1204, and contextual keyboard can use " Tamarine " As seed.The gesture 1206 of drawn arrow is received on keyboard as input.As described by the screenshot capture 1210 of Figure 12 B , contextual keyboard, in response to the gesture 1206 receiving and the seed content highlighting, determine gesture 1206 and operation phase Associate to retrieve the link of the information about content, dining room " Tamarine ", and insert the content link 1214 retrieving and arrive In source body 1212.Therefore, in certain embodiments, contextual keyboard uses gesture so that user links and other guide Strengthen previously written or presently written content, and produce other benefits.

In some instances, in before contextual keyboard allows users to using word is grand, they are reading or are inputting Hold.Figure 13 A-13B is screenshot capture, and the step being performed when insertion link or other information are in application program is described. Figure 13 A depicts the screenshot capture 1300 of the text message being imported in the text message applications program being shown by equipment.Literary composition This information includes the content 1302 being transfused to and highlighted content 1304, and contextual keyboard can use " Tamarine " to make For seed." linkyelp " word grand 1306 is received on keyboard as input.As described by the screenshot capture 1310 of Figure 13 B , contextual keyboard, the word in response to receiving grand 1306 and the seed content highlighting, determine word grand 1306 and operation phase Association retrieving the link of the information about content, dining room " Tamarine " from website " yelp.com ", and insert and retrieve Content link 1314 in source body 1312.Therefore, in certain embodiments, contextual keyboard uses word grand so that user Strengthen previously written or presently written content with link and other guide, and other benefits.

Certainly, it will be appreciated by the skilled addressee that contextual keyboard is advantageously possible in information or text and/or interior The insertion of other various contents in the other parts held or replacement.For example, link unless the context, contextual keyboard can be square Just image, video file, audio file, document, the insertion of extra content of text etc..For example, when text phrases are " public When table " is highlighted, user input word grand " definition " may result in contextual keyboard and enters determining of common block list in message interpolation Justice.Another example is, the gesture that user inputs circle afterwards in input " resume " may result in contextual keyboard with being transfused to Message further user resume pdf document.

When user inputs the text of message, contextual keyboard in retrieval information and may be linked to retrieval information Middle assistance user.Figure 14 A-14D is screenshot capture, illustrate when insert map or other information in application program when be performed Step.Figure 14 A depicts the screenshot capture 1400 that user input text 1402 arrives message application.Contextual keyboard expected and Highlight 1404 words grand " Startbuck ", determine that user will be in message based on some factors (for example, Sentence Grammar, user's history) The middle information added about Startbuck position.As shown in the screenshot capture 1410 of Figure 14 B, user execution draw from gesture key to The gesture 1415 in the path of " m " key.Response gesture 1415, contextual keyboard determines that user is look for checking in user location The map of the Startbuck position in domain, as shown in the screenshot capture 1420 of Figure 14 C.That is, contextual keyboard uses " Startbuck " to plant Son, the gesture being received and the location context of user equipment are being asked near Starbucks coffee shop with determining user Map, and assume map 1425 to user.Once user select a position, contextual keyboard address information and to select position The link new information 1432 of the map 1434 put, as shown in the screenshot capture 1430 in Figure 14 D.

Therefore, when, gesture grand in response to word, or other inputs as command process, when determining operation to be performed, on Hereafter keyboard can use explicit seed (for example, being designated as the word of the seed by the input receiving from user, be such as highlighted area) And/or the seed (for example, based on the word being designated as seed by the determination of contextual keyboard) of implicit expression.

Scene #3- storage information

In some instances, contextual keyboard allows users to storage content using using gesture and they are reading or defeated The other information entering, the content such as receiving in the message, the content read on webpage, etc..Figure 15 A-15B is that screen cuts Figure, illustrates the step being performed when storage information in data base.In Figure 15 A, screenshot capture 1500 assumes an envelope electronics Mail, including the associated person information 1502 of " Mike Jones ".User wishes storage in his/her contact database Information, and input the gesture 1504 of hand-written " C " on keyboard.Contextual keyboard, as shown in the screenshot capture 1510 of Figure 15 B, Determine received gesture 1504 and be associated with operating, using storage information and insert associated person information 1502 as new contact person 1515 in the contacts list of user.Therefore, in certain embodiments, contextual keyboard allows a user to store using gesture From message, document, and present to the information of other texts on his/her equipment for the user, and produce other benefits.

In some instances, contextual keyboard allows users to storage content and they are reading or input using word is grand Other information, the content such as receiving in the message, the content read on webpage, etc..Figure 16 A-16B is screenshot capture, The step being performed when storage information in data base is described.In Figure 16 A, screenshot capture 1600 assumes an envelope electronics postal Part, including the associated person information 1602 of " Mike Jones ".User wishes storage letter in his/her contact database Breath, and input word grand 1604 by following the tracks of " contact person " word on keyboard.Contextual keyboard, the such as screenshot capture of Figure 16 B Shown in 1610, determine received word grand 1604 and be associated with operating, using storage information and insert associated person information 1602 as New contact person 1615 is in the contacts list of user.Therefore, in certain embodiments, contextual keyboard use word grand so that User can store from message, document, and presents to the information of other texts on his/her equipment for the user, and other Benefit.

Except associated person information, contextual keyboard can be easy to other classes of storage in the various data bases in user equipment The information of type.For example, user can select the text of the event presenting by webpage to describe, and order contextual keyboard, passes through Input gesture or word are grand, are that event creates entry in the calendar of user.Based on receiving instruction input, contextual keyboard executes behaviour Make to create calendar using the event information filled in the entry for user.

In certain embodiments, the information that contextual keyboard can be utilized and/or retrieval has stored in device databases, To respond the order receiving by keyboard.For example, user can key in Email and wish that insertion is made a copy for electronics postal The associated person information of the lineup of part.This point to be accomplished, user can highlight the name of lineup and input " interpolation connection System " word is grand, and this can lead to contextual keyboard retrieval from the associated person information of lineup of user contact lists and to insert connection It is people's information to Email.

In certain embodiments, contextual keyboard can be utilized and/or retrieval is by device measuring, the letter of detection and/or capture Breath is to respond the order receiving by keyboard.These information potentially include geographical location information (for example, longitude and latitude information), Relative position information (for example, user is close to certain terrestrial reference or street), temperature and other Weather information, elevation information, speed Degree information, temporal information, image or other by equipment photographic head capture data, etc..For example, carry out globaetrotting use Family can send an email to his/her father and mother by intransit different time.When writing Email using contextual keyboard, User can execute and lead to contextual keyboard to be retrieved from equipment and insert the gesture (example of some information at the end of Email As check mark is drawn on the keyboard of equipment), for example:" passenger is in the Bratislava of Slovakia.Outdoor is 34 degree, Light rain." therefore, contextual keyboard can facilitate the interpolation of all kinds information user need not be led to search or measure such Information, saves the same enhancing user profile of the time and efforts of user, and produces other benefits.

Scene #4- releases news

In some instances, contextual keyboard allow users to issue using gesture or word are grand in perhaps other information to each Plant position, such as webpage, blog, social network sites, etc..Figure 17 A-17B is screenshot capture, illustrate when release news to website or its The step being performed during his position.Figure 17 A depicts the screenshot capture 1700 of notepad or other text editor application, User input text 1702 wherein, the brief comment user of for example up-to-date Harry Potter's film wishes to issue this message to various Different social networking website and blog, and the gesture of do so is executed on keyboard.Figure 17 A shows from start key to key The first gesture 1704 of the track path of " FB ".Alternatively, user is traceable or input " facebook " word is grand similar to input Order to contextual keyboard.Contextual keyboard determines that gesture 1704 is associated with operation with the Facebook of issue text to user The page, and, as shown in the screenshot capture 1710 of Figure 17 B, update or start shape on his/her Facebook page for the user The renewal of state 1715.

Next, as shown in the screenshot capture 1800 of Figure 18 A, user wishes that issuing identical message 1805 arrives him/her Twitter subscribe to, and again on keyboard execute gesture, i.e. the second gesture of the track path of " TW " from start key to key 1804.Or, user may follow the tracks of or input the grand order similar with input of " tw " word to contextual keyboard.Contextual keyboard is true Determine gesture 1804 to be associated with operation to issue text to the Twitter subscription of user, and, the such as screenshot capture of Figure 18 B Shown in 1810, update or start the renewal of subscription 1815 on his/her Twitter page for the user.

Therefore, in certain embodiments, contextual keyboard allows users to photos and sending messages to each website using word is grand with gesture And other positions, such as it is distributed to position (for example, Facebook webpage, Twitter account, the Google+ in user social contact network Page, LinkedIn configuration file etc.), and be each different position or site promoter application program without user, and produce Other benefits raw.

Conclusion

Unless the context clearly require that, otherwise in entire disclosure and claims, "comprising", " inclusion " etc. The meaning comprising can be interpreted, rather than exclusive or detailed meaning is that is to say, that the meaning of " including, but are not limited to ".As Used herein, term " connects ", " coupling " or its any deformation, refers to any connection or couples, either directly or indirectly, Between two or more elements, the coupling of connection between elements can be physics, logic, or combinations thereof.This Outward, " here " using in this application, " more than ", the word of " below " and similar meaning, refer to the application as entirety It is not any specific part of the application.If content is permitted, illustrate that the word of the odd number or plural number using can divide in detail above Do not include odd number or plural number.In the list of two or more projects, "or" covers all explained below of word:In list Any project, all items in list, the combination in any of project in list.

It is accurate shape disclosed above that the detailed description of the embodiment of the system above is not intended to exhaustion or restriction system Formula.Those skilled in the relevant art will be recognized that the specific embodiment of system as above and example are intended to illustration purpose, Various equivalent modifications in the range of system are possible.For example, although flow process or block are presented with given order, substitute real Apply example and can be executed in different order the routine with step or using the system with block, and some flow processs or block permissible It is deleted, mobile, increase, segmentation, combination, and/or modification.Each of these flow processs or block can be by with multiple different sides Formula is realized.Although additionally, flow process or block execute sometimes in a series arrangement, these flow processs or block may execute in a parallel fashion or Can be performed in the different time.

Although in many above-described embodiments using storage software on the mobile apparatus (before being given to a subscriber or with During the calling of family), above-mentioned script and other software can be hard-coded into mobile device and (for example, be stored in EEPROM, PROM etc. In).In addition, above-mentioned functions can be implemented without script or other special functional modules.

The teaching of system may apply to other system provided herein, is not necessarily above-mentioned system.Above-mentioned various enforcement The element of example and behavior can be combined to provide further embodiment.

All above-mentioned patents and application and other lists of references, including may be listed in present a paper in any file, It is cited in way of reference.The each side of system can be modified, if it is necessary, adopting system, function, and above-mentioned various The concept of list of references is to provide the embodiment of further system.

These and other change can be made for system according to above-mentioned detailed description.Although foregoing description describes in detail being Some embodiments of system simultaneously describe the optimal mode envisioned, no matter but how to display the details of in the text, system can be with many The mode of kind is realized.May be very different in its implementation detail based on the details of local support system, simultaneously still by here Disclosed system is comprised.As described above, when some features or the aspect of descriptive system, the particular term being used should not Be considered imply term by here redefine be restricted to any specific characteristic, feature or term be associated be The each side of system.Generally, the term used in claims below is not construed as limiting system It is made as the specific embodiment disclosed in this specification, unless above-mentioned detailed description part explicitly defines such term.Therefore, it is The actual range of system not only includes the disclosed embodiments, but is effected or carried out all equivalent of this system according to claim Mode.

Although some aspects of system are presented as follows in the form of some claim, inventor is with any number of power The form that profit requires envisions the many aspects of system.For example although the only one aspect of system be listed be embodied in computer can Read in medium, other aspects may similarly be embodied in computer-readable medium.Therefore, inventor has the right submitting application to The right retaining the extra claim of interpolation afterwards thinks that other aspects of system pursue the form of such additional claim.

Claims (15)

1. a kind of keyboard system of mobile device, including:
Keyboard components, the touch screen being configured to described mobile device assumes dummy keyboard to user;
Gesture receiving unit, is configured to described keyboard components and receives stroke gesture input;
Context component is configured to identify the contextual information describing context, and described stroke gesture input is described upper and lower Received in literary composition;
Interpretation component, is configured to the contextual information based on the stroke gesture input receiving and identification, without extra User input, determines that the stroke gesture input of described reception is the order for described keyboard system, or with input text word The related text input of symbol string,
Wherein, described keyboard system is different from the other application of described mobile device;
Text input assembly, be configured to described dummy keyboard receive be determined as text input stroke gesture defeated Enter, and the text input based on described reception updates text-string;And
Operating assembly, is configured to receive the stroke gesture input of the order being determined as described keyboard system;
Before the stroke gesture input of described reception is sent to another active application, intercept the stroke gesture of described reception Input;And
Execute the operation that described orders that are one or more and being received are associated by described dummy keyboard,
Wherein said operation is not the function of the key of described keyboard.
2. keyboard system as claimed in claim 1, wherein said operating assembly is configured to execute one or more operations, with The described order of response and the seed text shown by the described touch screen of described mobile device.
3. keyboard system as claimed in claim 2, the stroke gesture input of wherein said reception is and is passed through described keyboard On some keys the different shape gesture in path.
4. keyboard system as claimed in claim 2, wherein said operating assembly is configured to call application, and to being called Application provides the seed text of detection.
5. keyboard system as claimed in claim 1, wherein said operating assembly is configured to based on by described mobile device A part for highlighted text shown by described touch screen, executes one or more operations.
6. keyboard system as claimed in claim 1, wherein said text input assembly is configured as described mobile device User tap described dummy keyboard key, or the key being tracked through described dummy keyboard path when, by described dummy keyboard Receive text input.
7. a kind of method by computing device, it provides contextual keyboard, including:
Stroke gesture input is received by the keyboard being shown by described computing device, the keyboard of wherein said display is configured to connect Receive text input;
Identification describes the contextual information of context, and described stroke gesture input is received in described context;
Based on the contextual information of the stroke gesture input receiving and identification, without extra user input, determine described The stroke gesture input receiving whether include with by described contextual keyboard pending operate related order,
Wherein, described contextual keyboard is different from the other application of mobile device;And
Wherein said operation is not the function of the key of described keyboard;
When the stroke gesture input of described reception is confirmed as not including the order for described keyboard applications, then:
The stroke gesture input receiving described reception is as the input being used to help text input;And
When the stroke gesture input of described reception is confirmed as including the order for described keyboard applications, then:
Before the stroke gesture input of described reception is sent to another active application, intercept the stroke gesture of described reception Input, thus described stroke gesture input is not received as text input;And
The seed object of the display that detection operation can be carried out thereon;And
Based on a determination that order and detection seed object, the related operation of execution.
8. method as claimed in claim 7, the stroke gesture input wherein being received by the keyboard of described display is and wears Cross some keys on said keyboard the different word in path is grand or shape gesture.
9. method as claimed in claim 7, wherein based on the order of described determination and the seed object of described detection, executes institute The operation stating correlation includes, and stores the information being shown by described computing device in the data base of described computing device.
10. method as claimed in claim 7, wherein based on the order of described determination and the seed object of described detection, execution Described related operation includes, and issues the information related to the text input of described reception to webpage.
11. methods as claimed in claim 7, wherein based on the order of described determination and the seed object of described detection, execution Described related operation includes, and links to content based on the information insertion related to the seed object of described detection.
12. methods as claimed in claim 7, wherein based on the order of described determination and the seed object of described detection, execution Described related operation includes searching for the information related to the seed object of described detection.
A kind of 13. methods of the keyboard applications program performing by mobile device, methods described includes:
Stroke gesture input is received by the dummy keyboard of described keyboard applications, wherein said dummy keyboard is configured to receive literary composition This input;
Identification describes the contextual information of context, and described stroke gesture input is received in described context;
Based on the contextual information of the stroke gesture input receiving and identification, without extra user input, determine described The stroke gesture input receiving whether include with by described keyboard applications pending operate related order,
Wherein, described keyboard applications are different from the other application of described mobile device;And
Wherein said operation is not the function of the key of described keyboard;
When the stroke gesture input of described reception is confirmed as not including the order for described keyboard applications, then:
The stroke gesture input receiving described reception is as the input being used to help text input;And
When the stroke gesture input of described reception is confirmed as including the order for described keyboard applications, then:
Before the stroke gesture input of described reception is sent to another active application, intercept the stroke gesture of described reception Input, thus described stroke gesture input is not received as text input;And
Based on a determination that order execute described associated operation.
14. methods as claimed in claim 13, the wherein order based on described determination execute described associated operation and include: The operation treated performed by the single application program running is started on described mobile device.
15. methods as claimed in claim 13, the input of wherein said reception is text-string or gesture, and wherein true The input of fixed described reception be associated including the described text-string of determination with by the operation of described keyboard applications program performing or Gesture is associated with one or more operations performed by described keyboard applications program.
CN201180065367.6A 2010-11-20 2011-11-18 Performing actions on a computing device using a contextual keyboard CN103649876B (en)

Priority Applications (3)

Application Number Priority Date Filing Date Title
US41582810P true 2010-11-20 2010-11-20
US61/415,828 2010-11-20
PCT/US2011/061530 WO2012068544A2 (en) 2010-11-20 2011-11-18 Performing actions on a computing device using a contextual keyboard

Publications (2)

Publication Number Publication Date
CN103649876A CN103649876A (en) 2014-03-19
CN103649876B true CN103649876B (en) 2017-02-15

Family

ID=46063894

Family Applications (3)

Application Number Title Priority Date Filing Date
CN201180065379.9A CN103827779B (en) 2010-11-20 2011-11-18 The system and method for accessing and processing contextual information using the text of input
CN201180065367.6A CN103649876B (en) 2010-11-20 2011-11-18 Performing actions on a computing device using a contextual keyboard
CN201180065371.2A CN103314343B (en) 2010-11-20 2011-11-21 Using gestures to command a keyboard application, such as a keyboard application of a mobile device

Family Applications Before (1)

Application Number Title Priority Date Filing Date
CN201180065379.9A CN103827779B (en) 2010-11-20 2011-11-18 The system and method for accessing and processing contextual information using the text of input

Family Applications After (1)

Application Number Title Priority Date Filing Date
CN201180065371.2A CN103314343B (en) 2010-11-20 2011-11-21 Using gestures to command a keyboard application, such as a keyboard application of a mobile device

Country Status (5)

Country Link
US (3) US9244610B2 (en)
EP (3) EP2641145A4 (en)
KR (3) KR20130143697A (en)
CN (3) CN103827779B (en)
WO (3) WO2012068544A2 (en)

Families Citing this family (76)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103827779B (en) 2010-11-20 2017-06-20 纽昂斯通信有限公司 The system and method for accessing and processing contextual information using the text of input
KR20130052745A (en) * 2010-12-23 2013-05-23 한국전자통신연구원 Method of providing menu using gesture and mobile terminal using the same
US8572481B2 (en) * 2011-03-14 2013-10-29 Apple Inc. Device, method, and graphical user interface for displaying additional snippet content
US9094813B2 (en) * 2011-04-02 2015-07-28 Open Invention Network, Llc System and method for redirecting content based on gestures
KR101932270B1 (en) * 2012-01-04 2018-12-24 엘지전자 주식회사 Mobile terminal and control method therof
US8902198B1 (en) * 2012-01-27 2014-12-02 Amazon Technologies, Inc. Feature tracking for device input
WO2013130682A1 (en) * 2012-02-27 2013-09-06 5 Examples, Inc. Date entry system controllers for receiving user input line traces relative to user interfaces to determine ordered actions, and related systems and methods
US20130239030A1 (en) * 2012-03-06 2013-09-12 Apple Inc. Application for publishing journals
US8667414B2 (en) * 2012-03-23 2014-03-04 Google Inc. Gestural input at a virtual keyboard
KR20140014510A (en) * 2012-07-24 2014-02-06 삼성전자주식회사 Editing method of text generatied by a speech recognition and terminal thereof
US9495069B2 (en) * 2012-08-31 2016-11-15 Paypal, Inc. Expanded icon functionality
US9021380B2 (en) 2012-10-05 2015-04-28 Google Inc. Incremental multi-touch gesture recognition
US8782549B2 (en) 2012-10-05 2014-07-15 Google Inc. Incremental feature-based gesture-keyboard decoding
US8850350B2 (en) 2012-10-16 2014-09-30 Google Inc. Partial gesture text entry
US8843845B2 (en) 2012-10-16 2014-09-23 Google Inc. Multi-gesture text input prediction
US8701032B1 (en) 2012-10-16 2014-04-15 Google Inc. Incremental multi-word recognition
US9557818B2 (en) * 2012-10-16 2017-01-31 Google Inc. Contextually-specific automatic separators
US8977961B2 (en) * 2012-10-16 2015-03-10 Cellco Partnership Gesture based context-sensitive functionality
US8819574B2 (en) 2012-10-22 2014-08-26 Google Inc. Space prediction for text input
CN103092511B (en) * 2012-12-28 2016-04-20 北京百度网讯科技有限公司 Input method of a mobile terminal, the mobile terminal apparatus and
US8832589B2 (en) 2013-01-15 2014-09-09 Google Inc. Touch keyboard using language and spatial models
US20140208274A1 (en) * 2013-01-18 2014-07-24 Microsoft Corporation Controlling a computing-based device using hand gestures
US20140237425A1 (en) * 2013-02-21 2014-08-21 Yahoo! Inc. System and method of using context in selecting a response to user device interaction
US9799005B2 (en) * 2013-03-07 2017-10-24 Samsung Electronics Co., Ltd. Computing system with contextual interaction mechanism and method of operation thereof
US9477645B2 (en) * 2013-03-13 2016-10-25 Sap Se Systems and methods of active text markup
US8887103B1 (en) 2013-04-22 2014-11-11 Google Inc. Dynamically-positioned character string suggestions for gesture typing
US9081500B2 (en) 2013-05-03 2015-07-14 Google Inc. Alternative hypothesis error correction for gesture typing
US20140365878A1 (en) * 2013-06-10 2014-12-11 Microsoft Corporation Shape writing ink trace prediction
CN103390060A (en) * 2013-07-30 2013-11-13 百度在线网络技术(北京)有限公司 Song recommending method and device based on mobile terminal
US9277354B2 (en) 2013-10-30 2016-03-01 Sprint Communications Company L.P. Systems, methods, and software for receiving commands within a mobile communications application
US9383816B2 (en) * 2013-11-15 2016-07-05 Kopin Corporation Text selection using HMD head-tracker and voice-command
US9500867B2 (en) 2013-11-15 2016-11-22 Kopin Corporation Head-tracking based selection technique for head mounted displays (HMD)
US9904360B2 (en) 2013-11-15 2018-02-27 Kopin Corporation Head tracking based gesture control techniques for head mounted displays
US10209955B2 (en) 2013-11-15 2019-02-19 Kopin Corporation Automatic speech recognition (ASR) feedback for head mounted displays (HMD)
KR20150058965A (en) * 2013-11-21 2015-05-29 엘지전자 주식회사 Mobile terminal and controlling method thereof
US20150153949A1 (en) * 2013-12-03 2015-06-04 Google Inc. Task selections associated with text inputs
US20150242087A1 (en) * 2014-02-21 2015-08-27 Nod, Inc. Contextual Keyboard Located on a Remote Server for Implementation on Any Content Delivery and Interaction Application
US9330666B2 (en) * 2014-03-21 2016-05-03 Google Technology Holdings LLC Gesture-based messaging method, system, and device
CN104200170B (en) * 2014-04-15 2018-06-19 中兴通讯股份有限公司 The method for secret protection and electronic equipment of a kind of electronic equipment
US9575560B2 (en) 2014-06-03 2017-02-21 Google Inc. Radar-based gesture-recognition through a wearable device
US9811164B2 (en) 2014-08-07 2017-11-07 Google Inc. Radar-based gesture sensing and data transmission
US9921660B2 (en) 2014-08-07 2018-03-20 Google Llc Radar-based gesture recognition
US9588625B2 (en) 2014-08-15 2017-03-07 Google Inc. Interactive textiles
US10268321B2 (en) 2014-08-15 2019-04-23 Google Llc Interactive textiles within hard objects
US9778749B2 (en) 2014-08-22 2017-10-03 Google Inc. Occluded gesture recognition
US9940016B2 (en) 2014-09-13 2018-04-10 Microsoft Technology Licensing, Llc Disambiguation of keyboard input
US20160124636A1 (en) * 2014-10-29 2016-05-05 Sunrise Atelier, Inc. Keyboard utility for inputting data into a mobile application
CN104363322B (en) * 2014-11-20 2017-10-17 广东欧珀移动通信有限公司 Dialing Method and device
US10459608B2 (en) * 2014-12-01 2019-10-29 Ebay Inc. Mobile optimized shopping comparison
CN104571915B (en) * 2014-12-29 2018-11-27 科大讯飞股份有限公司 The control method and device of touch keyboard in touch screen terminal
US9983747B2 (en) 2015-03-26 2018-05-29 Google Llc Two-layer interactive textiles
CN104731554B (en) * 2015-03-30 2018-08-07 北京奇艺世纪科技有限公司 A kind of comment information dissemination method and device
WO2016176574A1 (en) 2015-04-30 2016-11-03 Google Inc. Wide-field radar-based gesture recognition
JP6517356B2 (en) 2015-04-30 2019-05-22 グーグル エルエルシー Type-independent RF signal representation
CN107430444A (en) 2015-04-30 2017-12-01 谷歌公司 For gesture tracking and the tracking of the micromotion based on RF of identification
US9693592B2 (en) 2015-05-27 2017-07-04 Google Inc. Attaching electronic components to interactive textiles
US10088908B1 (en) 2015-05-27 2018-10-02 Google Llc Gesture detection and interactions
CN104935812B (en) * 2015-05-29 2017-11-03 广东欧珀移动通信有限公司 A kind of method and device for controlling self-timer mode to open
CN104850770A (en) * 2015-06-05 2015-08-19 深圳市智诺微智能电子科技有限公司 Gesture password input virtual keyboard and application method
US10082931B2 (en) * 2015-06-12 2018-09-25 Microsoft Technology Licensing, Llc Transitioning command user interface between toolbar user interface and full menu user interface based on use context
US10300370B1 (en) 2015-10-06 2019-05-28 Google Llc Advanced gaming and virtual reality control using radar
WO2017079484A1 (en) 2015-11-04 2017-05-11 Google Inc. Connectors for connecting electronics embedded in garments to external devices
US10260830B2 (en) 2016-02-11 2019-04-16 John Hafen Smart-gun systems and methods
US10078673B2 (en) 2016-04-20 2018-09-18 Google Llc Determining graphical elements associated with text
US9965530B2 (en) 2016-04-20 2018-05-08 Google Llc Graphical keyboard with integrated search features
US10305828B2 (en) 2016-04-20 2019-05-28 Google Llc Search query predictions by a keyboard
US10222957B2 (en) 2016-04-20 2019-03-05 Google Llc Keyboard with a suggested search query region
US10140017B2 (en) 2016-04-20 2018-11-27 Google Llc Graphical keyboard application with integrated search
US10175781B2 (en) 2016-05-16 2019-01-08 Google Llc Interactive object with multiple electronics modules
US20170357696A1 (en) * 2016-06-10 2017-12-14 Apple Inc. System and method of generating a key list from multiple search domains
US20170357521A1 (en) * 2016-06-13 2017-12-14 Microsoft Technology Licensing, Llc Virtual keyboard with intent-based, dynamically generated task icons
US10409488B2 (en) 2016-06-13 2019-09-10 Microsoft Technology Licensing, Llc Intelligent virtual keyboards
US20180039406A1 (en) * 2016-08-03 2018-02-08 Google Inc. Image search query predictions by a keyboard
US10409487B2 (en) * 2016-08-23 2019-09-10 Microsoft Technology Licensing, Llc Application processing based on gesture input
KR101915789B1 (en) * 2016-10-18 2018-11-06 주식회사 한컴세이프티 Touch screen electronic device for supporting easy text input via the virtual keyboard and operating method thereof
US9924313B1 (en) 2017-02-23 2018-03-20 International Business Machines Corporation Location based generation of pertinent information

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1637686A (en) * 2004-01-06 2005-07-13 国际商业机器公司 System and method for improved user input on personal computing devices
CN101356493A (en) * 2006-09-06 2009-01-28 苹果公司 Portable electronic device for photo management
CN101546233A (en) * 2009-05-05 2009-09-30 上海华勤通讯技术有限公司 Identification and operation method of touch screen interface gestures

Family Cites Families (34)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US1027938A (en) 1909-11-02 1912-05-28 Westinghouse Air Brake Co Fluid-pressure brake.
US5502803A (en) * 1993-01-18 1996-03-26 Sharp Kabushiki Kaisha Information processing apparatus having a gesture editing function
JP3416268B2 (en) 1994-06-30 2003-06-16 キヤノン株式会社 Image recognition apparatus and method
US6999955B1 (en) * 1999-04-20 2006-02-14 Microsoft Corporation Systems and methods for estimating and integrating measures of human cognitive load into the behavior of computational applications and services
US7762453B2 (en) * 1999-05-25 2010-07-27 Silverbrook Research Pty Ltd Method of providing information via a printed substrate with every interaction
US20030023952A1 (en) 2001-02-14 2003-01-30 Harmon Charles Reid Multi-task recorder
US6792408B2 (en) * 2001-06-12 2004-09-14 Dell Products L.P. Interactive command recognition enhancement system and method
US8626823B2 (en) 2007-11-13 2014-01-07 Google Inc. Page ranking system employing user sharing data
US7251367B2 (en) * 2002-12-20 2007-07-31 International Business Machines Corporation System and method for recognizing word patterns based on a virtual keyboard layout
US7542029B2 (en) 2005-09-20 2009-06-02 Cliff Kushler System and method for a user interface for text editing and menu selection
US7098896B2 (en) * 2003-01-16 2006-08-29 Forword Input Inc. System and method for continuous stroke word-based text input
US7382358B2 (en) 2003-01-16 2008-06-03 Forword Input, Inc. System and method for continuous stroke word-based text input
US7453439B1 (en) 2003-01-16 2008-11-18 Forward Input Inc. System and method for continuous stroke word-based text input
US7480619B1 (en) 2003-03-04 2009-01-20 The Board Of Trustees Of The Leland Stanford Junior University Integration manager and natural interaction processor
US7401072B2 (en) * 2003-06-10 2008-07-15 Google Inc. Named URL entry
US7890396B2 (en) * 2005-06-07 2011-02-15 Cfph, Llc Enhanced system and method for managing financial market information
US20060024126A1 (en) 2004-07-31 2006-02-02 Johansson Kenneth L Page lifting spring
US20060242126A1 (en) * 2005-03-25 2006-10-26 Andrew Fitzhugh System and method for a context-sensitive extensible plug-in architecture
US7487461B2 (en) * 2005-05-04 2009-02-03 International Business Machines Corporation System and method for issuing commands based on pen motions on a graphical keyboard
US20160344581A9 (en) * 2005-09-21 2016-11-24 Amit Karmarkar Text message including a contextual attribute of a mobile device
US8726195B2 (en) 2006-09-05 2014-05-13 Aol Inc. Enabling an IM user to navigate a virtual world
US8564544B2 (en) * 2006-09-06 2013-10-22 Apple Inc. Touch screen device, method, and graphical user interface for customizing display of content category icons
US20080126075A1 (en) * 2006-11-27 2008-05-29 Sony Ericsson Mobile Communications Ab Input prediction
US8059101B2 (en) * 2007-06-22 2011-11-15 Apple Inc. Swipe gestures for touch screen keyboards
US8479118B2 (en) 2007-12-10 2013-07-02 Microsoft Corporation Switching search providers within a browser search box
US8504946B2 (en) * 2008-06-27 2013-08-06 Apple Inc. Portable device, method, and graphical user interface for automatically scrolling to display the top of an electronic document
US8319742B2 (en) * 2008-08-26 2012-11-27 Research In Motion Limited Portable electronic device and method of controlling same
US8769427B2 (en) 2008-09-19 2014-07-01 Google Inc. Quick gesture input
US20100185949A1 (en) 2008-12-09 2010-07-22 Denny Jaeger Method for using gesture objects for computer control
US8510665B2 (en) * 2009-03-16 2013-08-13 Apple Inc. Methods and graphical user interfaces for editing on a multifunction device with a touch screen display
US8294680B2 (en) * 2009-03-27 2012-10-23 Sony Mobile Communications Ab System and method for touch-based text entry
US8762893B2 (en) * 2010-05-14 2014-06-24 Google Inc. Automatic derivation of analogous touch gestures from a user-defined gesture
US20110314427A1 (en) * 2010-06-18 2011-12-22 Samsung Electronics Co., Ltd. Personalization using custom gestures
CN103827779B (en) 2010-11-20 2017-06-20 纽昂斯通信有限公司 The system and method for accessing and processing contextual information using the text of input

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1637686A (en) * 2004-01-06 2005-07-13 国际商业机器公司 System and method for improved user input on personal computing devices
CN101356493A (en) * 2006-09-06 2009-01-28 苹果公司 Portable electronic device for photo management
CN101546233A (en) * 2009-05-05 2009-09-30 上海华勤通讯技术有限公司 Identification and operation method of touch screen interface gestures

Also Published As

Publication number Publication date
KR20140001957A (en) 2014-01-07
KR20130116295A (en) 2013-10-23
WO2012068584A3 (en) 2012-10-11
EP2641146A4 (en) 2017-05-03
CN103314343B (en) 2017-02-15
CN103827779A (en) 2014-05-28
WO2012068584A2 (en) 2012-05-24
EP2641146A2 (en) 2013-09-25
CN103314343A (en) 2013-09-18
US20120127082A1 (en) 2012-05-24
KR20130143697A (en) 2013-12-31
US9244610B2 (en) 2016-01-26
EP2641147A2 (en) 2013-09-25
WO2012068544A3 (en) 2012-09-20
EP2641145A2 (en) 2013-09-25
WO2012068550A3 (en) 2012-10-04
US9189155B2 (en) 2015-11-17
WO2012068544A2 (en) 2012-05-24
EP2641145A4 (en) 2017-05-03
US9244611B2 (en) 2016-01-26
US20120127083A1 (en) 2012-05-24
CN103649876A (en) 2014-03-19
EP2641147A4 (en) 2017-05-03
US20120127080A1 (en) 2012-05-24
CN103827779B (en) 2017-06-20
WO2012068550A2 (en) 2012-05-24

Similar Documents

Publication Publication Date Title
US9244536B2 (en) Method, system, and graphical user interface for providing word recommendations
JP5912083B2 (en) User interface providing method and apparatus
JP6106787B2 (en) Device, method and graphical user interface for managing folders
CN103038728B (en) E.g. using a touch screen on the multi-mode mobile telephone text input system
KR101122869B1 (en) Annotation management in a pen-based computing system
AU2010345063B2 (en) Information search system with real-time feedback
KR101711110B1 (en) Glyph entry on computing device
KR20100135862A (en) Techniques for input recognition and completion
US20110201387A1 (en) Real-time typing assistance
KR20150070197A (en) Electronic text manipulation and display
US9189155B2 (en) Systems and methods for using entered text to access and process contextual information
KR20110016990A (en) User interface for application management for a mobile device
US20170132019A1 (en) Intelligent automated assistant in a messaging environment
US20110193795A1 (en) Haptic search feature for touch screens
JP6435305B2 (en) Device, method and graphical user interface for navigating a list of identifiers
EP2795441B1 (en) Systems and methods for identifying and suggesting emoticons
US9244905B2 (en) Communication context based predictive-text suggestion
US20180121528A1 (en) Process and Apparatus for Selecting an Item From a Database
US20140280093A1 (en) Social entity previews in query formulation
US9582549B2 (en) Computer application data in search results
KR20140105733A (en) Quick analysis tool for spreadsheet application programs
US9760541B2 (en) Systems and methods for delivery techniques of contextualized services on mobile devices
US20120113011A1 (en) Ime text entry assistance
JP5872691B2 (en) Text suggestions for images
CN105793840B (en) Previewing file and license while writing Email

Legal Events

Date Code Title Description
PB01 Publication
C10 Entry into substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
C14 Grant of patent or utility model
CF01 Termination of patent right due to non-payment of annual fee
CF01 Termination of patent right due to non-payment of annual fee

Granted publication date: 20170215

Termination date: 20171118