GB2487449A

GB2487449A - Touch screen input while the host device is a locked state

Info

Publication number: GB2487449A
Application number: GB1117863.9A
Authority: GB
Inventors: Kenneth Forbes Johnstone; Justin Philip Buck; Tsi Sheen Yap; Kevin David Joyce; Michael David Smith
Original assignee: INQ Enterprises Ltd
Current assignee: INQ Enterprises Ltd
Priority date: 2011-01-21
Filing date: 2011-10-14
Publication date: 2012-07-25
Also published as: WO2012098359A1; GB201101135D0; GB201117863D0

Abstract

Disclosed is a method of interacting with a touch screen device while the device is locked. The device has set of applications that can be activated when the device is unlocked, but cannot be activated when the device is locked. An input mechanism, such as a touch-sensitive display screen, detects user interactions with the device, and lock control logic transitions the device from a locked state to an unlocked state in response to a predefined user interaction. In response to predefined user interactions data capture logic invokes a data capture operation, independent of the lock state of the device.

Description

ELECTRONLC DEVICE AND METHOD WITH EFFICIENT DATA CAPTURE

Field of invention

The present invention relates to apparatus, methods and computer programs providing efficient data capture for electronic devices, including portable devices with touch-sensitive display screens ("touch-screenC). The invention provides solutions for efficient input, saving, retrieval and useofdata.

Backcirowtci Many mobile telephones, PDAs and other portable electronic devices feature touchscreens that provide ease of use by enabling intuitive user interaction with the device. Many touchscreens are implemented to enable users to select an item by finger touch, and to initiate an operation by finger gestures, avoiding the need for separate data input apparatus. Other touchscreens receive input from an input device such as a stylus, which can be packaged with the electronic device and is less cumbersome than a mouse and avoids the need for a trackpad. Various technologies have been used to implement touch-sensitivity, including capacitive touch sensors which measure a change in capacitance resulting from the effect of touching the screen, resistive touchscreens that measure a change in electrical current resulting from pressure on the touchscreen reducing the gap between conductive layers, and other technologies.

A problem with touchscreens is the increased likelihood of accidentally invoking an operation by applying pressure to the screen, such as accidentally making calls or switching the handset off.

This problem has been solved by locking' devices after a period of non-use or in response to an explicit lock request by the user, and then requiring users to carry out a predefined user-interaction to invoke an unlock' operation before they use any device functions. The unlock operation may involve a predefined touchscreen interaction, such as an identifiable gesture or using a drag operation to move an element of an interactive GUI object along a fixed path, although a dedicated hardware key for lock/unlock operations is provided on some mobile telephones. However, there is a trade-off between the speed with which users can access desired operations and the avoidance of accidental invocation of operations. The inventors of the present invention have determined that current solutions for avoidance of accidental invocation of operations on touch-sensitive devices have typically reduced useabillty.

Even for devices that do not have touchscreens, intuitive graphical user interfaces are now considered to be essential by most users of mobile telephones and many other data processing devices, Typical device users are very familiar with standard user-interaction operations to unlock a device, to navigate between display screens, and to select required applications from an on-screen menu by a sequence of key presses or by interacting with a displayed icon representing the application. TypicaJ device users can also work with multiple different applications on each device and can use complex interaction-sequences, when required, to unlock and activate required applications and to navigate back to a previous application or home screen. However, the inventors of the present invention have determined that significant improvements can be made to achieve more efficient data capture and more efficient access to required operations.

Summary

An electronic device according to one aspect of the invention comprises: an input mechanism for detecting user interactions with the device; a processing unit; wherein the processing unit is adapted to: compare user interactions with a set of predefined user interactions to identify a match; and respond to the user interaction matching a first predefined user interaction white the device is in S unlocked state, by invoking a data capture operation; and respond to the user interaction matching the first predefined user interaction white the device is in a locked state, by invoking the data capture operation.

A set of other device functions (other than the data capture operation) is inaccessible to the user in the locked state. The processing unit is adapted to transition the device from a locked state to an unlocked state in response to the user interaction matching a second predefined user interaction, and the set of inaccessible device functions becomes accessible to the user when the device is in the unlocked state.

The processing unit preferably comprises a data processor and program code for controlling the performance of operations by the data processor. The program code comprises data capture logic for displaying a data entry field on a display screen of the device and for setting user interface focus for the data entry field so that user inputs are entered into the displayed data

entry field.

The processing unit is adapted to respond to the first predefined user interaction when a device function is currently in use and is displayed on the device, to invoke the data capture operation and to position the data entry field so as to overlay the display of the in-use application.

Preferably, the invocation of the data capture operation is performed without changing the state of the in-use application.

In one embodiment, the data capture logic comprises logic for initiating a search operation using captured data, to search application content for one or more applications to identify application content matching the captured data. In one embodiment, the processing unit is adapted to carry out a persistent save operation in response to data being entered into the data entry field.

In one embodiment, the input mechanism is a touch-sensitive display screen. Gesture recognition logic performs the comparison between user interactions and the predefined set of user interactions.

A second aspect of the invention provides a method for data capture using an electronic device, comprising: detecting a user interaction with the device; comparing the user interaction with a set of predefined user interactions to identify a match; in response to the user interaction matching a first predefined user interaction while the device is in an unlocked state, invoking a data capture operation; and in response to the user interaction matching the first predefined user interaction while the device is in a locked state, invoking the data capture operation.

Another aspect of the invention provides an electronic device comprising: an input mechanism for detecting user interactions with the device; data capture logic that is responsive to a first predefined user interaction to invoke a data capture operation, independent of the lock state of the device Jock control logic that is responsive to a second predefined user interaction to transition the device from a Jocked state to an unlocked state, wherein a first set of device functions other than the data capture àperation can be used when the device is in an unlocked state, but cannot be used when the device is in a locked state.

By enabling the data capture operation to be invoked without a separately-invoked unlock operation, the user can access the data capture functionality faster than in conventionaJ devices.

The data capture operation preferably includes presenting an input screen including a data entry field on a display screen of the device, and giving the data entry field focus as the currently active GUI element on the display screen, in response to the first predefined user interaction.

This enables the user to start to enter data immediately after the data capture operation has been invoked. This reduces delays when compared with the sequence of process steps normally required for a device user to get to the text-entry stage of utilising an application. The data entry step is put at the start of the application use process, instead of requiring a screen unlock, a selection and launch of a data processing application, and creation of a new record (or navigation to the application's data entry screen), all of which are conventionally required before data can be entered into an electronic device. Using the present invention, data can be captured quickly and a suitable application can be selected and invoked later on, without delaying the actual data capture.

Data capture logic according to one embodiment of the invention invokes the data capture operation regardless of which applications are open on the device when the user interaction occurs. The user's current activity is not halted when the data capture operation is invoked, and application state is maintained for any open applications, but an entry screen of the data capture logic overlays the previously displayed application or applications. Data can be captured and either used straight away (e.g. sent as an instant message) or cached for later use, and then the user can return to his/her previous activity.

In one embodiment, the lock state of the device is responsive to input recognition logic for comparing detected user interactions with a predefined set of user interactions to identify a match. A first predefined user interaction is associated with the data capture logic that does not require a separately-invoked unlock operation, whereas another predefined user interaction is associated with an unlock operation that enables activation of a set of applications (lock-state-dependent applications). The input recognition logic may simply discard user inputs that do not match any of the predefined user interactions, but a user prompt is preferably displayed in response to non-matching interactions to remind the user of the need to unlock the device.

In one embodiment of the invention, the first predefined user interaction is associated with a function-specific unlock operation (i.e. specific to the data capture logic, to unlock data capture operations) and invocation of a data capture operation, such that this first predefined interaction initiates the data capture operation independently of the device's lock state whenever the interaction is performed. In another embodiment, the first predefined user interaction invokes a data capture operation, and an unlock operation is required before certain subsequent actions are performed on that data. Once again, this puts the data capture at the beginning of the process.

As noted above, one embodiment of the invention enables a fast data capture within a presented data entry field, when compared with solutions that require a device unlock operation and then a separate user interaction to explicitly select an application before a data entry screen is presented for that application. Indeed, many known applications require an unlock operation, then selection of an invocation of an application, and then selection of functions within an application, before data can be captured. Embodiments of the invention that unlock and invoke a data capture operation in response to a predefined user interaction, or invoke data capture prior to unlocking subsequent processing operations, enable the device user to capture required data more quickly.

Users often need to quickly record an item of data on an electronic device such as a mobile telephone, such as to record a phone number, an address, meeting arrangements, a train or flight departure or arrival time, and so on. Embodiments of the invention allow this to be done quickly without the need to select and navigate within the GUI of a conventional data management application. Additionally, embodiments of the invention allow various types of media to be captured, such as photographs, audio or video. According to one embodiment, the data entered via user interaction with a data entry screen of the data capture function can be written to persistent storage before navigating away from the data entry screen (e.g. saving each text character of media component as it is entered), so that the input data is reliably captured. This is described in more detail below and provides a further advantage over solutions that require an explicit user selection of an operation to be performed on the data before a persistent save is performed.

In one embodiment, the first predefined user interaction comprises a touch and drag operation in which a user touches a location on a touchscreen corresponding to an icon that represents the data capture function, and drags the icon by a continuous movement of the user's finger (or other input device) to an active' screen area. In other embodiments, the first predefined user interaction may be a voice command, a sequence of key presses or a sequence of touchscreen presses, or any other predefined and uniquely identifiable user interaction. In one embodiment, the data capture logic may start a recording of audio and/or video. In one embodiment, the data capture logic is responsive to a predefined user interaction to start a voice-to-text conversion operation -receiving audio into a device memory, converting to text, and writing the text to persistent storage, as well as displaying the converted text in the data entry field.

The data capture logic as described above preferably includes selection logic providing a mechanism for explicit user selection of an application or operation to be performed on captured data. The selection can be performed after capture of the data, instead of requiring an application to be selected and activated before any data is captured. Data can be captured quickly, and the user can subsequently specify which application or operation will use that data when convenient for the user.

The user-selectable operations may include an explicit save operation, a search operation to identify application content related to captured data, and/or a share operation to make the data available to one or more other users.

An electronic device according to another aspect of the invention comprises: a search engine; a first set of applications that each have a search interface that makes application content accessible to the search engine; and an adapter for translating application content of a second set of one or more applications into a format that is accessible to the search engine.

In a first embodiment, the adapter comprises an interface translator that is controlled by respective rules so as to adapt the translator for use with each of a piuraiity of the second set of applications. In one embodiment, new rules can be downloaded to the device to adapt the translator for use with new applications.

S

An electronic device according to one aspect of the invention comprises: a data storage unit providing persistent data storage; an input mechanism for detecting user interactions with the device; and data capture logic that is responsive to a predefined user interaction to invoke a data capture operation and to persistently save input data to said persistent data storage, wherein said persistent save operation is carried out automatically in response to data input and independent of user selection of a data processing operation to be performed on the acquired input data.

The invention also provides a method for automatic persistent saving of data that is input to an electronic device, comprising: detecting user interactions with the device; in response to detecting a predefined user interaction, invoking a data capture operation to capture input data; and persistently saving input data to persistent data storage on the electronic device, wherein said persistent save operation is carried out automatically in response to the inputof data, independent of user selection of a data processing operation to be performed on the acquired input data.

In particular, the data capture logic and method according to this aspect of the invention performs an automated persistent save without requiring user-selection of an explicit save operation or a specific data processing operation, so that the data capture includes a persistent save in response to data input, and the selection of a processing application or operation can be performed later.

The data capture logic may be provided as computer program code implementing a set of instructions for execution by a processor on the electronic device to control various operations performed on that device, for example as a program running on the Android operating system and integrated with Android search functionality. Equally, the term logic' applies to electronic hardware that controls the performance of data capture operations on the electronic device, and applies to combinations of hardware and software. The persistent save may be performed as the data is input or in response to a trigger event, such as a power-off instruction or in response to a user command to navigate to a different device function.

Brief Description of DrawirlQs

Embodiments of the invention are described below in more detail, by way of example, with reference to the accompanying drawings in which: Figure 1 is a schematic representation of a mobile telephone, as a first example of an electronic device in which the invention may be implemented; Figure 2 is a representation of the software architecture of an electronic device running the Android operating system; Figure 3 is a sequence diagram showing three possible results of a user interaction, for a method and apparatus according to an embodiment of the invention; Figure 4 is a schematic flow diagram representing the logic underlying the sequence of events of Figure 3, according to an embodiment the invention; Figure 5 depicts a possible UI design for the data capture logic on a touchscreen device according to an embodiment of the invention; Figure 6 is a second sequence diagram representing a second method according to the invention; and Figure 7 is a third sequence diagram representing the operation of a Rules Engine according to the invention.

Description of_Embodiments

Mobile telephones have evolved significantly over recent years to include more advanced computing ability and a great deal of additional functionality beyond standard telephony functions. Such advanced phones are commonly known as "smartphones". In particular, many phones are used for text messaging, Internet browsing and/or email as welt as gaming.

Touchscreen technology is useful in phones since screen size is limited and touch screen input provides direct manipulation of the items on the display screen such that the area normally required by separate keyboards or numerical keypads is saved and taken up by the touch screen instead. Embodiments of the invention wilt now be described in relation to handheld smartphones, but various features of the invention could be implemented within or adapted for use in other electronic devices such as handheld computers without telephony capability or touchscreens, for example in e-reader devices, tablet PCs and PDAs.

Figure 1 shows an example electronic device. The device in this example is a mobile telephone handset, comprising a wireless communication unit having an antenna 101 and a radio signal transceiver 102 for two-way communications, such as for GSM and UMTS telephony, and a wireless module 103 for other wireless communications such as WiFi. An input unit includes a microphone 104 and a touchscreen 105. An output unit includes a speaker 106 and a display 107 for presenting iconic and/or textual representations of the device's functions. Electronic control circuitry includes amplifiers 108 and a number of dedicated chips providing ADC/DAC signal conversion 109, compression/decompression 110, encoding and modulation functions 111, and circuitry providing connections between these various components, and a microprocessor 112 for handling command and control signalling. Associated with the specific processors is memory generally shown as memory unit 113. Random access memory (in some cases SDRAM) is provided for storing data to be processed, and ROM and Flash memory for storing the phone's operating system and other instructions to be executed by each processor.

A power supply 114 in the form of a rechargeable battery provides power to the phone's functions. The touchscreen 105 provides both an input mechanism and a display for presenting iconic or textual representations of the phone's functions, and is coupled to the microprocessor 112 to enable input via the touchscreen to be interpreted by the processor. There may be a number of separate microprocessors for performing different operations on the electronic device. These features are well known in the art and will not be described in more detail herein.

In addition to integral RAM and ROM, a small amount of storage capacity is provided by the telephone handsets Subscriber Identity Module (SIM card), which stores the user's service-subscriber key (IMS1) that is needed by GSM telephony service providers and handling authentication. The SIM card typically stores the user's phone contacts and can store additional data specified by the user, as well as an identification of the user's permitted services and network information.

Many older types of cellular telephone include a physical keyboard and a non-touch-sensitive display screen, but the example of a touchscreen phone is used here as some embodiments of the invention are particularly suitable for touchscreen devices, as explained below. Compared with older phones, the latest "Smartphones" tend to have improved battery life, increased RAM and higher speed processors, whereas mobile phones have until very recently been characterised by limited RAM and little persistent data storage.

During recent years, typical mobile telephones have included many additional features beyond basic telephony, such as text messaging, Bluetooth connectivity with nearby devices, FDA functions such as calendars and email, music playback, built-in cameras, alarm clock, etc. The previous differentiation between different types of portable electronic device is no longer clearly defined. Various aspects of the present invention are applicable to a number of different types of handheld electronic devices and other data processing apparatus, such as netbooks, e-reader devices, tablet PCs and PDAs, as well as mobile telephones.

As with most other electronic devices, the functions of a mobile telephone are implemented using a combination of hardware and software. In many cases, the decision on whether to implement a particular functionality using electronic hardware or software is a commercial one relating to the ease with which new product versions can be made commercially available and updates can be provided (e.g. via software downloads) balanced against the speed and reliability of execution (which can be faster using dedicated hardware), rather than because of a fundamental technical distinction. The term logic' is used herein to refer to hardware and/or software implementing functions of an electronic device. Where either software or hardware is referred to explicitly in the context of a particular embodiment of the invention, the skilled reader will recognize that alternative software and hardware implementations are also possible to achieve the desired technical effects, and this specification should be interpreted accordingly.

The latest mobile telephones and other portable electronic devices are able to run a great many different application programs, which each run on top of the device's operating system. The number of available applications is increasing very quickly, increasing the need for high speed processors, increased RAM and improved task management. Users typically choose required applications from the set of applications installed on their device by selecting from a list menu or interacting with an icon representing the application, it is also possibie for new software applications to be downloaded to an electronic device from a remote server computer, subject to acceptance of license terms, or to be installed from media that is insertable into the device.

Example operating systems used on mobile telephones are Google's Android mobile device operating system, Nokia's Symbian OS, Microsoft's Windows Mobile and the Blackberry OS from Research In Motion, Inc. As shown in Figure 2, the software architecture on a mobile telephone using the Android operating system, for example, comprises object oriented (Java and some C and C++) applications 200 running on a Java-based application framework 210 and supported by a set of libraries 220 (including Java core libraries 230) and the register-based Dalvik virtual machine 240. The Dalvik Virtual Machine is optimized for resource-constrained devices -i.e. battery powered devices with limited memory and processor speed. Java class files are converted into the compact Dalvik Executable (.dex) format before execution by an instance of the virtual machine. The Dalvik VM relies on the Linux operating system kernel for underlying functionality, such as threading and low level memory management. The Android operating system provides support for touchscreens, GPS navigation, cameras (still and video) and other hardware, as well as including an integral Web browser and graphics support and support for media playback in various formats. Android supports various connectivity technologies (CDMA, WiFi, UMTS, Bluetooth, WiMax, etc) and SMS text messaging and MMS messaging, as well as the Android Cloud to Device Messaging (C2DM) framework. Support for media streaming is provided by various plug-ins, and a lightweight relational database (SOLite) provides structured storage management. With a software development kit including various development tools, many new applications are being developed for the Android OS. Currently available Android phones include a wide variety of screen sizes, processor types and memory provision, from a large number of manufacturers. Which features of the operating system are exploited depends on the particular mobile device hardware.

The Android operating system provides a screen lock feature to prevent accidental initiation of communications and other operations, such as starting another application or switching off the device. Applications are typically inaccessible until the user has performed the required unlock operation. This is additional to the pattern of user interactions that can be set up as a security code. As mentioned above, screen Lock features have been considered desirable to prevent unintended activation of device functions.

A first embodiment of the present invention that is described in detail below (Embodiment 1) provides access to certain device functions without requiring the device user to perform the usual unlock operation as a first step, and without requiring the user to select an action to be performed as the next step. The choice of what action to perform on captured data can be deferred until after the data is captured. Data capture logic according to this first embodiment responds to a predefined user interaction to display a data input screen including a data entry field on the device screen, and to give focus to the data capture function's input screen. An input cursor is positioned in the data entry field so that any input text is captured within the data entry field. Alternative implementations of this first embodiment support audio data input, either using speech-to-text conversion software and adding the converted text to the displayed data entry field or creating an audio file such as an MP3 file of the captured audio data. Other embodiments enable input of video and/or images such as photographs. In one embodiment, the user is able to initiate a data capture operation and capture input data without having to specify in advance which application program will be used to perform operations on that data.

Data capture logic according to a second embodiment of the invention (Embodiment 2, described below) provides improved integration between the data capture logic and a set of data processing applications, to enable the data processing applications to be queried and/or to be invoked from the data capture logic. In one embodiment, adapters are created to make existing content provider applications into searchable sources of data content.

In a third embodiment (Embodiment 3, described below), input data and device state is saved persistently as a consequence of the initial data capture operation, without requiring an explicit persistent save operation to be invoked by the user.

Each of the embodiments mentioned above and described in detail below can be implemented independent of the other embodiments, and can each contribute to ease of use of an electronic device. However, preferred embodiments of the invention combine features of a plurality of the described embodiments as this can provide a data capture solution offering a significantly improved user experience.

Data capture logic according to one preferred embodiment of the invention achieves a synergistic combination of the above-described features arid embodiments, to enable fast and reliable data capture. The user can return to the captured data later, when he/she wishes to choose one or more operations to perform on that data. Therefore, at the time of data capture, the user can defer decisions about whether to send the data via an email or an SMS message, or to save the data in association with a particular application, or to carry out a search for application content on the device.

Thus, the various embodiments of the present invention have overcome a number of different constraints affecting the efficiency with which operations can be performed on an electronic device. A number of specific features and embodiments are described below in more detail.

1. Lock-independent and state-independent data capture An embodiment of the invention is described below with reference to Figures 3 and 4. In this embodiment, an electronic device touchscreen detects user interactions with the device and generates a measurable electrical effect (e.g. a change in capacitance) in response to that interaction. This electrical effect is converted to a digital signal representing the interaction, which signal is compared with a set of digital representations of predefined user interactions.

Gesture recognition logic for implementing such a comparison is well known in the art, such as for identifying a gesture that has been associated with an unlock operation. If the user input matches the required user interaction for the unlock operation, the unlock operation is invoked.

This operation transforms the device from a locked state to an unlocked state.

In the present embodiment, a number of operations are associated with respective user inputs such as operation-specific gestures, and the associations between operations and gestures are dependent on the state of the device. This can be handled in several ways, and two exemplary embodiments are described below.

In one embodiment, two separate gesture-responsive handlers are created: a 4Locked Handler' and an Unlocked Handler, and one of these handlers is used according to the current state of the device. The appropriate handler is assigned when the device changes state from locked to unlocked, or vice versa, It is preferred that the Locked Handler object has the functionality to identify both a first predefined user gesture that is assigned to unlock and activate the data capture logic without a general unlock, and a second predefined user gesture assigned to change the device from a locked to unlocked state. The Locked Handler object additionally contains the functionality to implement the state changes. The Unlocked Handler is only required to have the functionality to identify the first predefined user gesture, which is assigned to change the data capture logic from a baked to an unlocked state, and to implement said change. This implementation using two separate gesture-responsive handlers is preferred as it allows the logic flows of the two different operations to be separated, such that different functionality can easily be assigned to each handler as required.

If the first predefined user interaction is performed when the device is in the locked state, the locked handler handles both a function-specific unlock operation and a data capture operation.

The gesture that initiates the fast data capture is treated as a request to both unlock and initiate a data capture operation (i.e. an operation-specific unlock). This is shown in Figures 3 and 4.

Firstly, the detected user interaction (e.g. movement of an icon representing the data capture functionality, or a predefined distinctive finger gesture) 10 invokes an unlock operation 20 that changes the data capture logic from a locked to an unlocked state. Additionally, the same interaction invokes a data capture operation of the data capture logic 30. In one embodiment, this operation involves displaying a data entry screen on the touchscreen GUI of the device. The data entry screen includes a data entry field with a cursor indicating the position at which newly-typed text will appear. Figure 4 shows the logic underlying the gesture recognition and invocation of a data capture operation. A user input gesture 310 is detected 320 and compared 330 to a list of predefined gestures stored within the device. If this comparison operation 340 results in the finding that the user input gesture 310 corresponds to a predefined general unlock gesture, the screen is unlocked 350, restoring full functionality to the device. Alternatively, if the comparison operation 340 results in the finding that the user input gesture does not correspond to the general unlock gesture, a second comparison operation 360 is invoked in order to determine if the user input gesture 310 matches a predefined function-specific unlock or activation gesture. In some embodiments, function-specific gestures can be used to unlock or activate one of a plurality of different functions, using equivalent user-gestures (such as drag or flick an icon through an interface) but performed by interacting with different icons that represent different functions so as to unlock or activate the required function. In other embodiments, the interaction required to activate one function can differ from the interaction required to activate a different function, such as in the case of a dedicated data capture invocation gesture. In either case, the present invention invokes a data capture operation in response to recognition of an associated first defined user interaction, and this puts makes it possible to start a data capture more quickly than in known devices.

If a match is found for an input gesture corresponding to the data capture function, this specific function of the device is unlocked and a data input screen is displayed 380, or another data capture operation is activated. There may be different input gestures assigned to invoke different data capture operations (e.g. display data entry field, start audio or video recording).

Alternatively, a single gesture may result in a default data input screen being displayed with a text entry field, and other media capture operations such as video and audio capture can be activated from that screen. If the user then starts to input data 390, input text or video or an indicatio of progress of audio capture is displayed 400 on the display screen and saved to persistent storage. The data input may be iterative in the sense that each character of a text input can be written to persistent storage as it is typed. Subsequent to the fast data capture, the user is able to select a desired action to perform on the captured data. For example, text may be typed and the user subsequently decides whether to use this within an SMS message or within an email (or any mechanism for sharing the data with other device users), or to save that data for reuse, or to activate a search for content stored on the electronic device or elsewhere in a network that matches the captured data.

if no match is found by either comparison operation 340 or 360, an unlock reminder 370 is displayed on the screen of the device, to give feedback to the user on why no operation has been invoked.

The specific appearance of the data entry screen is not critical to this invention and so can vary according to different embodiments. This appearance may depend on whether the user has started to capture a piece of data or the data capture function is just being invoked. One possible appearance is presented in Figure 5. Within this embodiment, the state of the data capture logic is maintained when other applications are invoked, and the appearance of the keyboard 40 and of the view select buttons 50 and contents of the data entry field 60 are preserved from the last interaction with the data capture logic, in the case where the current view select mode is a search function, the list of displayed results is updated upon reinitialisation of the data capture component to reflect any changes having occurred in the intermediate period between the current and last interaction with the data capture logic. This functionality allows the user to immediately continue with a previously started task or to retrieve previously stored information rapidly.

If the first predefined user interaction is performed when the device is in an unlocked state, the Unlocked handler handles the fast data capture operation. Depending on the desired operation, the Locked Handler and Unlocked Handler may invoke different methods in response to data capture gestures (such as the method using a specialized operation during locked state, as described below).

In an alternative embodiment, a touchscreen handler has a single gesture recognition engine, which queries the state of the device, and uses a conditional statement to determine which functions to initiate. A software-implemented solution according to this second embodiment requires fewer lines of program code than a software implementation of the previously-described solution, but the flow of control through the handler is less intuitive. This alternative is shown in Figure 6.

In the embodiment of Figure 6, the detection of a predefined distinctive gesture 70 invokes the data capture logic 80. The data capture logic then queries the lock status of the device 90, with subsequent operations being conditional on the response to this query. In the case that the device state is returned as unlocked, the data capture logic proceeds in a state as described above in the first embodiment. ln the case that the device state is returned as locked 100, an instance of the data capture logic is created which provides restricted functionality to the user.

Non-limiting examples of such restrictions are to prevent access to personal user information stored on the device (for example, SMS messages or emails) or to prevent access to controls relating to the device's system settings (for example, pin entry settings). In this embodiment the data capture logic's GUI presented to the user may appear and behave as that detailed above in the first embodiment, subject to the removal of any information or functionality classed as restricted.

The predefined user interaction for initiating data capture may be a predefined finger gesture, detected by a touchscreen that provides the mechanism for detecting user interactions, although other interactions are also suitable as desôribed below. The data capture logic may be implemented as an event listener that cooperates with the input recognition logic so as to be triggered by detection of the predefined finger gesture. The data capture operation may comprise opening a data input screen on a GUI of the device, which enables the user to immediateiy input data. in this embodiment, the data input screen is accessible very quickiy and preferably includes a data entry field into which a device user can type data. In particular, the data capture logic of this embodiment enables a user to access the data entry screen and to start typing data (without having to carry out a conventional sequence of steps for unlock, application selection and navigation to select desired operations, in advance of inputting data).

2. Data cpture logic and integration with data processing applications As mentioned above, one embodiment of the invention allows captured data to be used in a search operation, to find and retrieve application content that matches the input data. In particular, the search function is one of a set of selectable operations for processing the data that is typed into the data entry field of the data capture logic.

In an electronic device that makes use of the Android operating system, a search manager provided by the Android operating system is used to execute searches. Each application program of a first set of application programs implements the ContentProvider class so as to expose their application content to queries initiated from other applications. In particular, application programs can implement an interface to make their application content accessible via searches initiated from the data capture logic. This can be implemehted using Search Providers to allow the Android search framework to be used, without having to build a dedicated search engine for each application. The search framework within Android provides a search dialog box (referred to as the Quick Search Box' in Android version 1.6) corresponding to the above-described data entry screen of the data capture logic. This can be extended with the above-described lock-state-independent access, and/or with an automatic persistent save of captured data and/or using application-specific adapters, for an enhanced data capture and to enable a search for related application content.

When a user types text into the data entry field and selects the search function, the Android search framework passes the query text to a chosen application program (either chosen by the user or selected automatically based on association with the entered data). If the application program provides an appropriate search API, the typed text can be used as a search key and compared with data held within tables of the application program. Matching data is identified and presented on the device's display screen.

For a second set of applications that do not impiement a suitable search API and do not expose their application content in a searchable form, adapters are provided to translate the application content into a suitable format for it to be accessed by the search manager.

For example, applications running on an electronic device that includes the Android operating system may use content providers to make their data available to other applications on the device in a certain format, and yet the standard methods of a content provider may not be suitable to make the application's data searchable from the search dialog box. An adapter can be provided to achieve the desired search and display capability, as an interface translator and a set of rules that are specific to individual applications, with the rules modifying the behaviour of the interface translator so as to perform an appropriate translation for the respective application. In one embodiment of the invention, new rules can be downloaded to the device to adapt the translator for use with new applications.

For example, let us assume that an application program such as the foursquare application (which allows users to "check" in at recently visited locations to identify their visits to those locations) exposes user details in the following format: [NAME rUSERID LASTCHECKIN [XYZ abcl23 Starbucks, Battersei flDEF ghi456 INQ OfficQ Battersea -Now, let us assume that the required presentation format is a different format: XYZ Foursquare. ShowUser(abcl 23) sV Starbucks, Battersea DEF DEF, Foursquare.ShowUser(ghi456) INQ Office, Battersea Where "XYZ" is the user name to be shown, the middle column provides the data element to show with the search results, and the last column provides the function to invoke when the user chooses that search result.

2.1 Typical behaviour without adaptors In this embodiment, the data capture logic is implemented as a set of Java Classes, and comprises query methods for retrieving data from other applications running on the same device and data formatting methods to adapt the retrieved data to a form that can be presented to the device user. The data capture logic is able to retrieve data from each of a set of applications, using application-specific instances of a searchProvider class that extends the Android operating system's contentProvider class. Application programs can be adapted to interoperate with the data capture logic of the present invention by implementing searchProvider methods within external adapters to retrieve, adapt and export required data.

The Android operating system thus provides a search framework that can be extended to enable users to search data content that is located on the device or elsewhere within a network.

Android's search framework provides a user interface from which users can initiate a search, and an interaction layer that communicates with applications, but an application-specific interface is needed that is appropriate for each application's data. When a suitable API has been implemented for an application, the search framework manages the life of the search dialog. When a user types data into the user interface and invokes a search, the search framework passes the query text to the application so that the application can perform a search.

2.2 Adapting to non-searchable data The data capture logic of one embodiment of the present invention uses and extends the search framework provided by the Android operating system, by means of application-specific interface adapters that implement searchProvider methods to export application data. These adapters perform a conversion between the data format and data storage structures exposed by the particular application program, and a format which is usable by the search framework and the data capture logic. In particular, for applications with databases that can export the database content via a generic interface but do not implement a searchProvider method, mapping rules are used to adapt the generic interface into a searchProvider interface. The resulting improved integration between the search framework and applications provides a much improved search capability when combined with a fast-access to the search user interface, such as described above with reference to embodiment 1. An implementation of the fast-access data capture logic includes a GUI that provides an option to carry out a search using user-typed data, and the rules-based adaptation to provide a searchProvider method for a plurality of applications extends the scope of the search. This data capture logic preferably provides a mechanism for initiating a number of different operations using user-typed data, including a search operation, a save operation and a share operation. One implementation of the save operation is described below (see embodiment 3). The share operation makes the typed data available to other users.

One implementation of search uses the Android SearchManager component to send a user-entered search query to the application. Specifically, the SearchManager creates an Intent' to pass the search query to either a searchable Activity' of an application that the user explicitly selects to handle the search, or to an intermediary generic searchable Activity that has been declared to receive all search queries, select applications and search their data, and display search results.

For applications that do not include a searchProvider method to search the application data, an adapter is required to implement that searchProvider functionality. The generic searchable Activity of the present embodiment interfaces with existing application APIs or with adapters as required. Although it is possible to hand-code a searchable Activity into each application program to facilitate integration with the search capability of the data capture logic of the invention, it is preferred to avoid such recoding and yet to make the data capture logic available to legacy' applications which do not expose their data in the format required by the search engine. This is achievable by providing a set of downloadable adapters for known applications, which adapters provide the required interface without having to rewrite the applications themselves. These adapters can be custom-written program code, but a preferred implementation involves use of a set of rules to map the data from the application-provided formats into the required format(s) for the search engine. The rules are interpreted at run4ime (for example by generating classes, or interpreted "on the fly") to make previously-unsearchable data sets into searchable data sets.

2.3 Usinq the captured data As discussed earlier, embodiments of the present invention provide means for data capture.

Once such data is captured, it is desirable to be able to perform one or more operations with the data; for example, sending text to an email or SMS composing application, or for posting on a social media website such as Twitter or Facebook. it is preferabie for the user Lo be abe to send the data they have entered directly to the application that handles the operation they wish to perform, without having to undergo the time consuming process of initialising the application separately and then entering (or copying) the data to this application. It is also preferable that the user be able to directly select the function they wish the application to perform, such that they do not need to navigate through the application to find the function manually.

In one embodiment of the present invention a Rules Engine for achieving the above mentioned objectives is provided. The Rules Engine can be integrated with any of the data capture logic described above, or alternatively may be provided as a standalone component.

The Rules Engine of the one embodiment provides a user interface displaying applications that match a user's input data format. This allows the user to choose, from a filtered list of applications installed on their device, the application that they wish to invoke to use the data they have entered and/or the operation that they wish the application to carry out on the data.

The Rules Engine allows the user to send the data they have entered directly to a specific function of their chosen application, without having to initialise the application manually or having to limit their interaôtion to that application's data input mechanism.

In one embodiment, the available applications are displayed to the user in the form of a list based on matches with user input. The list may be incorporated into the GUI of the data capture logic of previous embodiments, or it may be displayed separately. The list may be displayed automatically once the data capture logic is invoked, or the user may be required to press a button or key or to execute a gesture to cause the list to be displayed. Alternative means of display, such as a grid, are within the scope of the present invention.

The Rules Engine monitors the input data as the user enters it and filters the list of applications according to a set of rules, where each rule is associated with an individual application. A single application may have associated with it more than one rule, with each rule relating to a different function offered by the application. The list of applications is filtered such that only applications that are compatible with the data the user has currently entered are displayed. The list is dynamic, as it is updated every time the data entered by the user is changed. The appearance of each entry in the list is defined by the Layout component of the rule it is associated with, as discussed later.

The Rules Engine also provides instructions to the user-selected application informing it of the operation it should perform on receipt of the data. This may be achieved by identifying a function within the application that performs the action desired by the user. The Rules Engine determines the appropriate instructions by parsing the rule associated with the application the user has selected. If multiple rules are associated with a single application, the user is given the option of selecting which function they wish the application to perform and the rule corresponding to this function is parsed.

ln one embodiment the Rules Engine initialises an application and passes control to the application once it has sent it the user's data. Once the application has completed whatever operations are required, it may then return control to the Rules Engine.

In an alternative embodiment, the Rules Engine retains control and directly invokes code to perform the operation desired by the user. This may be achieved by an API call to the application associated with the rule selected by the user. When executing in this manner, the Rules Engine may check for a return code or success string indicating the success or failure of the API call.

The framework of the present embodiment is readily extensible, as new rules can be written as new applications are developed and published. New rules are obtained e.g. by download from a web server over the internet and integrated into the Rules Engine, such that an option to transfer data to the new application that the new rule is associated with is provided. A rule may be bundled with an application installer, such that both can be downloaded as a single unit.

Alternatively, the rules may be self-contained and downloaded independently from any application. A combination of downloading with an application and separately from an application is equally possible.

The rules framework is also flexible, as the Rules Engine can ignore any rules that are associated with applications not currently installed on the user's device. This ensures that the user is only presented with the option to transfer their data to an application that is currently available.

The Rules Engine itself will now be described in more detail.

As mentioned previously, the Rules Engine comprises a set of rules and a rules parser, where each rule is associated with a functionality provided by an application. The rule itself comprises a data structure and structured data (such as XML, JSON, CSV or other machine-readable format).

Each rule in the set of rules of the Rules Engine has the following features: 1) A regular expression (regex) of the type well known in the art. The Rules Engine compares the regular expression with the data the user has entered, to determine whether it is appropriate to display an option to send data to the application associated with the rule.

An exemplary regex is: ((+[0-9],{9,}) 1(0(0-91(10,))) Those skilled in the art will recognise this regex as specifying a text string that starts with a plus sign that is followed by nine or more numerical digits or a text string that starts with a zero that is followed by ten or more numerical digits. It will therefore be apparent that this regex is suited for identifying a telephone number in the data a user has entered.

2) An Action component. The Action component provides instructions for the Rules Engine to allow it to transform the user entered data so that is can be used by the application associated with the rule. The transformation may include variable substitution from the regular expression.

The Action component also provides details of the operation that should be performed by the application associated with the rule. This may include providing an identification of the application that needs to be executed in order to carry out the operation, as well as identifying the function of the application that handles the operation that should be executed. Alternatively the Action component may only identify the function of the application that handles the operation that should be executed. Any known function of any application currently installed on the user's device can be identified in a rule.

In the case of the telephone number example, the Action component may identify that the Dialer application (which provides functionality to make a telephone call) is the appropriate application to invoke. The Action identifies the function of the Dialer that causes it to execute a Dial operation, and this is the appropriate function to execute. The Action would additionally contain instructions for the Rules Engine to send the user entered telephone number to the Dialer application in a format the Dialer application expects, such as a string containing eleven digits.

The Action may also specify a transformation of the number, such as adding a local dialing prefix for international dialing.

3) A Layout. The Layout determines the appearance of the list item corresponding to a particular rule. A Layout typically comprises text identifying the function of the application that the rule is associated with, although any other means of identification (icons, images, sounds etc.) may be used. The Layout may incorporate the icon of the application that the rule is associated with.

Combinations of items may also be used.

The Layout may optionally define additional Layout Rules for the Rules Engine to parse. These may refer to the regular expression, the Action and/or the application that will carry out the operation itself.

In the telephone number example, the Layout may instruct the Rules Engine to display the text Dial $1'in the list of available applications, where $1' is a placeholder that identifies to the Rules Engine that the telephone number entered by the user should bedisplayed. The Layout may also instruct the Rules Engine to display the icon associated with the Dialer application next to the Dial $1' text. Layout Rules such as displaying a telephone number in red when it corresponds to an international call and showing the flag of the destination country, or showing the icon of the person whose number is to be called may also be implemented in this example.

An exemplary rule (called Tweet') according to the present invention is provided below.

Comments to aid the skilled reader in interpreting this rule are provided in square parentheses.

The Tweet' rule allows user entered data to be posted directly to social networking and microblogging service Twitter as a so-called Tweet'.

<?xml version="l.O" encoding="utf-8"?> <share> C!---Tweet --> <action shareType="pattern" pattern="&4xOO5E; .{l, 140}$" tRegex component (1-140 characters)] intentAction="android. intent, action. SEND" [Action component: SEND] type=" text/plain" nane="Share to Twitter" <---[Layout component (text to be shown to the user)] extraName="android.intent. extra.TEXT" componèntName="com. twitter. android/ . PostActivity" [Action component: Posthctivity function] </share> The skilled reader will appreciate that the regex set out in the Tweet rule above is suited for identifying user entered data that is between I and 140 characters in length (140 characters being the maximum number of characters a single post to Twitter can contain). The Action component identifies that the user entered data should be sent by the Rules Engine to another application (the Twitter application in this case), and further identifies that the PostActivity' function of the Twitter application handle the posting of the data to the Twitter service.

With reference to Figure 7, the process by which the Rules Engine operates will now be described.

The process begins at step 700 with the user entering data into the data capture logic of any of the previously described embodiments. At step 705 the Rules Engine evaluates whether the entered data matches any of the regular expressions associated with one or more rules currently installed on the device. At step 710 the Rules Engine updates the display of the device to show only the GUI's of rules that have a regular expression matching the currently entered data.

Steps 700, 705 and 710 are then repeated until the user ceases to enter data. At this point the GUI's associated with all the rules having regular expressions matching the data the user has entered are displayed in a list of action options for the user to select from. At 715 the user selects one of the action options in a manner known to the skilled person (e.g. by pressing a button or touching the screen of the device).

At 720, the user interface informs the Rules Engine which action option was selected. The Rules Engine then parses the rule associated with the selected action option and at 725 gets the Action associated with the selected action option. Once the Action has been identified, at 730 the Rules Engine instructs the Operating System (OS) running on the device to launch the application specified by the Action, transforming and substituting the user entered data as required in the manner described earlier. At 735 the OS launches the application and function that are specified by the Action, as appropriate, so that the operation associated with the selected action option is carried out.

The process described with reference to Figure 7 demonstrates the delegation of an action to an application through the operating system. The skilled person, upon consideration of the teaching contained herein, will açpreciate that invocation could equally be through direct invocation of a function of the application by the Rules Engine (i.e. without intervention of the OS), or through direct action from the Rules Engine, such as invocation of web services.

The data capture logic of this embodiment also preferably handles writing to persistent storage, such as described below.

3. Automatic Persistent Save In this embodiment, data capture logic is provided that automatically saves state information persistently to a persistent memory location (this can be battery-backed RAM) or a database (the SQLite database in this example). Input data is saved as soon as the data is input (to a data entry field created by the data capture logic), in the following manner.

When the data capture logic is invoked to enable the user to capture a new data item, a data entry field is presented within the GU1 on the device screen and is given focus within the GUI, and a table is automatically created for that data item in the device's persistent SQLite database. As the user types input data into the data entry field, the data is written directly to the newly created database table. Any editing of the typed data is written directly to the database, rather than saving to a separate area of RAM and requiring an explicit persistent save instruction from the user before the database is updated. Thus, data is written to the database without an explicit save request, before any specific data processing operation is selected and even before a data processing application is selected. The user may have only performed a simple interaction (such as a predefined gesture on the touchscreen) to invoke the data capture logic, and then started typing into the newly displayed data entry field.

If the execution of the data capture logic is paused for any reason, such as if the user starts a new task while the data entry screen of the capture logic has focus within the Gui, the pause method triggers generation of a database commit request that is applied to the database. This commits changes to the database just before the data capture logic responds to the pause request. If the data capture logic is invoked to provide a new data entry field (switching focus within the GUI) while the first instance of the data capture logic is already active, any changes made to the first instance are committed.

This automatic creation and writing of data to tables in a persistent database, and explicit commit of changes to the database in response to certain events during the lifecycle of the particular instance of the data capture logic, has the advantage that the user can navigate away to a different activity or the device can be turned off or can terminate running applications without losing the input data typed into the data entry field of the data capture logic. Note that the initially created SOLite database table is not yet associated with any particular data processing application -the table is associated with the data capture logic only, until the device user subsequently selects a required operation/application.

When the user wishes to work with the captured data, the saved state information for the data capture logic enables the populated data entry screen to be displayed in the same state as when the user navigated away from that entry screen. The data that is persistently saved to an SQLite database is not read directly by other applications. Instead, the data capture application includes a set of options for operations that can be performed on the captured data, including a share operation that enables transmission to other device users (using email, SMS, bluetooth connectivity or other standard device functions) of the data that currently appears in the data entry field or in a list of search results within a user interface of the data capture logic. Another option is a user invokable save operation, and another option is the search option described above.

4. Addit[onal Embodiments In addition to the embodiments of the invention described in detail above, the skilled person will recognize that various of the features described herein can be modified and/or combined with additional features, and the resulting additional embodiments of the invention are also within the scope of the accompanying claims.

For exampie, the device and data capture iogic described above can be irripiemented with pre-installed application-specific program code for handling the retrieval and adaptation of data from each specific application program. However, a device or data capture logic according to another embodiment of the invention includes a rules-based adaptation engine that allows new processing rules to be downloaded to the device to handle data formats of additional applications.

The device and data Oapture logic described above includes search functions for retrieving data from other applications and data repositories on the device. However, a device according to another embodiment of the invention can integrate the data capture logic with additional search methods to invoke a communication with a nearby electronic device or via communication network links to a remote server computer or remote device.

The embodiment described in detail above refers to an implementation for the Android operating system, but the invention is clearly not limited to any particular operating system of particular hardware device. Electronic device manufacturers make a choice as to which operating system to adopt to support software applications running on their devices, and application developers write their applications for a particular operating system and making use of its available functions, but it is well understood that computer programs can typically be ported or adapted for other operating systems. In other cases, it may be appropriate to rewrite application code from scratch for a new operating system, but with equivalent functions and consistent design features. The required modifications to implement equivalent application functionality across a range of operating systems are well understood in the art and the references above to one specific operating system should be recognized as an example only.

Claims

Claims 1. An electronic device, comprising: an input mechanism for detecting user interactions with the device; a processing unit; wherein the processing unit is adapted to: compare user interactions with a set of predefined user interactions to identify a match; and respond to the user interaction matching a first predefined user interaction while the device is in an unlocked state, by invoking a data capture operation; and respond to the user interaction matching the first predefined user interaction while the device is in a locked state, by invoking the data capture operation.
2. An electronic device according to claim 1, wherein the processing unit is adapted to transition the device from a locked state to an unlocked state in response to the user interaction matching a second predefined user interaction, wherein a first set of device functions that is inaccessible in the locked state becomes accessible to the user in the unlocked state.
3. An electronic device according to claim I or claim 2, wherein the processing unit comprises a data processor and program code for controlling the performance of operations by the data processor.
4. An electronic device according to claim 3, wherein the program code comprises data capture logic for displaying a data entry field on a display screen of the device and for setting user interface focus for the data entry field so that a subsequent data input is entered into thedisplayed data entry field.
5. An electronic device accordingto claim 4, wherein the processing unit is adapted to respond to the first predefined user interaction when a device function is currently in use and is displayed on the device, to invoke the data capture operation and to position the data entry field so as to overlay the display of the in-use application.
6. An electronic device according to claim 5, wherein the invocation of the data capture operation is performed without changing the state of the in-use application.
7. An electronic device according to any one of claims 4 to 6, wherein the data capture logic comprises logic for initiating a search operation using captured data, to search application content for one or more applications to identify application content matching the captured data.
8. An electronic device according to any one of claims 4 to 7, wherein the processing unit is adapted to carry out a persistent save operation in response to data being entered into the dataentry field.
9. An electronic device according to any preceding claim, wherein the input mechanism is a touch-sensitive display screen.
10. An electronic device according to any one of the preceding claims, wherein the data capture operation is an audio capture operation.
11. An electronic device according to any one of the preceding claims, wherein the data capture operation comprises activating a data receive function and saving received data.
12. A method for data capture using an electronic device, comprising: detecting a user interaction with the device; comparing the user interaction with a set of predefined user interactions to identify a match; in response to the user interaction matching a first predefined user interaction while the device is in an unlocked state, invoking a data capture operation; and in response to the user interaction matching the first predefineci user interaction while the device is in a locked state, invoking the data capture operation.
13. A method according to claim 12, further comprising: in response to the user interaction matching a first predefined user interaction, transitioning the device from a locked state to an unlocked state from which each of a first set of device functions can be accessed, which first set of device functions is inaccessible by a device user when the device is in a locked state.
14. A method according to claIm 12 or claim 13, wherein the data capture operaUon comprises displaying a data entry field on a display screen of the device and setting user interface focus for the data entry field, so that subsequent user inputs are entered into thedisplayed data entry field.
15. A method according to claim 14, further comprising: responding to a user selection of a search operation, subsequent to entering data in the displayed data entry field, to initiate a search operation using the entered data to search one or more applications on the device for application content matching the captured data.
16. A method according to any one of claims 14 or 15, further comprising: performing a persistent save operation in response to data being entered into thedisplayed data entry field.