KR20120113770A - Apparatus and method for conditionally enabling or disabling soft buttons - Google Patents

Abstract

A method of operating a multifunction device includes displaying a soft keyboard having a plurality of unconditionally enabled buttons and one or more conditionally enabled buttons, wherein the first input by the first button is received at a first time. Detecting and responding to detecting the first input by activating the first button. In response to detecting the second input by the second button at a second time, the second button if the first button is an unconditionally enabled button and the second button is an unconditionally enabled button; Is activated. If the second button is a conditionally enabled button, if the period between the first time and the second time is greater than a predefined threshold, the second button is activated, and the period between the first time and the second time is If less than or equal to the predefined threshold, the second button is prevented from being activated.

Description

Apparatus and method for conditionally enabling or disabling soft buttons {APPARATUS AND METHOD FOR CONDITIONALLY ENABLING OR DISABLING SOFT BUTTONS}

The present invention generally includes, but is not limited to, electronic devices having touch sensitive displays that can conditionally activate or deactivate soft buttons on a displayed user interface, such as softkeys or other objects on a soft keyboard. It relates to electronic devices having touch sensitive surfaces.

Recently, the use of touch-sensitive surfaces as input devices for computers and other electronic computing devices has increased significantly. Exemplary touch sensitive surfaces include touch pads and touch screen displays. Such surfaces are widely used to manipulate user interface objects on a display.

Exemplary operations include entering characters using one or more soft keyboards. The user may need to perform such operations on user interface objects in any program or application in which character input is required.

However, existing methods of using such keyboards are cumbersome and inefficient. For example, buttons that hide the keyboard or switch to keyboards of different languages may be intentionally activated while typing on the keyboard, which makes the text input experience boring and causes a significant recognition burden on the user. In addition, some conventional methods take longer than necessary to complete a task, thus wasting user time and power reserve of the device, which can be a particularly important consideration for battery operated devices.

Thus, there is a need for computing devices with faster and more efficient methods and interfaces for activating buttons on a soft keyboard. Such methods and interfaces may complement or replace conventional methods for activating buttons on a soft keyboard. These methods and interfaces reduce the perceived burden on the user and create a more efficient human-machine interface. In battery operated computing devices, such methods and interfaces save power and extend battery charge intervals.

These drawbacks and other problems associated with user interfaces for computing devices having touch sensitive displays (also known as “touch screens” or “touch screen displays”, for example) are reduced by the devices of the present invention. do. As an example, the keyboard select button in the soft keyboard is temporarily disabled in response to the activation of any other soft keyboard buttons (eg, any of the letter buttons in the displayed soft keyboard). By conditionally disabling the keyboard select button, faster and more efficient typing by the user is possible, and accidental activation of the keyboard select button is avoided. Another example of a conditionally enabled button that facilitates faster and more efficient typing by a user is a keyboard hide button. Users typically do not want to hide the soft keyboard while actively typing, thus allowing the keyboard hide button (shown anywhere within the soft keyboard or in the user interface on the touch-sensitive display) to facilitate faster and more efficient typing by the user. Made with conditionally enabled buttons, saving energy and power reserves for the device.

In some embodiments, the device is a desktop computer. In some embodiments, the device is a portable (eg, notebook computer, tablet computer, or handheld device). In some embodiments, a device has a graphical user interface (GUI), one or more processors, memory and one or more modules, programs or instruction sets stored in memory that performs multiple functions. In some embodiments, the user interacts with the GUI primarily through finger touches and gestures on the touch sensitive display. In some embodiments, the functions may include various types of editing, drafting, presentation, word processing, website creation, disc authoring, spreadsheet creation, game play, telephone, video conferencing, emailing, instant messaging, practice support, digital Photo shoot, digital video shoot, web browsing, digital music play, and / or digital video play. Executable instructions for performing these functions may be included in a computer readable storage medium or other computer program product configured for execution by one or more processors.

According to some embodiments, a multifunction device includes a touch screen display, one or more processors, memory, and one or more programs. One or more programs are stored in memory and are configured to be executed by one or more processors. The one or more programs display a soft keyboard having a plurality of buttons including a plurality of unconditionally enabled buttons and one or more conditionally enabled buttons, the first input being a first input by the first button. Instructions for detecting in time and responsive to detecting the first input by activating the first button. The programs further include instructions responsive to detecting a second input by a second button at a second time and detecting a second input by a second button at a second time. For example, if the first button is an unconditionally enabled button and the second button is an unconditionally enabled button, the instructions activate the second button. If the first button is an unconditionally enabled button and the second button is a conditionally enabled button, and if the period between the first time and the second time is greater than a predefined threshold, the instructions are generated. 2 Activate the second button in response to detecting the input. However, if the period between the first time and the second time is less than or equal to the predefined threshold, the instructions prevent the second button from being activated.

According to some embodiments, the method is performed in a multifunction device having a touch screen display. The method includes displaying a soft keyboard having a plurality of buttons, the plurality of buttons including a plurality of unconditionally enabled buttons and one or more conditionally enabled buttons, the first time being input by the first button. Detecting, and responsive to detecting the first input by activating the first button. The method further includes detecting a second input by a second button at a second time, and responding to detecting a second input by a second button at a second time. If both the first button and the second button are unconditionally enabled buttons, the method activates the second button. However, if the first button is an unconditionally enabled button and the second button is a conditionally enabled button, the method conditionally disables the second button. Specifically, if the period between the first time and the second time is greater than the predefined threshold, the method activates the second button, and the period between the first time and the second time is equal to or greater than the predefined threshold. If small, the method disables activation of the second button.

According to some embodiments, a computer-readable storage medium, when executed by a multifunction device having a touch screen display, causes the device to cause a plurality of unconditionally enabled buttons and one or more conditionally enabled buttons. Instructions are displayed to display a soft keyboard having a plurality of buttons, to detect a first input by a first button at a first time, and to respond to detecting the first input by activating the first button. . Execution of the instructions also causes the apparatus to respond to detecting a second input by the second button at a second time and detecting a second input by the second button at a second time. If the first button is an unconditionally enabled button and the second button is an unconditionally enabled button, execution of the instructions causes the device to activate the second button. However, if the first button is an unconditionally enabled button and the second button is a conditionally enabled button, execution of the instructions causes the device to conditionally activate the second button. . Specifically, if the period between the first time and the second time is greater than the predefined threshold, execution of the instructions causes the device to activate the second button and the period between the first time and the second time. If equal to or less than this predefined threshold, execution of the instructions causes the device to prevent the second button from being activated.

According to some embodiments, a graphical user interface on a multifunction device having a touch screen display, a memory, and one or more processors executing one or more programs stored in the memory, includes a plurality of unconditionally enabled buttons and one or more conditional. It includes a soft keyboard having a plurality of buttons including the enabled button. When the first input by the first button is detected at the first time, the detection of the first input is responded by the activation of the first button. The second input by the second button is detected at the second time, and the second input by the second button is detected at the second time. If the first button is an unconditionally enabled button and the second button is an unconditionally enabled button, the second button is activated. However, when the first button is an unconditionally enabled button and the second button is a conditionally enabled button, the second button is conditionally activated. Specifically, the second button is activated if the period between the first time and the second time is greater than the predefined threshold, and if the period between the first time and the second time is less than or equal to the predefined threshold, the second button is activated. 2 Button activation is prevented.

According to some embodiments, a multifunction device includes means for displaying a soft keyboard having a touch screen display, a plurality of buttons including a plurality of unconditionally enabled buttons and one or more conditionally enabled buttons, Means for detecting a first input by a first button at a first time and means for responding to detecting the first input by activating the first button. The apparatus further includes means for detecting a second input by the second button at a second time and means for responding to detecting the second input by the second button at a second time. The means for responding to detecting the second input activates the second button if the first button is an unconditionally enabled button and the second button is an unconditionally enabled button. However, means for responding to detecting the second input is conditionally activating the second button if the first button is an unconditionally enabled button and the second button is conditionally enabled. Let's do it. Specifically, if the period between the first time and the second time is greater than a predefined threshold, the second button is activated by means responsive to detecting the second input. And if the period between the first time and the second time is less than or equal to the predefined threshold, the second button is prevented from being activated by means responsive to detecting the second input.

According to some embodiments, an information processing device for use in a multifunction device having a touch screen display includes a plurality of buttons including a plurality of unconditionally enabled buttons and one or more conditionally enabled buttons. Means for displaying a soft keyboard having, means for detecting a first input by a first button at a first time, and means for responding to detecting the first input by activating the first button. The apparatus further includes means for detecting a second input by the second button at the second time, and means for responding to detecting the second input by the second button at the second time. If the first button is an unconditionally enabled button and the second button is an unconditionally enabled button, the means responsive to detecting the second input activates the second button. If the first button is an unconditionally enabled button and the second button is a conditionally enabled button, the means responsive to detecting the second input conditionally activates the second button. Specifically, if the period between the first time and the second time is greater than the predefined threshold, the means for responding to detecting the second input activates the second button and the period between the first time and the second time. If equal to or less than this predefined threshold, the means responsive to detecting the second input prevents the second button from being activated.

According to some embodiments, a multifunction device includes a touch screen display, one or more processors, memory, and one or more programs. One or more programs are stored in memory and configured to be executed by one or more processors. One or more programs display a soft keyboard having a plurality of buttons, the plurality of buttons including a plurality of unconditionally enabled buttons and one or more conditionally enabled buttons, each with an unconditionally enabled button. Detects the first input at a first time, and responds to detecting the first input at a first time by activating each unconditionally enabled button, and conditionally for a predefined period starting from the first time. Instructions for disabling at least one of the enabled buttons. The one or more programs detect a second input by a conditionally enabled button disabled in response to the first input, the second input being detected at a second time within a predefined period of time; Further comprising instructions responsive to detecting the second input at a second time by ignoring the two inputs.

According to some embodiments, the method is performed in a multifunction device having a touch screen display. The method includes displaying a soft keyboard having a plurality of buttons, the plurality of buttons including a plurality of unconditionally enabled buttons and one or more conditionally enabled buttons, each by an unconditionally enabled button. Detecting a first input at a first time, and by activating each unconditionally enabled button and disabling at least one of the conditionally enabled buttons for a predefined period starting from the first time And responding to detecting the first input at a first time. The method includes detecting a second input by the conditionally enabled buttons disabled in response to the first input, the second input being detected at a second time within a predefined period of time, and And responding to detecting the second input at a second time by ignoring the second input.

According to some embodiments, a computer-readable storage medium, when executed by a multifunction device having a touch screen display, causes the device to cause a plurality of unconditionally enabled buttons and one or more conditionally enabled buttons; Displaying a soft keyboard having a plurality of buttons, each detecting a first input by an unconditionally enabled button at a first time, respectively activating the unconditionally enabled button, Instructions are stored that, in response to detecting the first input at a first time, by disabling at least one of the conditionally enabled buttons for a predefined period starting from one hour. Execution of the instructions also causes the apparatus to detect a second input by a conditionally enabled button that is disabled in response to the first input, the second input being at a second time within a predefined period of time. Detected-to respond to detecting the second input at a second time by ignoring the second input.

According to some embodiments, a graphical user interface on a multifunction device having a touch screen display, a memory, and one or more processors executing one or more programs stored in the memory, includes a plurality of unconditionally enabled buttons and one or more conditional. It includes a soft keyboard having a plurality of buttons including the enabled button. A first input by each unconditionally enabled button of the graphical user interface is detected at a first time. In response to the first input being detected at the first time, each unconditionally enabled button is activated, and during a predefined period of time at least one of the conditionally enabled buttons starts from the first time. Is disabled. A second input by a conditionally enabled button that is disabled in response to the first input is detected at a second time that is within a predefined period, and the second input is ignored.

According to some embodiments, a multifunction device includes means for displaying a soft keyboard having a touch screen display, a plurality of buttons including a plurality of unconditionally enabled buttons and one or more conditionally enabled buttons, Means for detecting a first input by a first unconditionally enabled button at a first time, and means for activating each unconditionally enabled button in response to detecting a first input at a first time It includes. The apparatus also includes means for disabling at least one of the conditionally enabled buttons for a predefined period starting from the first time, the disabled conditionally enabled button in response to the first input. Means for detecting a second input by means, and means for ignoring the second input if the second input is detected at a second time that is within a predefined period.

According to some embodiments, an information processing device for use in a multifunction device having a touch screen display includes a plurality of buttons including a plurality of unconditionally enabled buttons and one or more conditionally enabled buttons. Means for displaying a soft keyboard having, respectively, means for detecting a first input at a first time by an unconditionally enabled button, and unconditionally in response to detecting a first input at a first time, respectively. Means for activating the enabled button. The device includes means for disabling at least one of the conditionally enabled buttons for a predefined period starting from a first time, by means of a conditionally enabled button disabled in response to the first input. Means for detecting a second input, and means for ignoring the second input if the second input is detected at a second time that is within a predefined time period.

Thus, multifunction devices with touch screen displays are provided with faster and more efficient methods and interfaces for activating keys, thereby increasing the effectiveness, efficiency and user satisfaction with such devices. Such methods and interfaces may complement or replace conventional methods for activating keys.

BRIEF DESCRIPTION OF THE DRAWINGS For a better understanding of the foregoing and further embodiments of the invention, reference should be made to the following description of embodiments, taken in conjunction with the accompanying drawings, in which like reference numerals indicate corresponding .
Figures 1A and 1B are block diagrams illustrating portable multifunction devices with touch-sensitive displays in accordance with some embodiments.
1C is a block diagram illustrating exemplary components for event processing in accordance with some embodiments.
2 illustrates a portable multifunction device having a touch sensitive display in accordance with some embodiments.
3 is a block diagram of an exemplary multifunction device having a display and a touch sensitive surface in accordance with some embodiments.
4A and 4B illustrate exemplary user interfaces for a menu of applications on a portable multifunctional device according to some embodiments.
5A-5I illustrate example user interfaces for activating buttons in accordance with some embodiments.
6A and 6B are flow charts illustrating a method of activating buttons in accordance with some embodiments.
7 is a flow chart illustrating a method of activating buttons according to some embodiments.

DETAILED DESCRIPTION Hereinafter, exemplary embodiments will be described in detail with reference to the accompanying drawings. In the following detailed description, numerous specific details are set forth in order to provide a thorough understanding of the present invention. However, it will be apparent to one skilled in the art that the present invention may be practiced without these specific details. On the other hand, well-known methods, procedures, components, circuits, and networks have not been described in detail so as not to unnecessarily obscure aspects of the embodiments.

In addition, although terms such as first and second may be used herein to describe various components, these components should not be limited by these terms. These terms are only used to distinguish between components. For example, without departing from the spirit of the present invention, a first contact may be referred to as a second contact, and similarly a second contact may be referred to as a first contact. The first contact and the second contact are both contacts, but they are not the same contact.

The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. As used in the specification and the appended claims, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will also be understood that the term "and / or" as used herein refers to and includes all possible combinations of one or more of the related items listed. The use of the terms " comprise, "" comprise," " comprise, "and / But do not preclude the presence or addition of one or more other features, integers, steps, operations, components, components, and / or groups thereof.

As used herein, the term " when "may be interpreted to mean" when in " or "in response to, " Similarly, the phrase " when judged "or" when a stated condition or event is detected " Quot; or "in response to detection of [the stated condition or event].

Embodiments of computing devices, user interfaces for such devices, and related processes for using such devices are described. In some embodiments, the computing device is a portable communication device, such as a mobile phone, that further includes other functions such as PDA and / or music player functions. Exemplary embodiments of portable multifunction devices include, but are not limited to, iPhone® and iPod Touch® devices from Apple Inc. of Cupertino, Calif. Other portable devices such as laptops or tablet computers with touch sensitive surfaces (e.g., touch screen displays and / or touch pads) may also be used. It should also be appreciated that, in some embodiments, the devices are not portable communication devices, but desktop computers having a touch sensitive surface (e.g., a touch screen display and / or a touchpad).

In the following description, a computing device including a display and a touch sensitive surface is described. However, it should be appreciated that a computing device may include one or more other physical user interface devices, such as a physical keyboard, a mouse, and / or a joystick.

The device may be any of a variety of applications, such as drafting applications, presentation applications, word processing applications, web site creation applications, disk authoring applications, spreadsheet applications, gaming applications, currency applications, video conferencing applications, email applications, instant messaging applications, , Digital camera applications, digital video camera applications, web browsing applications, digital music player applications, and / or digital video player applications.

The various applications that may be executed on the device may use at least one common physical user interface, such as a touch sensitive surface. One or more functions of the touch sensitive surface as well as corresponding information displayed on the device may be adjusted and / or modified within the application and / or within each application. In this way, a common physical architecture (such as a touch sensitive surface) of the device can support a variety of applications with intuitive, user-transparent user interfaces.

The user interface may include one or more soft keyboard embodiments. A soft keyboard embodiment is disclosed in U.S. Patent Application No. 11 / 459,606, entitled " Keyboards For Portable Electronic Devices, "filed July 24, 2006, the entire contents of which are incorporated herein by reference. Standard configuration (QWERTY) and / or non-standard configuration symbols on the displayed icons of the keyboard, such as those described in U.S. Patent Application Serial No. 11 / 459,615, entitled "Touch Screen Keyboards For Portable Electronic Devices". Keyboard embodiments may include fewer icons (or soft keys) than the number of keys on an existing physical keyboard, such as for a typewriter. This may facilitate the user selecting one or more icons of the keyboard and thus one or more corresponding symbols. Keyboard embodiments may be adaptive. For example, the displayed icons may be modified according to user actions such as selection of one or more icons and / or selection of one or more corresponding symbols. One or more applications on the portable device may use common keyboard embodiments and / or different keyboard embodiments. Thus, the keyboard embodiment used may be tailored for at least some of the applications. In some embodiments, one or more keyboard embodiments may be customized for each user. For example, one or more keyboard embodiments may be customized for each user based on each user's word usage history (dictionary, slang, individual usage). Some of the keyboard embodiments may be adjusted to reduce the probability of a user making a mistake when selecting one or more icons and thus selecting one or more symbols when using a soft keyboard embodiment.

Hereinafter, an embodiment of a portable device having a touch-sensitive display is discussed. 1A and 1B are block diagrams illustrating a portable multifunction device 100 having a touch sensitive display 112 in accordance with some embodiments. The touch sensitive display 112 is conveniently referred to as a "touch screen" and may also be known or referred to as a touch sensitive display system. Device 100 may include memory 102 (which may include one or more computer readable storage media), memory controller 122, one or more processing units (CPUs) 120, peripheral interface 118, RF circuitry. 108, audio circuitry 110, speakers 111, microphones 113, input / output (I / O) subsystem 106, other input or output devices 116, and external ports 124. have. Device 100 may include one or more photosensors 164. These components may communicate via one or more communication buses or signal lines 103.

It should be appreciated that the device 100 is only one example of a portable multifunction device and that the device 100 may have more or fewer components than shown, couple two or more components, or have different component configurations or arrangements do. The various components shown in FIGS. 1A and 1B may be implemented in hardware, software, or a combination of both hardware and software, including one or more signal processing and / or application specific integrated circuits.

Memory 102 may include high-speed random access memory, and may also include one or more magnetic disk storage devices, flash memory devices, or other non-volatile solid-state memory devices. non-volatile memory such as device). Access to memory 102 by other components of device 100, such as CPU 120 and peripheral interface 118, may be controlled by memory controller 122.

Peripheral device interface 118 may be used to couple the input and output peripheral devices of the device to CPU 120 and memory 102. One or more processors 120 may execute various software programs and / or instruction sets stored in memory 102 to perform various functions and process data for device 100.

In some embodiments, peripheral device interface 118, CPU 120 and memory controller 122 may be implemented on a single chip, such as chip 104. In some other embodiments, they may be implemented on separate chips.

RF (radio frequency) circuit 108 transmits and receives RF signals, also known as electromagnetic signals. The RF circuit 108 converts electrical signals to electromagnetic signals or converts them from electromagnetic signals to electrical signals, and communicates with the communication network and other communication devices through electromagnetic signals. The RF circuitry 108 may include circuitry known in the art to perform such functions including but not limited to an antenna system, an RF transceiver, one or more amplifiers, a tuner, one or more oscillators, a digital signal processor, a CODEC chipset, Subscriber identity module (SIM) card, memory, and the like. RF circuitry 108 may be a network such as a cellular telephone network, a wireless local area network (LAN) and / or a wireless network such as a metropolitan area network (MAN), an intranet, and / or the Internet, also called the World Wide Web (WWW); It can communicate with other devices by wireless communication. Such wireless communications include Global System for Mobile Communications (GSM), Enhanced Data GSM Environment (EDGE), high-speed downlink packet access (HSDPA), wideband code division multiple access (W-CDMA), code division multiple access (CDMA) , Time division multiple access (TDMA), Bluetooth, Wi-Fi (Wireless Fidelity) (such as IEEE 802.11a, IEEE 802.11b and IEEE 802.11b), voice over Internet Protocol (VoIP) , Wi-MAX, e-mail protocols such as IMAP (Internet message access protocol) and POP (post office protocol), extensible messaging and presence protocol (XMPP), Session Initiation Protocol for Instant Messaging and Presence Leveraging Extensions), instant messaging (such as Instant Messaging and Presence Service (IMPS), and / or Short Message Service (SMS)), or any other suitable communication protocol including communication protocols not developed at the time of the filing of this application, Limited to That can be used is any of a plurality of communications standards, protocols and technologies.

The audio circuitry 110, the speaker 111 and the microphone 113 provide an audio interface between the user and the device 100. The audio circuit 110 receives audio data from the peripheral device interface 118, converts the audio data into an electrical signal, and transmits the electrical signal to the speaker 111. The speaker 111 converts the electrical signal into a human-audible sound wave. Audio circuitry 110 also receives electrical signals converted from sound waves by microphone 113. Audio circuitry 110 converts the electrical signal into audio data and transmits the audio data to peripherals interface 118 for processing. Audio data may be retrieved from or transmitted to memory 102 and / or RF circuitry 108 by peripherals interface 118. In some embodiments, the audio circuitry 110 further includes a headset jack (such as 212 in FIG. 2). The headset jack provides an interface between the audio circuitry 110 and a mobile audio input / output peripheral, such as a headset or output-only headphones that have both an output (such as one or both ears) and an input (such as a microphone). to provide.

I / O subsystem 106 couples input / output peripherals on device 100 such as touch screen 112 and other input / control devices 116 to peripheral interface 118. I / O subsystem 106 may include one or more input controllers 160 for display controller 156 and other input or control devices. One or more input controllers 160 may receive or transmit electrical signals from other input or control device 116. The other input / control device 116 may include physical buttons (such as push buttons, rocker buttons, etc.), dials, slider switches, joysticks, click wheels, and the like. In some alternative embodiments, the input controller (s) 160 may be associated with (or may not be coupled to) any of the pointer devices, such as a keyboard, infrared port, USB port, and mouse. One or more buttons (such as 208 of FIG. 2) may include up / down buttons for volume control of the speaker 111 and / or the microphone 113. One or more buttons may include push buttons (such as 206 of FIG. 2). A short press of a push button, as described in US Patent Application No. 11 / 322,549, "Unlocking a Device by Performing Gestures on an Unlock Image," filed December 23, 2005, incorporated herein by reference, The process of unlocking the screen 112 or unlocking the device using a gesture on the touch screen can be initiated. Press and hold a push button (such as 206) to turn the power on device 100 on or off. The user may be able to customize the functionality of one or more of the buttons. The touch screen 112 is used to implement virtual buttons or soft buttons and one or more soft keyboards.

The touch sensitive display 112 provides an input interface and an output interface between the device and the user. The display controller 156 may receive an electric signal from the touch screen 112, transmit an electric signal to the touch screen 112, or both transmit and receive. The touch screen 112 displays the visual output for the user. The visual output may include graphics, text, icons, video, and any combination thereof (collectively referred to as "graphics"). In some embodiments, some or all of the visual output may correspond to a user-interface object.

The touch screen 112 includes a touch sensitive surface, sensor, or sensor set that accepts input from a user based on haptic contact and / or tactile contact. The touch screen 112 and the display controller 156 may make a contact (and any movement or release of the contact) on the touch screen 112 (along with any associated module and / or instruction set in memory 102). Detects and converts the detected contacts into interaction with a user-interface object (such as one or more soft keys, icons, web pages or images) displayed on touch screen 112. In an exemplary embodiment, the point of contact between the touch screen 112 and the user corresponds to the user's finger.

The LCD 112 (liquid crystal display) technology or LPD (light emitting polymer display) technology or LED (light emitting diode) technology may be used for the touch screen 112, but other display technologies may be used in other embodiments. The touch screen 112 and the display controller 156 are already known or include, but are not limited to, capacitive, resistive, infrared and surface acoustic wave technologies. Using any of a plurality of touch sensing technologies to be developed at the same time, and using other proximity sensor arrays or other components for determining one or more contact points with the touch screen 112 Can detect movement or release. In an exemplary embodiment, projection mutual capacitance sensing techniques such as those found in iPhone® and iPod Touch® from Apple Inc. of Cupertino, Calif., Are used.

Touch sensitive displays in some embodiments of touch screen 112 are described in US Pat. Nos. 6,323,846 (Westerman et al.), 6,570,557 (Westerman et al.) And / or 6,677,932 (Westerman), and / or US Patent Publication No. 2002/0015024 A1. However, the touch screen 112 displays visual output from the portable device 100, while the touch sensitive touch pad does not provide a visual output.

A touch sensitive display in some embodiments of touch screen 112 is described in US patent application Ser. No. 11 / 381,313, "Multipoint Touch Surface Controller," filed May 6, 2006. U.S. Patent Application No. 10 / 840,862, "Multipoint Touchscreen", 미국 U.S. Patent Application 10 / 903,964, "Gestures For Touch Sensitive Input Devices", "U.S., filed January 31, 2005 Patent Application No. 11 / 048,264, "Gestures For Touch Sensitive Input Devices", 미국 US Patent Application No. 11 / 038,590, filed January 18, 2005, "Mode-Based Graphical User Interfaces For Touch Sensitive Input Devices," US Patent Application No. 11 / 228,758, filed September 16, 2005, "Virtual Input Device Placement On A Touch Screen User Interface," US Patent Application No. 11 / 228,700, filed September 16, 2005, "Operation. Of A Computer With A Touch Screen Interface ", 미국 United States Patent Application No. 11 / 228,737, filed September 16, 2005 Quot; Activating Virtual Keys Of A Touch-Screen Virtual Keyboard ", U.S. Patent Application No. 11 / 367,749, filed March 3, 2006, entitled " Multi-Functional Hand-Held Device ". All of the above applications are incorporated herein by reference in their entirety.

The touch screen 112 may have a resolution of over 100 dpi. In some embodiments, the touch screen can have a resolution of approximately 160 dpi. The user may contact the touch screen 112 using any suitable object or appendage, such as a stylus, a finger, or the like. In some embodiments, the user interface is designed primarily to operate using finger-based contacts and gestures, such that the finger-based contacts and gestures have a wider contact area of the finger on the touch screen compared to the stylus- Because it is less accurate. In some embodiments, the device translates the rough finger-based input into an exact pointer / cursor position or command for performing the action desired by the user.

In some embodiments, the device 100 may include a touchpad (not shown) for activating or deactivating certain functions in addition to the touch screen. In some embodiments, the touchpad is a touch sensitive area of the device that does not display visual output, unlike a touch screen. The touchpad may be a touch sensitive surface separate from the touch screen 112, or it may be an extension of the touch sensitive surface formed by the touch screen.

In some embodiments, the device 100 may include a physical or virtual wheel (e.g., a click wheel) as the input control device 116. The user can move the point of contact with the click wheel by rotating the click wheel (e.g., when the amount of movement of the touch point is measured by the angular displacement of the touch point relative to the center point of the click wheel) You can navigate and interact with objects (such as icons). The click wheel may also be used to select one or more of the displayed icons. For example, a user may press at least a portion of an associated button or click wheel. User commands and navigation commands provided by the user via the click wheel may be processed by the input controller 160 as well as by one or more of the instruction set and / or modules of the memory 102. In the case of a virtual click wheel, the click wheel and click wheel controllers may be part of the touch screen 112 and the display controller 156, respectively. In the case of a virtual click wheel, the click wheel may be an opaque or translucent object that appears and disappears on the touch screen display in response to user interaction with the device. In some embodiments, the virtual click wheel is displayed on the touch screen of the portable multifunction device and is operated by user contact with the touch screen.

Apparatus 100 also includes a power system 162 for powering various components. The power system 162 may include a power management system, one or more power sources (such as a battery, AC, etc.), a charging system, a power failure detection circuit, a power converter or inverter, And any other components associated with power generation, management and distribution of the portable device.

Device 100 may also include one or more photosensors 164. 1A and 1B show an optical sensor coupled to the optical sensor controller 158 of the I / O subsystem 106. The photosensor 164 may include a charge coupled device (CCD) or a complementary metal oxide semiconductor (CMOS) phototransistor. Optical sensor 164 receives light from the environment, which is projected through one or more lenses, and converts the light into data representing an image. Along with the imaging module 143 (also referred to as a camera module), the optical sensor 164 can capture still images or video. In some embodiments, an optical sensor is mounted on the device 100, opposite the touch screen display 112 on the front of the device, so that the touch screen display can be used as a viewfinder for either stop and / As shown in FIG. In some embodiments, an optical sensor is disposed in front of the device such that the user's image can be acquired for video conferencing while the user is viewing another video conferencing participant on the touch screen display. In some embodiments, when a user (e.g., rotates a lens and a sensor in the device housing) such that one optical sensor 164 can be used with the touch screen display for both video conferencing and still and / The position of the optical sensor 164 can be changed.

Device 100 may also include one or more proximity sensors 166. 1A and 1B show proximity sensor 166 coupled to peripherals interface 118. Alternatively, proximity sensor 166 may be coupled to input controller 160 of I / O subsystem 106. Proximity sensor 166 is described in U. S. Patent Application Serial No. 11 / 241,839, "Proximity Detector In Handheld Device ", which is incorporated herein by reference in its entirety; No. 11 / 240,788, "Proximity Detector In Handheld Device"; 11 / 602,702, "Using Ambient Light Sensor To Augment Proximity Sensor Output "; 11 / 586,862, “Automated Response To And Sensing Of User Activity In Portable Devices”; And 11 / 638,251, "Methods And Systems For Automatic Configuration Of Peripherals." In some embodiments, the proximity sensor turns off and disables the touch screen 112 when the multifunction device is placed adjacent to the user ' s ear (e.g., such as when the user is on a phone call).

Device 100 may also include one or more accelerometers 168. 1A and 1B show accelerometer 168 coupled to peripherals interface 118. Alternatively, the accelerometer 168 can be coupled to the input controller 160 of the I / O subsystem 106. Accelerometer 168 is described in U. S. Patent Publication No. 20050190059, "Acceleration-based Theft Detection System for Portable Electronic Devices ", and U.S. Patent Application Publication No. 20060017692," Methods and " Apparatuses < / RTI > For Operating A Portable Device Based On An Accelerometer ". In some embodiments, the information is displayed on the touch screen display in a portrait or landscape view based on an analysis of data received from the one or more accelerometers. The device 100 may optionally include, in addition to the accelerometer (s) 168, a jammer (not shown) and GPS (not shown) to obtain information about the location and orientation Or GLONASS or other global navigation system) receiver (not shown).

In some embodiments, the software components stored in the memory 102 may include an operating system 126, a communication module (or a set of instructions) 128, a contact / motion module (or a set of instructions) 130, (Or instruction set) 132, a text input module (or instruction set) 134, a GPS module (or instruction set) 135 and an application (or instruction set) Moreover, in some embodiments, memory 102 stores device / global internal state 157 as shown in FIGS. 1A, 1B, and 3. The device / global internal state 157 is an active application state that indicates which application is currently active when present; A display state that indicates which applications, views or other information occupy various areas of the touch screen display 112; A sensor state comprising information obtained from the various sensors of the device and the input control devices 116; And location information regarding the location and / or posture of the device.

The operating system 126 (such as an embedded operating system such as Darwin, RTXC, Linux, Unix, OS X, Windows, or VxWorks) controls and manages common system tasks (such as memory management, storage control, and power management). It includes various software components and / or drivers for the purpose of enabling communication between various hardware and software components.

The communication module 128 enables communication with other devices via one or more external ports 124, and various software components for processing data received by the RF circuit 108 and / or the external port 124. It also includes. External ports 124 (such as USB, Firewire, etc.) are adapted to couple directly to other devices or indirectly via a network (such as the Internet, wireless LAN, etc.). In some embodiments, the external port is a multi-pin connector that is identical, similar and / or compatible (e.g., 30 pins) to a 30-pin connector used on an iPod (trademark of Apple Computer, Inc.).

Contact / motion module 130 may detect contact with touch screen 112 (along with display controller 156) and with other touch sensitive devices (such as a touch pad or physical click wheel). The contact / movement module 130 may determine whether a contact has occurred (eg, detecting a finger down event), determine whether there is a touch movement, and track the movement along the touch screen (eg Various software components for performing various operations related to contact detection, such as detecting one or more finger dragging events) and determining whether the contact is interrupted (eg, detecting a finger up event or contact separation). It includes. The contact / motion module 130 receives contact data from the touch sensitive surface. The act of determining the movement of the contact point represented by the series of contact data may include determining the speed (magnitude), velocity (magnitude and direction) and / or acceleration (change in magnitude and / or direction) have. This action can be applied to a single contact (such as one finger contact) or to multiple simultaneous contacts (such as “multi-touch” / multiple finger contacts, etc.). In some embodiments, contact / motion module 130 and display controller 156 also detect contact on the touchpad. In some embodiments, contact / motion module 130 and controller 160 detect contact on the click wheel.

Contact / motion module 130 may detect a gesture input by a user. Different gestures on the touch sensitive surface have different contact patterns. Therefore, the gesture can be detected by detecting a specific contact pattern. For example, the detection of a finger tap gesture may be a finger up (lift off) event at the same position (or substantially the same position) as a finger down event (e.g., at the location of the icon) . As another example, detection of a finger swipe gesture on a touch sensitive surface includes detecting one or more finger dragging events after detecting a finger down event, and then detecting a finger up (lift off) event.

Graphics module 132 includes various known software components for rendering and displaying graphics on touch screen 112 or other displays, including components for changing the brightness of displayed graphics. The term "graphics" as used herein refers to a user, including but not limited to text, web pages, icons (such as user-interface objects, including soft keys), digital images, video and animations, and the like. It includes any object that can be displayed.

In some embodiments, graphics module 132 stores data representing graphics to be used. Each graphic may be assigned a corresponding code. Graphics module 132 generates screen image data to be output to display controller 156 after receiving one or more codes from an application or the like that specify graphics to be displayed with coordinate data and other graphical property data if necessary.

The text input module 134, which may be a component of the graphics module 132, may be any of a variety of applications, including (contact 137, email 140, IM 141, browser 147, ) Provides a soft keyboard for entering text in various applications.

The GPS module 135 determines the location of the device and provides this information for use in various applications (e.g., providing it to the telephone 138 for use in location-based dialing and using the camera 143 as the picture / ) And provides to applications providing location-based services such as weather widgets, local phone book widgets, and map / navigation widgets).

The application 136 may include the following modules (or sets of instructions), or a subset or superset thereof.

A contact module 137 (mainly referred to as an address book or a contact list);

A telephone module 138;

Video conferencing module 139;

An email client module 140;

An instant messaging (IM) module 141;

An exercise support module 142;

Camera module (143) for stationary and / or video images;

Image management module 144;

Video player module 145;

Music player module 146;

Browser module 147;

A calendar module 148;

● Weather Widget (149-1), Stock Widget (149-2), Calculator Widget (149-3), Alarm Clock Widget (149-4), Dictionary Widget (149-5) A widget module 149 that may include one or more of the user-generated widgets 149-6;

A widget generator module 150 for creating a user-created widget 149-6;

A retrieval module 151;

A video and music player module 152 integrating the video player module 145 and the music player module 146;

A note module 153;

Map module 154; And / or

● Online video module (155).

Other image processing applications such as other word processing applications, other image editing applications, drafting applications, presentation applications, JAVA-enabled applications, encryption, digital rights management, voice recognition recognition and voice replication.

The contact module 137, along with the touch screen 112, the display controller 156, the contact module 130, the graphics module 132, and the text input module 134, contacts or contacts list (eg, memory 102). Or stored in the application internal state 192 of the contact module 137 in memory 370, which adds a name to the address book, deletes the name (s) from the address book, and calls. Associate number (s), email address (es), physical address (es) or other information with a name, associate a name with an image, classify and sort names, telephone 138, video conferencing 139, Providing a phone number or email address for initiating and / or enabling communication by email 140 or IM 141, and the like.

The telephone module 138 includes an RF circuit 108, an audio circuit 110, a speaker 111, a microphone 113, a touch screen 112, a display controller 156, a contact module 130, and a graphics module 132. And a text input module 134 to enter a string corresponding to the telephone number, access one or more telephone numbers of the address book 137, modify the entered telephone number, dial each telephone number, It can be used to conduct a conversation, disconnect or hang up when the conversation is complete. As already mentioned, any of a plurality of communication standards, protocols and techniques can be used for wireless communication.

The video conferencing module 139 includes an RF circuit 108, an audio circuit 110, a speaker 111, a microphone 113, a touch screen 112, a display controller 156, an optical sensor 164, Along with a user interface, along with a user interface 158, a contact module 130, a graphics module 132, a text input module 134, a contact list 137, and a phone module 138, And includes executable instructions to initiate, perform, and terminate the conference.

The email client module 140 may be coupled to the web server 110 in response to a user command with an RF circuit 108, a touch screen 112, a display controller 156, a contact module 130, a graphics module 132 and a text input module 134 And executable instructions for creating, sending, receiving, and managing email. The email client module 140 makes it very easy to create and send an email containing still or video images acquired with the camera module 143 in conjunction with the image management module 144.

The instant messaging module 141 is capable of handling instant messages with the RF circuitry 108, the touch screen 112, the display controller 156, the contact module 130, the graphics module 132 and the text input module 134 (E.g., using an SMS or MMS (Multimedia Message Service) protocol for phone-based instant messaging, XMPP, SIMPLE or IMPS for Internet-based instant messaging) Send instant messages, receive instant messages, and view executable instant messages. In some embodiments, the instant messages that are sent and / or received may include graphics, pictures, audio files, video files, and / or other attachments supported by MMS and / or Enhanced Messaging Service (EMS). As used herein, "instant messaging" refers to both a phone-based message (such as a message sent using SMS or MMS) and an Internet-based message (such as a message sent using XMPP, SIMPLE or IMPS) It refers to all.

The exercise support module 142 may include an RF circuit 108, a touch screen 112, a display controller 156, a contact module 130, a graphics module 132, a text input module 134, a GPS module 135, Create exercises (eg, with time, distance, and / or calorie burning targets) in conjunction with the map module 154 and the music player module 146; Communicate with practice sensors (sports devices); Receiving practice sensor data; Adjusting sensors used to monitor the practice; Select and play music for practice; Includes executable instructions for displaying, storing, and sending exercise data.

The camera module 143 includes a touch screen 112, a display controller 156, an optical sensor (s) 164, an optical sensor controller 158, a contact module 130, a graphics module 132, 144 to capture still images or videos (including video streams) and store them in memory 102, modify the properties of still images or video, or delete still images or video from memory 102 ≪ / RTI >

The image management module 144 may include a touch screen 112, a display controller 156, a touch module 130, a graphics module 132, a text input module 134 and a camera module 143, Includes executable instructions for arranging, modifying (e.g., editing) or manipulating, labeling, deleting (e.g., in digital slide show or album form) and arranging video images.

The video player module 145 may be coupled to a touch screen 112 such as a touch screen 112, a display controller 156, a touch module 130, a graphics module 132, an audio circuit 110 and a speaker 111 112), or on an external display connected via an external port 124) for displaying and displaying video.

The music player module 146 includes a touch screen 112, a display system controller 156, a contact module 130, a graphics module 132, an audio circuit 110, a speaker 111, an RF circuit 108 and a browser Along with module 147, executable instructions for enabling a user to download and play back recorded music and other sound files stored in one or more file formats, such as MP3 or AAC files. In some embodiments, the device 100 may include functionality of an MP3 player such as the iPod (trademark of Apple Computer, Inc.).

The browser module 147 may include a web page or a web page, along with the RF circuitry 108, the touch screen 112, the display system controller 156, the contact module 130, the graphics module 132, Including executable instructions for browsing the Internet in accordance with a user command, including searching for, linking to, receiving, and displaying attachments and other files that are part of and linked to a web page.

The calendar module 148 includes an RF circuit 108, a touch screen 112, a display system controller 156, a touch module 130, a graphics module 132, a text input module 134, an email client module 140, And a browser module 147, executable instructions for generating, displaying, and storing data (calendar entries, to-do lists, etc.) associated with calendars and calendars in accordance with user commands.

The widget module 149 includes an RF circuit 108, a touch screen 112, a display system controller 156, a contact module 130, a graphics module 132, a text input module 134, and a browser module 147. Together, downloaded by the user (such as weather widget 149-1, stock widget 149-2, calculator widget 149-3, alarm clock widget 149-4, and dictionary widget 149-5). It is a mini-application that can be used and used or can be generated by a user (such as user-generated widget 149-6). In some embodiments, the widget includes an HTML file, a CSS file, and a JavaScript file. In some embodiments, the widget (e.g., a Yahoo! widget) includes an XML file and a JavaScript file.

Widget generator module 150 includes RF circuitry 108, touch screen 112, display system controller 156, contact module 130, graphics module 132, text input module 134, and browser module 147. Together with the widget (eg, replacing a user-specified portion of the web page with a widget).

The search module 151 may be implemented in accordance with a user command in conjunction with a touch screen 112, a display system controller 156, a touch module 130, a graphics module 132 and a text input module 134 Music, sound, images, video, and / or other files in the memory 102 that match one or more search conditions (e.g., search terms).

The note module 153 generates a note, a to-do list, and the like in accordance with a user command together with the touch screen 112, the display controller 156, the contact module 130, the graphic module 132 and the text input module 134 And executable instructions for managing and managing.

The map module 154 includes an RF circuit 108, a touch screen 112, a display system controller 156, a touch module 130, a graphics module 132, a text input module 134, a GPS module 135, Along with the browser module 147, receives and displays map-related data and maps (e.g., in the driving direction, data for a shop or other nearby point of interest or other relevant point of interest, and other location-based data) And can be used to modify and store.

The online video module 155 includes a touch screen 112, a display system controller 156, a contact module 130, a graphics module 132, an audio circuit 110, a speaker 111, an RF circuit 108, Along with input module 134, email client module 140 and browser module 147, allows a user to access, browse (e.g., through streaming and / or downloading) emails with a link to a particular online video (E.g., on an external display connected via a touch screen or external port 124), or to manage online video in one or more file formats, such as H.264, . In some embodiments, the instant messaging module 141, rather than the email client module 140, is used to send a link to a particular online video. Additional descriptions of on-line video applications may be found in U.S. Provisional Patent Application Serial No. 60 / 936,562 entitled "Portable Multifunction Device, Method, and Graphical User Interface " filed on June 20, 2007, the entire contents of which are incorporated herein by reference. for Playing Online Videos ", and U.S. Patent Application No. 11 / 968,067, filed December 31, 2007, entitled " Portable Multifunction Device, Method, and Graphical User Interface for Playing Online Videos. "

Each of the above-referenced modules and applications includes a set of executable instructions for performing one or more of the functions described hereinabove and the methods described herein (e.g., computer implemented methods and other information processing methods described herein) . These modules (i.e., instruction sets) need not be implemented as discrete software programs, procedures, or modules, and various subsets of such modules may be integrated or relocated in various embodiments. For example, video player module 145 may be combined with music player module 146 into a single module (eg, such as video and music player module 152 of FIG. 1B). In some embodiments, the memory 102 may store a subset of the above-mentioned modules and data structures. Furthermore, memory 102 may store additional modules and data structures not mentioned so far.

In some embodiments, the device 100 is a device on which a predetermined set of functions on the device is performed only through a touch screen and / or a touch pad. By using the touch screen and / or touchpad as a basic input / control device for the operation of the device 100, the number of physical input / control devices (such as push buttons, dials) on the device 100 can be reduced. .

The predetermined set of functions that can be performed only through the touch screen and / or the touch pad include navigation between the user interfaces. In some embodiments, the touchpad navigates the device 100 from a main menu, a home menu, or a root menu from any user interface that can be displayed on the device 100 when the user touches it. In such embodiments, the touchpad may be referred to as a "menu button." In some other embodiments, the menu button may be a physical push button or other physical input / control device instead of the touchpad.

1C is a block diagram illustrating exemplary components for event processing in accordance with some embodiments. In some embodiments, the memory 102 (FIG. 1A) and the memory 370 (FIG. 3) (of FIG. 1A and 1B) may include an event classifier 170 (eg, within the operating system 126) -1) (e.g., any of the applications 137-151, 155, 380-390 described above).

The event classifier 170 receives the event information and determines the application 136-1 to which to send the event information and the application view 191 of the application 136-1. The event classifier 170 includes an event monitor 171 and an event dispatcher module 174. In some embodiments, the application 136-1 includes an application internal state 192 that indicates the current application view (s) displayed on the touch sensitive display 112 when the application is active or running. In some embodiments, the event classifier 170 uses the device / global internal state 157 to determine which application (s) are currently active and the event classifier 170 uses the application internal state 192 To determine application views 191 to which to send event information.

In some embodiments, the application internal state 192 may include resume information to be used when the application 136-1 resumes execution, a user interface < RTI ID = 0.0 > State information, a state queue for allowing the user to return to the previous state or view of the application 136-1, and a re-execute / cancel queue of previous actions taken by the user.

The event monitor 171 receives event information from the peripheral device interface 118. The event information includes information about a sub-event (e.g., a user touch on the touch-sensitive display 112 that is part of a multi-touch gesture). The peripheral device interface 118 may be coupled to an I / O subsystem 106 such as proximity sensor 166, accelerometer (s) 168 and / or microphone 113 (via audio circuitry 110) Information. The information that the peripheral device interface 118 receives from the I / O subsystem 106 includes information from the touch-sensitive display 112 or the touch-sensitive surface.

In some embodiments, the event monitor 171 sends requests to the peripheral device interface 118 at predetermined intervals. In response, the peripheral device interface 118 transmits event information. In other embodiments, peripheral interface 118 transmits event information only when there is a significant event (eg, receiving input over a predetermined noise threshold and / or longer than a predetermined duration). .

In some embodiments, the event classifier 170 also includes a hit view determination module 172 and / or an active event recognizer determination module 173.

The hit view determination module 172 provides software procedures for determining where a sub-event occurred within one or more views when the touch-sensitive display 112 displays more than one view. Views consist of controls and other elements that a user can see on the display.

Another aspect of a user interface associated with an application is a set of views, sometimes referred to herein as application views or user interface windows, in which information is displayed and touch-based gestures occur. The application views of the touch (each application) may correspond to the program levels in the program or view hierarchy of the application. For example, a lowest level view at which a touch is detected may be referred to as a hit view, and a set of events recognized as appropriate inputs may be determined based at least in part on a hit view of an initial touch starting a touch-based gesture.

The hit view determination module 172 receives information related to sub-events of the touch-based gesture. When the application has multiple views configured in a hierarchical structure, the hit view determination module 172 identifies the hit view as the lowest view in the hierarchy that should handle the sub-event. In most situations, the hit view is the lowest level view at which an initiation sub event (i.e., the first sub event in the sequence of sub events that form the event or potential event) occurs. If a hit view is identified by the hit view determination module, the hit view typically receives all the sub-events associated with the same touch or input source that he identified as a hit view.

The active event recognizer determination module 173 determines which views or views in the view hierarchy should receive a particular sequence of sub events. In some embodiments, the active event recognizer determination module 173 determines that only the hit view should receive a particular sequence of sub-events. In other embodiments, the active event recognizer determination module 173 determines that all views including the physical location of the sub-event are validly related views, so that all validly related views must receive a specific sequence of sub-events . In other embodiments, even if touch sub-events are completely restricted to the area associated with one particular view, the top views in the hierarchy will still remain validly related views.

The event dispatcher module 174 dispatches the event information to an event recognizer (e.g., event recognizer 180). In embodiments including the active event recognizer determination module 173, the event dispatcher module 174 sends the event information to the event recognizer determined by the active event recognizer determination module 173. In some embodiments, event dispatcher module 174 stores event information retrieved by each event receiver module 182 in an event queue.

In some embodiments, operating system 126 includes event classifier 170. Alternatively, application 136-1 includes event classifier 170. In yet other embodiments, the event classifier 170 is an independent module or is part of another module stored in the memory 102, such as the contact / motion module 130.

In some embodiments, application 136-1 includes one or more application views 191, each of which includes instructions for handling touch events occurring within each view of an application's user interface. . Each application view 191 of the application 136-1 includes one or more event recognizers 180 and a plurality of event handlers 190. Typically, each application view 191 includes a plurality of event recognizers 180 and a plurality of event handlers 190. In other embodiments, one or more of the event recognizers 180 are part of a separate module, such as a user interface kit (not shown) or a higher level object to which the application 136-1 inherits methods and other properties . In some embodiments, each application view 191 may also include event data received from data updater 176, object updater 177, GUI updater 178, and / or event classifier 170. 179).

Each event recognizer 180 receives event information (e.g., event data 179) from the event classifier 170 and identifies the event from the event information. The event recognizer 180 includes an event receiver 182 and an event comparator 184. In some embodiments, event recognizer 180 also includes at least a subset of metadata 183 and event transfer instructions 188 (which may include sub event transmission instructions).

The event receiver 182 receives event information from the event classifier 170. The event information includes information about a sub event, e.g., touch or touch movement. Depending on the sub event, the event information also includes additional information such as the position of the sub event. When the sub event is associated with the movement of the touch, the event information may also include the speed and direction of the sub event. In some embodiments, events include rotation of the device from one orientation to another (e.g., from portrait orientation to landscape orientation, or vice versa), and the event information is stored in the device's device And corresponding information on the current orientation.

The event comparator 184 compares the event information with the predefined event or sub event definitions, determines an event or sub event based on the comparison, or determines or updates the state of the event or sub event. In some embodiments, event comparator 184 includes event definitions 186. Event definitions 186 include definitions of events (e.g., sequences of predefined sub-events), such as Event 1 187-1, Event 2 187-2, and the like. In some embodiments, the sub-events in event 187 include, for example, touch start, touch end, touch motion, touch cancel, and multiple touches. In one example, the definition for event 1 187-1 is a double tap on the displayed object. The double tap may include, for example, a first touch (touch start) on the display object during a predetermined step, a first lift-off (touch end) during a predetermined step, a second touch on the display object Start) and a second lift-off (touch termination) during a predetermined step. In another example, the definition for event two 187-2 is dragging on the display object. Dragging includes, for example, touch (or touch) on the display object during a predetermined phase, movement of the touch across the touch sensitive display 112, and lift off (touch end) of the touch. In some embodiments, the event also includes information about the associated event handlers 190 of the event.

In some embodiments, event definition 187 includes a definition of an event for each user interface object. In some embodiments, event comparator 184 performs a hit test to determine which user interface object is associated with the sub-event. For example, in an application view where three user interface objects are displayed on the touch-sensitive display 112, when a touch is detected on the touch-sensitive display 112, the event comparator 184 performs a hit test, Determines which of the user interface objects is associated with the touch (sub event). When each display object is associated with a respective handler event 190, the event comparator uses the result of the hit test to determine which event handler 190 should be activated. For example, the event comparator 184 selects an event handler associated with the sub-event and the object that trigger the hit test.

In some embodiments, the definition for each event 187 also includes delayed actions that delay the transmission of event information until after it is determined whether the sequence of sub-events corresponds to the event type of the event recognizer.

When each event recognizer 180 determines that the series of subevents does not match any of the events in the event definitions 186, each event recognizer 180 enters an event disable, or event cancel state. Then ignore subsequent subevents of the touch-based gesture. In this situation, if present, other event recognizers that remain active for the hit view continue to track and process sub-events of the ongoing touch-based gesture.

In some embodiments, each event recognizer 180 may include a meta-tag with configurable properties, flags, and / or lists that indicate how the event transmission system should effectively perform sub event transmission to the associated event recognizers. Data 183. In some embodiments, the metadata 183 includes configurable properties, flags, and / or lists that indicate how the event recognizers can interact with each other. In some embodiments, metadata 183 includes configurable properties, flags, and / or lists that indicate whether sub-events are to be transmitted at various levels within the view or program hierarchy.

In some embodiments, each event recognizer 180 activates an event handler 190 associated with the event when one or more specific sub-events of the event are recognized. In some embodiments, each event recognizer 180 sends event information to the event handler 190 in association with the event. The activation of the event handler 190 is different from the transmission (and default transmission) of each of the sub-events to the hit view. In some embodiments, the event recognizer 180 casts a flag associated with the recognized event, and the event handler 190 associated with the flag receives the flag to perform the predefined process.

In some embodiments, the event transfer instructions 188 include sub-event transfer instructions that transmit event information for a sub-event without activating the event handler. Instead, sub-event transfer instructions send event information to event handlers associated with views associated with or effectively related to a series of sub-events. Event handlers associated with a series of sub-events or validly related views receive event information and perform predetermined processes.

In some embodiments, data updater 176 generates and updates data used in application 136-1. For example, the data updater 176 updates the telephone number used in the contact module 137 or stores the video file used in the video player module 145. [ In some embodiments, object updater 177 creates and updates objects used in application 136-1. For example, object updater 176 creates a new user interface object or updates the location of the user interface object. GUI updater 178 updates the GUI. For example, the GUI updater 178 prepares the display information and transmits it to the graphics module 132 for display on the touch-sensitive display.

In some embodiments, the event handler (s) 190 include, or have access to, a data updater 176, an object updater 177, and a GUI updater 178. In some embodiments, data updater 176, object updater 177 and GUI updater 178 are included in a single module of each application 136-1 or application view 191. [ In other embodiments, these are included in two or more software modules.

It should be appreciated that the above discussion of event handling of user touches on touch-sensitive displays also applies to other types of user inputs for manipulating multifunction devices 100 with input devices, But are not limited to, for example, adjusting mouse movements and mouse button presses in the presence or absence of a single or multiple keyboard push or hold, user movement tabs on the touch pads, dragging, Pen stylus input, device motion, verbal commands, eye movement detection, biometric input, and / or combinations thereof, and may be used as inputs corresponding to sub-events defining the event to be recognized.

Figure 2 illustrates a portable multifunction device 100 having a touch screen 112 according to some embodiments. The touch screen may display one or more graphics within the user interface (UI) In this and other embodiments described below, a user may use one or more fingers 202 (not shown in scale in the figure) or one or more stylus 203 (not shown in scale in the figure) You can select one or more of the graphics by touching or touching the graphic. In some embodiments, a selection is made for the one or more graphics when the user releases the contact with the one or more graphics. In some embodiments, the contact may be rolled (from left to right, from right to left, up and down), from left to right (right to left, up and / Gestures such as one or more swipes and / or one or more taps (e.g., downwardly / downwardly). In some embodiments, the graphic may not be selected by accidental contact with the graphic. For example, when the gesture corresponding to the selection is a tap, the corresponding application may not be selected by the swipe gesture passing on the application icon.

Device 100 may also include one or more physical buttons, such as a “home” or menu button 204. The menu button 204 may be used to navigate to any of the application sets 136 that may be executed on the device 100. [ Or, in some embodiments, the menu button is implemented as a soft key of the GUI of the touch screen 112. [

In one embodiment, the device 100 includes a touch screen 112, a menu button 204, a push button 206 for turning the device on and off and locking the device, a volume adjustment button 208, a subscriber identification module. (SIM) card slot 210, headset jack 212 and external port 124 for docking / charging. Push button 206 includes: turning on / off the power on the device by pressing the button to hold the button pressed for a predetermined time; Device locking by pressing the button to release the button before a predetermined time has elapsed; And / or to initiate an unlock process or to unlock a device. In another embodiment, the device 100 may also accept verbal input for activation or deactivation of some functions via the microphone 113.

3 is a block diagram of an exemplary multifunction device having a display and a touch sensitive surface in accordance with some embodiments. The device 300 need not be portable. In some embodiments, the device 300 may be a laptop computer, a desktop computer, a tablet computer, a multimedia player device, a navigation device, an educational device (such as a child's learning toy), a gaming system or a control device to be. Typically, the device 300 includes one or more processing units (CPUs) 310, one or more network or other communication interfaces 360, a memory 370, and one or more communication buses 320 for interconnecting these components. . Communication busses 320 may include circuitry (sometimes referred to as a chipset) that interconnects system components and controls communication therebetween. The device 300 includes an input / output (I / O) interface 330 that includes a display 340 that is typically a touch screen display. The I / O interface 330 may also include a keyboard and / or mouse (or other pointing device) 350 and a touchpad 355. The memory 370 includes high speed random access memory, such as DRAM, SRAM, DDR RAM or other random access semiconductor memory devices, and may be coupled to one or more magnetic disk storage devices, optical disk storage devices, flash memory devices, or other non- Non-volatile memory such as devices. The memory 370 may optionally include one or more storage devices remotely located from the CPU (s) In some embodiments, memory 370 includes programs, modules and data structures similar to programs, modules and data structures stored in memory 102 of portable multifunctional device 100 (FIG. 1) ≪ / RTI > Moreover, the memory 370 may store additional programs, modules and data structures that are not present in the memory 102 of the portable multifunction device 100. For example, the memory 370 of the apparatus 300 may include the drafting module 380, the presentation module 382, the word processing module 384, the website generation module 386, the disk authoring module 388, and / or While spreadsheet module 390 may be stored, memory 102 of portable multifunction device 100 (FIG. 1) may not store such modules.

Each of the foregoing elements of FIG. 3 may be stored in one or more of the memory devices described above. Each of the modules described above corresponds to a set of instructions for performing the functions described above. The above-described modules or programs (ie, instruction sets) need not be implemented as individual software programs, procedures or modules, and accordingly various embodiments of these modules may be combined or rearranged in various embodiments. have. In some embodiments, memory 370 may store a subset of the modules and data structures described above. Furthermore, the memory 370 may store the above described additional modules and data structures.

Hereinafter, an embodiment of a user interface (UI) that can be implemented on the portable multifunctional device 100 will be described.

4A and 4B illustrate an exemplary user interface for an application menu on a portable multifunction device 100 according to some embodiments. Similar user interfaces may be implemented on the device 300. In some embodiments, the user interface 400A may include the following elements, or a subset or superset thereof.

Signal strength indicator 402 for wireless communication, such as cellular signals and Wi-Fi signals;

Time 404;

A Bluetooth indicator 405;

A battery status indicator 406;

A tray 408 containing icons for frequently used applications, such as the following applications; And

○ a phone 138, which may include an indicator 414 for the number of missed calls or voicemail messages,

○ an email client 140 that may include an indicator 410 for the number of unread emails,

○ browser 147, and

○ music player (146)

Icons for other applications, such as the following applications:

○ IM (141),

○ image management (144),

○ camera (143),

○ video player 145,

○ weather (149-1),

○ shares (149-2),

○ Practice support (142),

○ calendar (148),

○ Calculator (149-3),

○ alarm clock (149-4),

○ dictionary (149-5), and

○ User-generated widget (149-6).

In some embodiments, the user interface 400B may include the following elements, or a subset or superset thereof.

[0215] The components 402, 404, 405, 406, 141, 148, 144, 143, 149-3, 149-2, 149-1, 149-4, 410, 414, 138, 140, 147);

A map 154;

Notes 153;

A setting 412 that provides access to the settings for the device 100 and its various applications 136, as described in detail later;

A video and music player module 152, also referred to as an iPod module 152 (a trademark of Apple Computer, Inc.); And

An online video module 155 (also referred to as YouTube module 155) (trademark of Google Inc.).

In the description provided below, the term "contact" is used as a shorthand term for "touch gesture" (except when used to describe an entry in a contact list, address book, etc.) and is therefore referred to or described below. Each contact may be any suitable touch gesture detected by a sensor (or set of sensors) on a touch sensitive display or other touch sensitive surface. Furthermore, in some embodiments, the "touch gestures" include not only gestures made by one or more fingers or one or more stylus that physically touch the touch sensitive screen 112 or another touch sensitive surface, ) Or gestures that occur totally or partially near other touch sensitive surfaces, and one or more sensors of the touch sensitive screen 112 or other touch sensitive surface may sense such gestures.

The term "conditionally enabled" button means a button, key or object displayed on a soft keyboard or user interface that can be activated when enabled and cannot be activated when disabled. The term "conditionally enabled button" is logically equivalent to "conditionally disabled button" because it is conditional whether the button is enabled or disabled.

Activating a soft button or other user interface object in response to a touch gesture initiated by a user at a location associated with the soft button or object may cause a touchdown event, liftoff event, or any other detection associated with the touch gesture initiated by the user. Can occur at possible events. The term "activation" means an event that activates a button or a function associated with a button. For example, activation of a key on a soft keyboard may include adding a character corresponding to the key to a text buffer, or shifting the text input mode from lowercase to uppercase, one or more keys on the keyboard representing alternate symbols or functions. An event that performs keyboard functions, such as shifting, replacing the current soft keyboard with each replacement keyboard, and so on.

Attention is now directed to embodiments of a user interface (“UI”) and associated processes that can be implemented on a multifunction device having a display and a touch-sensitive surface, such as device 300 or portable multifunction device 100.

5A-5I illustrate example interfaces for conditionally activating buttons or other displayed objects in accordance with some embodiments. The interfaces in these figures are used to illustrate the processes described below, including the processes in FIGS. 6A, 6B, and 7. Activating a button or object refers to performing an action or function corresponding to the button or object; In some cases, the action or function performed depends on both the identity of the button or object being activated and the circumstances in which the button or object is activated. An incomplete set of examples of actions performed in response to the activation of each button or object include: adding a symbol (corresponding to the active button or object) to the document or input text field, adding the displayed keyboard to the active button from the currently displayed keyboard. Or changing to a keyboard corresponding to the object, initiating or resuming execution, or resuming a function of the display or activated button or object of the application. As described below, in some cases, activation of a button or object may include activation of a primary function (eg, inserting a letter or symbol into an input text field) and auxiliary function (eg, activation of one or more buttons or objects displayed in a user interface. To temporarily enable or disable, or to temporarily enable or disable the performance of a function related to the primary function.

In FIGS. 5A-5I, some finger contact movement magnitudes may be exaggerated for illustrative purposes. The description of figure bearings regarding finger contact movements should not be taken as a requirement or limitation for understanding the size and ratios associated with the methods and apparatuses described herein.

UI 500A (FIG. 5A) shows an exemplary text input user interface (eg, note module 153, email client module 140, etc.) on touch screen 112 of device 100. In this example, the UI 500A includes a text input area 502 and a QWERTY soft keyboard 504. Text input area 502 can include input text 506 with text 506-1 and cursor 508 at location 508-A. In some embodiments, soft keyboard 504 is associated with one or more languages. Soft keyboard 504 includes one or more soft buttons (or “soft keys”) 510. Some of the buttons 510 are associated with each letter. For example, the button 510-A is letter "Q", the button 510-B is letter "W", the button 510-C is letter "E", and the button 510-E is Associated with a new line or line break character. Some others of the buttons 510 are associated with specific text input functions or operations. For example, the button 510 -D is associated with a delete / backspace operation and the button 510 -F is associated with a shift function (eg, to change the text input of other buttons 510).

The keyboard 504 also includes soft buttons (or “soft keys”) 512 related to the operation of the keyboard. For example, buttons 512 may be used to switch from QWERTY keyboard 504 to a keyboard of numbers and symbols, 512-A, from keyboard 504 to a keyboard associated with another language, or to keyboard 504. Buttons 512-B for switching to different languages for, and buttons 512-C for hiding the keyboard 504. Thus, the buttons 512 in this example are mode change buttons, which change the operating mode of the device by changing the displayed soft keyboard or removing the keyboard from the display.

UI 500H (FIG. 5H) shows another exemplary text input user interface (eg, for note module 153, email client module 140, etc.) on touch screen 112 of device 100. UI 500H includes a text input area 502 and a numeric and symbol soft keyboard 520. Text input area 502 can include input text 506 with text 506-1 and cursor 508 at location 508-A. In some embodiments, keyboard 520 is associated with one or more languages. Soft keyboard 520 includes one or more soft buttons (or “soft keys”) 522. Some of the buttons 522 are associated with respective numbers or symbols. For example, button 522-A is associated with a number "1", button 522-B with a number "2", and button 522-C is associated with a question mark ("?"). Some of the other buttons 522 are associated with specific text input functions or operations. For example, the buttons 522-D, 522-E, and 522-F have the same function as the buttons 510-D, 510-E, and 510-F, respectively. The keyboard 522 also includes buttons 512-B and 512-C as well as buttons 512-D for switching to the alphabet / letter keyboard.

In some embodiments, buttons 510 or 522 are buttons that are unconditionally enabled and buttons 512 are buttons that are conditionally enabled. Conditionally enabled buttons 512 have certain conditions upon their activation, and unconditionally enabled buttons 510 or 522 do not condition those conditions. Or, in some embodiments, buttons 512 are disabled under certain conditions, and buttons 510 or 522 are not disabled under certain conditions. One example of a conditionally enabled button is a keyboard select button for activating a second keyboard (eg, a keyboard having numeric keys and other symbols, at least some of which are provided by the first keyboard including a keyboard select button). In this example, the keyboard select button is conditionally enabled because it is temporarily disabled in response to the activation of any other soft keyboard buttons (eg, any of the letter buttons in the displayed soft keyboard). Otherwise, it is enabled for activation.

In some embodiments, the condition for activation of each conditionally enabled button 512 is such that contact at a location corresponding to each button 512 corresponds to another button on the keyboard 504 or 520. Whether it is detected within a predefined time period (i.e., a time window of a predefined amount of time) starting from the detection of a contact at a location to be made, and activation of button 512 is disabled during that time window. In some embodiments, the time window starts with contact detection on any button on keyboard 504 or 520 that includes buttons 510, 522, 512. In some other embodiments, the time window starts with contact detection on button 510 or 520, but not on button 512. As described below, the time window has a predefined duration, i.e., lasts for a predefined amount of time, which amount is fixed (i.e. invariant) in some embodiments and variable in other embodiments, or It is situation dependent.

UI 500B (FIG. 5B) shows contact 514 to touch screen 112. In this example, contact 514 is at location 514-A on touch screen 112, which corresponds to the location of button 510-C and also corresponds to the active area of button 510-C. In response to the detection of contact 514 at position 514-A, the letter “e” 516 is inserted into the input text 506 and changes the text 506-1 to the text 506-2. And advance the cursor 508 to position 508-B.

Detection of contact 514 at location 514-A corresponding to the location of button 510-C may also be performed by one or more conditionally enabled buttons (eg, button 512) of keyboard 504. Initiates a predefined amount of time window in which detected contacts at locations on the touch screen 112 corresponding to the locations are ignored, and the buttons are disabled during that time window. In some embodiments, a predefined duration of the time window, for example, having a duration between 200 milliseconds and 500 milliseconds, is fixed (ie, invariant), and in another example, a predefined duration of the time window. The duration is fixed at a value between 150 milliseconds and 700 milliseconds, and in another example, the predefined duration of the time window is fixed at a value between 300 milliseconds and 750 milliseconds. In other embodiments, the length of the time window depends on heuristics, user history or other relevant contextual information. In one example, the device measures a user average delay between key strokes or between keystrokes in a word, compares the value with a predefined average key stroke delay value, and between the user average key stroke delay and a predefined average. The length of the time window is adjusted according to the difference (or a ratio thereof). In another example, the time window length for one set of applications (eg, note and email application) is different than for another application (or set of applications) (eg, browser application). Because keyboard mode changes are suppressed during the time window in accordance with some embodiments, the time window is sometimes called the mode change suppression time window.

UI 500C (FIG. 5C) shows contact 518 on touch screen 112 immediately following contact 514. UI 500C shows three possible locations (among many possible locations) on keyboard 504 where contact 518 can be made and detected. In this example, contact 518 may be detected at location 518 -A, 518 -B, or 518 -C on touch screen 112. The subsequent figures show the results of the contact 518 detected at each position 518 -A, 518 -B, or 518 -C following the contact 514.

UI 500D (FIG. 5D) shows contact 518 on touch screen 112 immediately following contact 514. In this example, contact 518 is at location 518-A on touch screen 112, which corresponds to the location of button 510 -B and also corresponds to the active area of button 510 -B. In response to detecting contact 518 at position 518-A, a letter “w” 520 is inserted into input text 506 to change text 506-2 to text 506-3 and Cursor 508 moves to position 508 -C. The button 510 -B is an unconditionally enabled button that is not affected by the time window from the contact 514, and character insertion and cursor movement are detected by the contact 518 detected at position 518-A. It occurs depending on whether it occurs inside or outside the time window from contact 514.

In addition, detecting contact 518 at position 518-A starts a new time window with a predefined amount of time. In fact, the time window is shifted or otherwise changed to begin with the detection of contact 518 or adjust accordingly.

UI 500E (FIG. 5E) shows contact 518 on touch screen 112 immediately following contact 514. In this example, contact 518 is at position 518 -B on touch screen 112, which corresponds to the position of button 510 -D and also corresponds to the activation area of button 510 -D. In response to detecting contact 518 at position 518-B, if the letter “e” 516 in text 506-2 is detected, text 506-2 is replaced with text 506-1. Returning, the cursor 508 is moved to the position 508 -A. The button 510-D is a conditionally enabled button that is not affected by the time window actuated by the contact 514, and the contact 518 detected at the location 518-B is the contact 514. ), Character deletion and cursor movement occur depending on whether it occurs inside or outside the time window. In addition, detecting contact 518 at position 518 -B starts a new time window. As mentioned above, the time window has a predefined duration, which is fixed (ie invariant) in some embodiments, and variable or situation dependent in other embodiments.

UI 500F (FIG. 5F) shows contact 518 to touch screen 112 immediately following contact 514. In this example, contact 518 is at position 518 -C on touch screen 112, which corresponds to the position of button 512 -C and also corresponds to the activation area of button 512 -C. Button 512-C is a conditionally enabled button that is affected by the time window from contact 514. Depending on when the contact 518 was made (by which time the contact 518 was detected), the keyboard 504 is hidden or nothing occurs in response to detecting the contact 518 at position 518-C. I never do that. If contact 518 is detected at location 518-C at a time in the time window, keyboard 504 is not hidden, UI 500F continues to be displayed, and contact 518 is ignored. If contact 518 is detected at location 518-C at a time after the time window, keyboard 504 is hidden as shown in UI 500G (FIG. 5G).

Timeline 550 (not shown to scale) (FIG. 5I) represents the time at which the above-described contacts can be detected and shows the time window initiated by the detected contact. Timeline 550 represents time 552 when a contact was detected, and times 554, 556, and 558 at which subsequent contacts can be detected (eg, a time within a time window length of time 552). Assume no contact was detected before 552). A detected contact (eg, location) at time 552 at a location corresponding to button 510 or 522 (or in other embodiments, any button on soft keyboard 504 or 522 that includes button 512). Contact 514 at 514-A starts a time window 560-A which is a duration starting from time 552. The button corresponding to the location of the contact detected at time 552 is activated. If a contact is detected at time 554 and the contact (eg, contact 514 at location 514-A corresponding to button 510 -C) is at a location corresponding to button 510 or 522. The corresponding button is activated (eg, button 510 -C is activated), and activation of button 510 or 520 is not affected by the time windows. A contact is detected at time 554 and the contact (e.g., contact 518 at location 518-C corresponding to button 512-C) is located at a location corresponding to the conditionally enabled button. If so, activation of the corresponding button is prevented (eg, activation of the button 512-C), and the conditionally enabled button is disabled during the time window 560 -A. On the other hand, the contact at the location corresponding to the conditionally enabled button without time intervening contact between time 552 and time 558 is detected outside time 558-A. When detected, the corresponding conditionally enabled button is activated.

The time window 560 -B in which the contact on the button 510 or 522 detected at time 554 immediately following the contact detected at time 552 described above can have the same length as the time window 560 -A. Start). If the contact immediately following the contact detected at time 554 is a contact at a position corresponding to the conditionally enabled button, and if the contact is detected at time 556, the activation of the conditionally enabled button is enabled. Is prevented. Even if time 556 is outside time window 560-A, activation is prevented because time 556 is inside time window 560-B. If a contact is instead detected at time 558, the corresponding conditionally enabled button is activated because time 558 is outside of time windows 560-A and 560-B.

6A and 6B are flow diagrams illustrating a method 600 of activating buttons in accordance with some embodiments. The method 600 is performed on a multifunction device having a display and a touch sensitive surface (eg, the device 300 of Figure 3 or the portable multifunction device 100 of Figure 1. In some embodiments, the display is touch sensitive). And a touch sensitive surface is on the display.

As described below, the method 600 provides an intuitive method of activating buttons that reduces the user's perceived burden, creating a more efficient human-machine interface. What is claimed is: 1. A battery-operated computing device comprising: means for enabling a user to activate buttons more quickly and more efficiently while preventing improper activation of mode change lanes or other conditionally enabled buttons, Increase the time.

The device displays 602 a soft keyboard having a plurality of buttons including a plurality of unconditionally enabled buttons and one or more conditionally enabled buttons. For example, in FIG. 5A, keyboard 504 is displayed on touch screen 112. The keyboard 504 includes buttons 510 and buttons 512 that are unconditionally enabled, and one or more of the buttons 512 (eg, buttons 512 -C) are conditional. These buttons are enabled. As another example, in FIG. 5H, keyboard 520 is displayed on touch screen 112. The keyboard 520 includes buttons 522 and buttons 512 that are unconditionally enabled, and one or more of the buttons 512 are buttons that are conditionally enabled.

In some embodiments, one or more conditionally enabled buttons include a button (eg, button 512 -C) 620 that hides the keyboard; A button 622 for changing a language associated with the soft keyboard (eg, button 512 -B) 622; The soft keyboard is associated with the first language and has a button 624 for switching to a soft keyboard associated with the second language (eg, button 512 -B) 624; A button 626 for switching from an alphabetic keyboard to a numeric and symbolic keyboard (eg, button 512 -A) 626; And buttons (eg, buttons 512 -D) 628 for switching from numeric and symbol keyboards to alphabetic keyboards.

The device detects (604) a first input by the first button at a first time. For example, in FIG. 5B, contact 514 (first input) is detected at time 552 (FIG. 5I) at location 514-A corresponding to button 510 -C.

The device responds (606) to detecting the first input by activating the first button. For example, in FIG. 5B, the device responds to detection of contact 514 at location 514-A by activating button 510 -C. The effects of activation include the insertion of the letter "e" 516 into the input text 506 and the movement of the cursor 508 to the location 508 -B.

After detecting and operating upon activation of the first button, the device detects (608) a second input by the second button at a second time. For example, in FIG. 5C, the device detects contact 518 at a second time after time 552 (time 554 or 556 of FIG. 5I). Depending on where the contact 518 is detected at the second time (e.g., which button is selected by the contact 518) and also the second time is within the time window initiated by the conventional button activation. Different effects may occur depending on whether there is.

In response to detecting the second input by the second button at a second time (610), when the first button is an unconditionally enabled button and the second button is an unconditionally enabled button, The device activates 612 the second button. For example, in FIG. 5D, contact 518 is detected at a location 518-A corresponding to an unconditionally enabled button 510-B. The button 510-B is activated, and the effects of the activation include the insertion of the letter "w" 520 into the input text 506 and the movement of the cursor 508 to the location 508 -C. As another example, in FIG. 5E, contact 518 is instead detected at a location 518-B corresponding to an unconditionally enabled button 510 -D. The button 510-D is activated, and the effects of the activation include the letter "e" 516 being deleted from the input text 506 and the cursor 508 moving to position 508 -A.

In response to detecting a second input by a second button at a second time (610), when the first button is an unconditionally enabled button and the second button is a conditionally enabled button (614) Whether the second button is activated depends on the period between the first time that the contact 514 is detected and the second time that the contact 518 is detected. If the period between the first time and the second time is greater than a predefined threshold (which in some embodiments is a fixed value, in other embodiments a variable or context-dependent value), the apparatus Activate the second button (616). If the period between the first time and the second time is less than or equal to the predefined threshold (ie, when the second contact occurs within the mode change suppression time window), the device prevents the second button from being activated. (618).

For example, in FIG. 5F, contact 518 is detected at location 518-C corresponding to conditionally enabled button 512-C. If contact 518 is detected at time 556, button 512-C is activated in response to detection of contact 518, since the period between time 552 and time 556 is a threshold (time window). (560-A), and time 556 is outside the time window 560-A. The effects of activation include hiding the keyboard 504, for example, as shown in FIG. 5G. When contact 518 is detected at time 554, the button 512-C is prevented from being activated in response to the detection of contact 518, because a period between time 552 and time 554 This is because the time 554 is on or within the time window 560 -A, which is less than or equal to the threshold. In this case, the keyboard 504 is not hidden and the UI 500F is displayed continuously.

Some operations of method 600 may be combined and / or the order of some operations may be changed. While the above discussion of FIG. 6A and FIG. 6B (and the discussion below in FIG. 7) use the terms “buttons” and “soft keyboards”, other embodiments of methods 600 and 700 are non-keyboard (non- keyboard) is applicable to user-initiated activations of objects other than buttons such as icons. In addition, the method 600 discloses conditionally and temporarily disabling activation of certain buttons (also called conditional deactivation), although other embodiments of the method 600 may be modified by various buttons or apparatus. Conditionally and temporarily applicable to the user interface of the running application or other objects of the user interface of the device (also called conditional activation). For example, in embodiments of the method 600 in which the roles of operations 616 and 618 are reversed, the method may include a particular set of buttons (eg, keyboard buttons or non-response in response to activation of another button or object). Keyboard buttons) or objects temporarily and conditionally.

7 is a flow diagram illustrating a method 700 of activating buttons in accordance with some embodiments. The method 700 is performed in a multifunctional device having a display and a touch sensitive surface (e.g., the device 300 of FIG. 3 or the portable multifunctional device 100 of FIG. 1). In some embodiments, the display is a touch screen display and the touch sensitive surface is on the display.

As described below, the method 700 provides an intuitive way to activate the buttons. The method reduces the user's recognition burden when activating the buttons, thereby creating a more efficient human-machine interface. In battery operated computing devices, enabling a user to activate buttons faster and more efficiently conserves power and increases the time between battery charges.

The device displays 702 a soft keyboard having a plurality of buttons including a plurality of unconditionally enabled buttons and one or more conditionally enabled buttons. For example, in FIG. 5A, keyboard 504 is displayed on touch screen 112. The keyboard 504 includes buttons 510 and buttons 512, and one or more of the buttons 512 (eg, buttons 512 -C) are conditionally enabled. As another example, in FIG. 5H, keyboard 520 is displayed on touch screen 112. The keyboard 520 includes buttons 522 and buttons 512 that are unconditionally enabled, and one or more of the buttons 512 are buttons that are conditionally enabled.

The device detects 704 a first input by each unconditionally enabled button at a first time. For example, in FIG. 5B, contact 514 (first input) is detected at time 552 at location 514-A corresponding to unconditionally enabled button 510-C.

In response to detecting the first input at a first time (706), the apparatus activates each of the unconditionally enabled buttons (708) and releases at least one of the conditionally enabled buttons at the first time. Disable for a predefined period starting at 710. For example, in FIG. 5B, the device responds to detection of contact 514 at location 514-A at time 552 by activating button 510-C. The effects of activation include the insertion of the letter "e" 516 into the input text 506 and the movement of the cursor 508 to the location 508 -B. In addition, in response to the detection of contact 514, button 512-C is disabled for time window 560-A starting at time 552.

The device detects a second input by the disabled, conditionally enabled button in response to the first input, the second input being detected 712 at a second time that is within a predefined period. For example, in FIG. 5F, the contact 518 is located at the time window (at position 518-C) corresponding to the button 512-C disabled in response to the detection of the contact 514 at time 552. It is detected at time 554 within 560 -A.

In response to detecting the second input at a second time, the device disregards 714 the second input. For example, the keyboard 504 is not hidden despite the contact 518 at position 518-C (ie, UI 500F continues to be displayed) and the contact at position 518-C ( 518) is ignored.

In other embodiments, the aforementioned methods and apparatus for implementing conditionally enabled buttons may be implemented in other user interfaces other than keys, buttons, or soft keyboards that are conditionally enabled or conditionally disabled. Can be used to implement objects

Operations in the information processing methods described above may be implemented by executing one or more functional modules of the information processing device, such as general purpose processors or application specific chips. These modules, combinations of these modules, and their combination with general hardware (as described above with respect to Figures 1A, 1B, and 3, for example) are all within the protection scope of the present invention.

The operations shown in FIGS. 6A, 6B and 7 may be implemented by the components shown in FIGS. 1A-1C. For example, “detecting a first input by a first button at a first time” 604 may be implemented by event classifier 170, event recognizer 180, and event handler 190. The event monitor 171 in the event classifier 170 detects a contact on the touch sensitive display 112, and the event dispatcher module 174 sends event information to the application 136-1. In this case, application 136-1 includes methods for manipulating a soft keyboard and text display area and graphical user interfaces. Each event recognizer 180 of the application 136-1 compares the event information with the respective event definitions 186, and the first contact at a first location on the touch sensitive surface is a key or specific button on the soft keyboard. Determine whether it corresponds to a predefined event or subevent, such as the selection of. If a predefined event or subevent is detected, the event recognizer 180 activates the event handler 180 associated with the detection of the first contact. Event handler 180 uses or invokes data updater 176 or object updater 177 to update the data or text display area. Likewise, it will be apparent to those skilled in the art how other processes are implemented based on the components shown in FIGS. 1A-1C.

The foregoing description has been described with reference to specific embodiments for purposes of explanation. However, the illustrative discussions above are not intended to be exhaustive or to limit the invention to the precise forms disclosed. Many modifications and variations are possible in light of the above teachings. The embodiments have been selected and described in order to best explain the principles of the present invention and their practical applications, thereby enabling those skilled in the art to best utilize various embodiments having various modifications suitable for the present invention and the particular use contemplated. .

Claims (22)

As a multifunction device,
Touch screen display;
One or more processors;
Memory; And
One or more programs
/ RTI >
The one or more programs are stored in the memory and are configured to be executed by the one or more processors, wherein the one or more programs are:
Display a soft keyboard having a plurality of buttons including a plurality of unconditionally enabled buttons and one or more conditionally enabled buttons,
Detect a first input by a first button at a first time,
In response to detecting the first input by activating the first button,
Detect a second input by a second button at a second time,
In response to detecting the second input by the second button at the second time,
When the first button is an unconditionally enabled button and the second button is an unconditionally enabled button, the second button is activated.
If the first button is an unconditionally enabled button and the second button is a conditionally enabled button,
If the period between the first time and the second time is greater than a predefined threshold, activate the second button,
Preventing the second button from being activated if the period between the first time and the second time is less than or equal to the predefined threshold.
A multifunction device comprising instructions. The method of claim 1,
And the at least one conditionally enabled button comprises a button to hide the soft keyboard. The method of claim 1,
And the at least one conditionally enabled button comprises a button for changing a language associated with the soft keyboard. The method of claim 1,
Wherein the soft keyboard is associated with a first language and the one or more conditionally enabled buttons include a button to switch to a soft keyboard associated with a second language. The method of claim 1,
And the at least one conditionally enabled button comprises a button to switch from an alphabetic keyboard to a numeric and symbolic keyboard. The method of claim 1,
And the at least one conditionally enabled button comprises a button for switching from a numeric and symbol keyboard to an alphabetic keyboard. As a method,
In a multifunction device with a touch screen display,
Displaying a soft keyboard having a plurality of buttons including a plurality of unconditionally enabled buttons and one or more conditionally enabled buttons;
Detecting a first input by a first button at a first time;
Responsive to detecting the first input by activating the first button;
Detecting a second input by a second button at a second time; And
In response to detecting the second input by the second button at the second time,
When the first button is an unconditionally enabled button and the second button is an unconditionally enabled button, the second button is activated.
If the first button is an unconditionally enabled button and the second button is a conditionally enabled button,
If the period between the first time and the second time is greater than a predefined threshold, activate the second button,
Preventing the second button from being activated if the period between the first time and the second time is less than or equal to the predefined threshold.
/ RTI > The method of claim 7, wherein
And the one or more conditionally enabled buttons include a button to hide the soft keyboard. The method of claim 7, wherein
And the one or more conditionally enabled buttons include a button for changing a language associated with the soft keyboard. The method of claim 7, wherein
Wherein the soft keyboard is associated with a first language, and the one or more conditionally enabled buttons include a button to switch to a soft keyboard associated with a second language. The method of claim 7, wherein
And the one or more conditionally enabled buttons comprise a button to switch from an alphabetic keyboard to a numeric and symbolic keyboard. The method of claim 7, wherein
And the one or more conditionally enabled buttons comprise a button to switch from a numeric and symbolic keyboard to an alphabetic keyboard. A graphical user interface on a multifunction device having a touch screen display, a memory, and one or more processors to execute one or more programs stored in the memory,
Soft keyboard having a plurality of buttons including a plurality of unconditionally enabled buttons and one or more conditionally enabled buttons
Including,
A first input by the first button is detected at a first time,
Detection of the first input is responded to by activation of the first button,
The second input by the second button is detected at the second time,
In response to detecting the second input by the second button at the second time,
When the first button is an unconditionally enabled button and the second button is an unconditionally enabled button, the second button is activated.
If the first button is an unconditionally enabled button and the second button is a conditionally enabled button,
If the period between the first time and the second time is greater than a predefined threshold, the second button is activated,
If the period between the first time and the second time is less than or equal to the predefined threshold, activation of the second button is prevented. A computer readable storage medium for storing one or more programs,
The one or more programs, when executed by a multifunction device having a touch screen display, cause the device to:
Display a soft keyboard having a plurality of buttons including a plurality of unconditionally enabled buttons and one or more conditionally enabled buttons,
Detect a first input by a first button at a first time,
In response to detecting the first input by activating the first button,
Detect a second input by a second button at a second time,
In response to detecting the second input by the second button at the second time,
When the first button is an unconditionally enabled button and the second button is an unconditionally enabled button, the second button is activated.
If the first button is an unconditionally enabled button and the second button is a conditionally enabled button,
If the period between the first time and the second time is greater than a predefined threshold, activate the second button,
If the period between the first time and the second time is less than or equal to the predefined threshold, preventing the second button from being activated.
A computer readable storage medium comprising instructions. As a multifunction device,
Touch screen display;
Means for displaying a soft keyboard having a plurality of buttons, the plurality of buttons including a plurality of unconditionally enabled buttons and one or more conditionally enabled buttons;
Means for detecting a first input by a first button at a first time;
Means for responding to detecting the first input by activating the first button;
Means for detecting a second input by a second button at a second time; And
When the first button is an unconditionally enabled button and the second button is an unconditionally enabled button, the second button is activated.
If the first button is an unconditionally enabled button and the second button is a conditionally enabled button,
If the period between the first time and the second time is greater than a predefined threshold, activate the second button,
If the period between the first time and the second time is less than or equal to the predefined threshold, by preventing the second button from being activated,
Means responsive to detecting the second input by the second button at the second time
Including, multifunctional device. An information processing apparatus for use in a multifunction device having a touch screen display,
Means for displaying a soft keyboard having a plurality of buttons, the plurality of buttons including a plurality of unconditionally enabled buttons and one or more conditionally enabled buttons;
Means for detecting a first input by a first button at a first time;
Means for responding to detecting the first input by activating the first button;
Means for detecting a second input by a second button at a second time; And
When the first button is an unconditionally enabled button and the second button is an unconditionally enabled button, the second button is activated.
If the first button is an unconditionally enabled button and the second button is a conditionally enabled button,
If the period between the first time and the second time is greater than a predefined threshold, activate the second button,
If the period between the first time and the second time is less than or equal to the predefined threshold, by preventing the second button from being activated,
Means responsive to detecting the second input by the second button at the second time
Including, the information processing device. As a multifunction device,
Touch screen display;
One or more processors;
Memory; And
One or more programs
/ RTI >
The one or more programs are stored in the memory and are configured to be executed by the one or more processors, wherein the one or more programs are:
Display a soft keyboard having a plurality of buttons including a plurality of unconditionally enabled buttons and one or more conditionally enabled buttons,
Respectively detect a first input by a button unconditionally enabled at a first time,
Activating the respective unconditionally enabled buttons and disabling the first input by disabling at least one of the conditionally enabled buttons for a predefined period starting from the first time. In response to detecting in one hour,
Detect a second input by a disabled, conditionally enabled button responsive to the first input, the second input being detected at a second time within the predefined time period;
In response to detecting the second input at the second time by ignoring the second input.
A multifunction device comprising instructions. As a method,
In a multifunction device with a touch screen display,
Displaying a soft keyboard having a plurality of buttons including a plurality of unconditionally enabled buttons and one or more conditionally enabled buttons;
Detecting at a first time a first input by each unconditionally enabled button;
In response to detecting the first input at the first time, each of the unconditionally enabled buttons is activated, and the conditionally enabled button for a predefined period starting from the first time. Disabling at least one of them;
Detecting a second input by a disabled, conditionally enabled button in response to the first input, the second input being detected at a second time within the predefined time period; And
In response to detecting the second input at the second time, ignoring the second input
/ RTI > A graphical user interface on a multifunction device having a touch screen display, a memory, and one or more processors to execute one or more programs stored in the memory,
Soft keyboard having a plurality of buttons including a plurality of unconditionally enabled buttons and one or more conditionally enabled buttons
Including,
A first input by each unconditionally enabled button is detected at a first time,
In response to detecting the first input at the first time, the respective unconditionally enabled buttons are activated and at least one of the conditionally enabled buttons starts from the first time. Disabled for a predefined period of time,
A second input by a conditionally enabled button disabled in response to the first input is detected at a second time within the predefined period of time,
The second input is ignored in response to detecting the second input at the second time. A computer readable storage medium for storing one or more programs,
The one or more programs, when executed by a multifunction device having a touch screen display, cause the device to:
Display a soft keyboard having a plurality of buttons including a plurality of unconditionally enabled buttons and one or more conditionally enabled buttons,
Respectively detect a first input by a button unconditionally enabled at a first time,
In response to detecting the first input at the first time, each of the unconditionally enabled buttons is activated, and the conditionally enabled button for a predefined period starting from the first time. Disable at least one of them,
Detect a second input by a disabled, conditionally enabled button responsive to the first input, the second input being detected at a second time within the predefined time period;
Ignore the second input in response to detecting the second input at the second time.
A computer readable storage medium comprising instructions. As a multifunction device,
Touch screen display;
Means for displaying a soft keyboard having a plurality of buttons, the plurality of buttons including a plurality of unconditionally enabled buttons and one or more conditionally enabled buttons;
Means for detecting at a first time a first input by a button, each of which is unconditionally enabled;
Activating the respective unconditionally enabled buttons and disabling the first input by disabling at least one of the conditionally enabled buttons for a predefined period starting from the first time. Means responsive to detecting at one hour;
Means for detecting a second input by a disabled, conditionally enabled button in response to the first input, the second input being detected at a second time within the predefined time period; And
Means for responding to detecting the second input at the second time by ignoring the second input.
Including, multifunctional device. An information processing device for use in a multifunctional device having a touch screen display,
Means for displaying a soft keyboard having a plurality of buttons, the plurality of buttons including a plurality of unconditionally enabled buttons and one or more conditionally enabled buttons;
Means for detecting at a first time a first input by a button, each of which is unconditionally enabled;
Activating the respective unconditionally enabled buttons and disabling the first input by disabling at least one of the conditionally enabled buttons for a predefined period starting from the first time. Means responsive to detecting at one hour;
Means for detecting a second input by a disabled, conditionally enabled button in response to the first input, the second input being detected at a second time within the predefined time period; And
Means for responding to detecting the second input at the second time by ignoring the second input.
Including, the information processing device.

Images