CN115167745A

CN115167745A - Apparatus and method for processing and disambiguating touch input

Info

Publication number: CN115167745A
Application number: CN202210877529.6A
Authority: CN
Inventors: N·M·威尔斯; L·M·古姆; N·A·法赫纳; K·L·考西恩达; C·穆赛特; J-P·穆莱西奥克斯; J·B·考皮恩; J·K·芬尼斯
Original assignee: Apple Inc
Current assignee: Apple Inc
Priority date: 2016-09-06
Filing date: 2017-07-12
Publication date: 2022-10-11
Also published as: CN110058757B; AU2017100980B4; AU2017100980A4; CN110058757A

Abstract

The present disclosure relates generally to devices and methods for processing and disambiguating touch inputs. An electronic device detects a first increase in intensity of an input on an input followed by a first decrease in intensity of the input on the input and determines whether the first decrease satisfies up-click detection criteria. The up-click detection criteria require that the intensity of the input decrease below a first up-click intensity threshold, which is selected based on the intensity of the input during the first intensity increase. In accordance with a determination that the first decrease in intensity of the input satisfies the up-click detection criteria, the device provides first feedback, such as performing an operation that changes the displayed user interface and/or providing a tactile output, indicating that the first decrease in intensity of the input is identified as an up-click.

Description

Apparatus and method for processing and disambiguating touch input

The present application is a divisional application of the inventive patent application having an application number of 201910352081.4, an application date of 2017, 7/month, 12, entitled "apparatus and method for processing and disambiguating a touch input".

Technical Field

This document relates generally to electronic devices having one or more intensity-sensitive input elements, including but not limited to electronic devices having touch-sensitive displays and optionally including other input elements for detecting intensity of contacts on touch-sensitive surfaces.

Background

The use of intensity sensitive input elements, including but not limited to touch sensitive surfaces, as input devices for computers and other electronic computing devices has increased significantly in recent years. Exemplary intensity sensitive input elements include buttons with contact intensity sensors, and touch pads and touch screen displays with contact intensity sensors. Touch input on such surfaces is used to manipulate user interfaces and user interface objects on the display.

Exemplary user interface objects include digital images, videos, text, icons, control elements (such as buttons), and other graphics. Exemplary manipulations include: adjusting the position and/or size of one or more user interface objects or activating a button represented by a user interface object or opening a file/application represented by a user interface object; scroll or change the user interface within the application or otherwise manipulate the user interface. Certain manipulations of user interface objects or user interfaces are associated with a particular type of touch input, known as gestures.

Conventional methods and interfaces for processing touch inputs are inefficient in disambiguating certain touch inputs to determine intended gestures and intended user interface object manipulations. It is therefore desirable to have an improved framework for processing and disambiguation of touch inputs.

Disclosure of Invention

Accordingly, there is a need for electronic devices having faster and more efficient methods and interfaces for processing and disambiguating touch inputs. Such methods and interfaces optionally complement or replace conventional methods for processing and disambiguating touch inputs. Such methods and interfaces reduce the number, extent, and/or nature of inputs from a user and result in a more efficient human-machine interface. For battery-driven devices, such methods and interfaces may conserve power and increase the time between battery charges.

The above-described deficiencies and other problems associated with user interfaces for electronic devices having touch-sensitive surfaces may be reduced or eliminated with the disclosed devices. In some embodiments, the device is a desktop computer. In some embodiments, the device is portable (e.g., a laptop, tablet, or handheld device). In some embodiments, the device is a personal electronic device (e.g., a wearable electronic device, such as a watch). In some embodiments, the device has a touch pad. In some embodiments, the device has a touch-sensitive display (also referred to as a "touch screen" or "touch screen display"). In some embodiments, the device has a Graphical User Interface (GUI), one or more processors, memory, and one or more modules, programs or sets of instructions stored in the memory for performing a plurality of functions. In some embodiments, the user interacts with the GUI primarily through stylus and/or finger contacts and gestures on the touch-sensitive surface. In some embodiments, the functions optionally include image editing, drawing, presenting, word processing, spreadsheet making, game playing, telephone answering, video conferencing, e-mail sending and receiving, instant messaging, fitness support, digital photography, digital video recording, web browsing, digital music playing, note taking, and/or digital video playing. Executable instructions for performing these functions are optionally included in a non-transitory computer-readable storage medium or other computer program product configured for execution by one or more processors.

According to some embodiments, a method performed on an electronic device with a display and an intensity-sensitive input element (e.g., a hardware button, a touch-sensitive surface, or an area of the device associated with one or more intensity sensors) includes: detecting a first increase in intensity of an input on the input element that satisfies a down-click detection criterion, and detecting a first decrease in intensity of a contact after detecting the first increase in intensity of the input on the input element, the intensity-sensitive input element to detect an intensity of a user input with the input element. The method also includes determining whether the first decrease in intensity of the input satisfies up-click detection criteria, wherein: for the first decrease in intensity, the up-click detection criteria require that the intensity of the input decrease below a first up-click intensity threshold selected based on the intensity of the input during the increase in intensity of the detected contact before the first decrease in intensity of the input is detected so that the up-click detection criteria are met. The method also includes, in accordance with a determination that the first decrease in intensity of the input satisfies the up-click detection criteria, providing first feedback indicating that the first decrease in intensity of the input is identified as an up-click input, and in accordance with a determination that the decrease in intensity of the input does not satisfy the up-click detection criteria, forgoing providing the first feedback.

According to some embodiments, a method performed on an electronic device having a display and an intensity-sensitive input element (e.g., a hardware button, a touch-sensitive surface, or a region of the device associated with one or more intensity sensors) for detecting intensity of user input with the input element includes: detecting a change in intensity of an input on the input element, the change in intensity comprising an increase in intensity of the input on the input element followed by a decrease in intensity of the input on the input element; identifying at least a portion of the change in intensity of the input as a first input event associated with a first operation; and delaying execution of the first operation after identifying the first input event while monitoring a subsequent change in intensity of the input for a second input event, wherein the delay is limited by a default delay time period. The method also includes, after delaying execution of the first operation: in accordance with a determination that the second input event has been identified before the default delay time period has elapsed, performing a second operation and aborting the performance of the first operation; in accordance with a determination that early-confirmation criteria for the first input event have been met before the default delay time period elapses and the second input event is not recognized, performing the first operation before the default delay time period elapses; and in accordance with a determination that the default delay time period has elapsed without the early-confirmation criteria for the first input event being met and the second input event being unrecognized, performing the first operation once the default delay time period has elapsed.

According to some embodiments, a method performed on an electronic device having a display and an intensity-sensitive input element (e.g., a hardware button, a touch-sensitive surface, or an area of the device associated with one or more intensity sensors) for detecting an intensity of user input with the input element includes detecting an input sequence that includes an increase in intensity of an input corresponding to a first input event. The method also includes, in response to detecting the input sequence: in accordance with a determination that a second input event (including a decrease in intensity of the input after the first input event) is detected within a first time period after the first input event is detected, a first operation is performed. The method also includes, in accordance with a determination that the second input event is not detected for a second time period that is longer than the first time period and that the input has a characteristic intensity above a respective intensity threshold between when the first input event is detected and when the second time period elapses, performing a second operation once the second time period has elapsed, wherein the second time period is determined based at least in part on the intensity of the input after the first input event is detected. The method also includes, in accordance with a determination that the second input event was not detected for a third time period that is longer than the second time period and that the input did not have a characteristic intensity above the respective intensity threshold between when the first input event was detected and when the second time period elapsed, performing the second operation once the third time period has elapsed.

In accordance with some embodiments, an electronic device includes a display, a touch-sensitive surface, memory, one or more processors, one or more programs, and optionally one or more sensors to detect intensity of contacts with the touch-sensitive surface; the one or more programs are stored in the memory and configured to be executed by the one or more processors and the one or more programs include instructions for performing or causing the performance of the operations of any of the methods described herein. According to some embodiments, a computer-readable storage medium has stored therein instructions that, when executed by an electronic device with a display, a touch-sensitive surface, and optionally one or more sensors to detect intensity of contacts with the touch-sensitive surface, cause the device to perform or cause to be performed the operations of any of the methods described herein. According to some embodiments, a graphical user interface on an electronic device with a display, a touch-sensitive surface, optionally one or more sensors to detect intensity of contacts with the touch-sensitive surface, a memory, and one or more processors to execute one or more programs stored in the memory includes one or more elements displayed in any of the methods described herein that are updated in response to an input, as described in any of the methods described herein. According to some embodiments, an electronic device comprises: a display, a touch-sensitive surface, and optionally one or more sensors for detecting intensity of contacts with the touch-sensitive surface; and means for performing or causing performance of the operations of any of the methods described herein. According to some embodiments, an information processing apparatus for use in an electronic device with a display and a touch-sensitive surface and optionally one or more sensors to detect intensity of contacts with the touch-sensitive surface comprises means for performing, or causing to be performed, operations of any method described herein.

Accordingly, electronic devices having a display, one or more touch-sensitive surfaces, and one or more sensors for detecting intensity of contacts with the touch-sensitive surfaces are provided with faster, more efficient methods and interfaces for processing and disambiguating touch inputs, thereby improving the efficiency, and user satisfaction of such devices. Such methods and interfaces may complement or replace conventional methods for processing and disambiguating touch inputs.

Drawings

For a better understanding of the various described embodiments, reference should be made to the following detailed description taken in conjunction with the following drawings, wherein like reference numerals designate corresponding parts throughout the figures.

FIG. 1A is a block diagram illustrating a portable multifunction device with a touch-sensitive display in accordance with some embodiments.

Fig. 1B is a block diagram illustrating exemplary components for event processing, according to some embodiments.

Fig. 1C is a block diagram illustrating a haptic output module according to some embodiments.

Figure 2A illustrates a portable multifunction device with a touch screen in accordance with some embodiments.

2B-2C illustrate exploded views of an intensity-sensitive input device, according to some embodiments.

Fig. 3 is a block diagram of an exemplary multifunction device with a display and a touch-sensitive surface in accordance with some embodiments.

FIG. 4A illustrates an exemplary user interface of an application menu on a portable multifunction device according to some embodiments.

FIG. 4B illustrates an exemplary user interface for a multifunction device with a touch-sensitive surface separate from a display, according to some embodiments.

Fig. 4C-4E illustrate examples of dynamic intensity thresholds according to some embodiments.

Fig. 4F-4G illustrate a set of sample haptic output patterns, according to some embodiments.

Fig. 5A-5II illustrate an exemplary user interface and multiple timeout periods and intensity thresholds for detecting gestures, according to some embodiments.

6A-6F are flow diagrams illustrating methods of processing and disambiguating touch inputs according to some embodiments.

7A-7E are flow diagrams illustrating methods of processing and disambiguating touch inputs according to some embodiments.

8A-8C are flow diagrams illustrating methods of processing and disambiguating touch inputs according to some embodiments.

Fig. 9-11 are functional block diagrams of electronic devices according to some embodiments.

Detailed Description

In electronic devices that display graphical user interfaces and have intensity sensitive input elements, it is challenging to both accurately and quickly detect and distinguish multiple different gestures, such as deep press gestures, long press gestures, single tap gestures, double tap gestures, and possibly triple tap gestures. Such gestures involve input having a time-varying intensity. Distinguishing such gestures requires analyzing the intensity of the input on the input element as well as analyzing timing aspects or features of the input. Furthermore, some users have "heavier touch" than others, applying more pressure on average than others. Similarly, some users enter gestures at a higher speed than others. Despite these different patterns or styles of user input, the electronic device needs to accurately discern the user intent, and must discern the user intent with low latency so that operation of a user request or command is quickly performed in response to the user's touch input. In some embodiments, to achieve such accuracy and touch input processing performance, one or more intensity thresholds for detecting a gesture or portion of a gesture vary according to the intensity of the user's input during one or more prior portions of the gesture. Further, in some embodiments, to achieve such accuracy and touch input processing performance, one or more time periods used in the analysis of the touch input vary according to the intensity of the touch input, thereby enabling a particular gesture to be recognized more quickly when predefined criteria are met.

In another aspect, tactile feedback (also referred to as haptic output) may be used to facilitate user input, confirm recognition of various user inputs, and prompt a user for the occurrence of various events, various input conditions, and the like. As the number and complexity of tactile feedback events in a device increases, it becomes important to ensure that particular tactile outputs are generated consistently, even when the detection criteria for triggering those tactile outputs change (e.g., the detection criteria for triggering long press tactile outputs may be different for different applications or different for different contexts in the same application).

1A-1B, FIG. 2, and FIG. 3 provide a description of exemplary devices. 4A-4B and 5A-5II illustrate exemplary user interfaces of an electronic device configured to monitor input on intensity-sensitive input elements and detect various events (such as a release click, a press click, a single click, a double click, a deep press input, and a long press input) using various input intensity criteria and timing criteria for quickly and efficiently determining user input. 6A-6F illustrate a flow diagram of a method of monitoring input on an intensity-sensitive input element and detecting an up-click and/or a down-click in the monitored input using one or more intensity thresholds based on a previous input intensity of the input. 7A-7E illustrate a flow diagram of a method of monitoring a change in intensity of an input and applying early validation criteria for identifying single-tap inputs, as distinguished from double-tap inputs, at an accelerated rate. 8A-8C illustrate a flow diagram of a method of monitoring intensity changes of an input and applying intensity sensitivity criteria for expedited recognition of a long press input. The user interfaces in fig. 5A-5N are used to illustrate the processes in fig. 6A-6F. The user interfaces in FIGS. 5O-5Y are used to illustrate the processes in FIGS. 7A-7E. The user interfaces in fig. 5Z-5II are used to illustrate the processes in fig. 8A-8C.

Exemplary device

Reference will now be made in detail to the embodiments, examples of which are illustrated in the accompanying drawings. In the following detailed description, numerous specific details are set forth in order to provide a thorough understanding of various described embodiments. It will be apparent, however, to one skilled in the art that the various described embodiments may be practiced without these specific details. In other instances, well-known methods, procedures, components, circuits, and networks have not been described in detail as not to unnecessarily obscure aspects of the embodiments.

It will also be understood that, although the terms first, second, etc. may be used herein to describe various elements in some cases, these elements should not be limited by these terms. These terms are only used to distinguish one element from another. For example, a first contact can be termed a second contact, and, similarly, a second contact can be termed a first contact, without departing from the scope of the various described embodiments. The first contact and the second contact are both contacts, but they are not the same contact unless the context clearly indicates otherwise.

The terminology used in the description of the various described embodiments herein is for the purpose of describing particular embodiments only and is not intended to be limiting. As used in the description of the various described embodiments and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It should also be understood that the term "and/or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items. It will be further understood that the terms "comprises," "comprising," "includes" and/or "including," when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.

As used herein, the term "if" is optionally interpreted to mean "when … …" ("while" or "upon") or "in response to a determination" or "in response to a detection", depending on the context. Similarly, the phrase "if it is determined … …" or "if [ stated condition or event ] is detected" is optionally interpreted to mean "at determination … …" or "in response to determination … …" or "upon detection [ stated condition or event ] or" in response to detection [ stated condition or event ] ", depending on the context.

Embodiments of electronic devices, user interfaces for such devices, and related processes for using such devices are described herein. In some embodiments, the device is a portable communication device, such as a mobile phone, that also contains other functions, such as PDA and/or music player functions. Exemplary embodiments of portable multifunction devices include, but are not limited to, those from Apple inc

iPod

And

an apparatus. Other portable electronic devices are optionally used, such as laptops or tablets with touch-sensitive surfaces (e.g., touch screen displays and/or touch pads). It should also be understood that in some embodiments, the device is not a portable communication device, but is a desktop computer with a touch-sensitive surface (e.g., a touch screen display and/or touchpad).

In the following discussion, an electronic device including a display and a touch-sensitive surface is described. However, it should be understood that the electronic device optionally includes one or more other physical user interface devices, such as a physical keyboard, mouse, and/or joystick.

The device typically supports various applications, such as one or more of the following: a note taking application, a drawing application, a rendering application, a word processing application, a website creation application, a disc editing application, a spreadsheet application, a gaming application, a telephone application, a video conferencing application, an email application, an instant messaging application, a fitness support application, a photo management application, a digital camera application, a digital video camera application, a web browsing application, a digital music player application, and/or a digital video player application.

Various applications executing on the device optionally use at least one common physical user interface device, such as a touch-sensitive surface. One or more functions of the touch-sensitive surface and corresponding information displayed on the device are optionally adjusted and/or varied for different applications and/or within respective applications. In this way, a common physical architecture of the device (such as a touch-sensitive surface) optionally supports various applications with a user interface that is intuitive and clear to the user.

Attention is now directed to embodiments of portable devices having touch sensitive displays. FIG. 1A is a block diagram illustrating a portable multifunction device 100 with a touch-sensitive display system 112 in accordance with some embodiments. Touch-sensitive display system 112 is sometimes referred to as a "touch screen" for convenience and is sometimes referred to simply as a touch-sensitive display. Device 100 includes memory 102 (which optionally includes one or more computer-readable storage media), a memory controller 122, one or more processing units (CPUs) 120, a peripheral interface 118, RF circuitry 108, audio circuitry 110, a speaker 111, a microphone 113, an input/output (I/O) subsystem 106, other input or control devices 116, and an external port 124. The device 100 optionally includes one or more optical sensors 164. Device 100 optionally includes one or more intensity sensors 165 for detecting intensities of contacts on device 100 (e.g., a touch-sensitive surface, such as touch-sensitive display system 112 of device 100). Device 100 optionally includes one or more tactile output generators 167 for generating tactile outputs on device 100 (e.g., generating tactile outputs on a touch-sensitive surface such as touch-sensitive display system 112 of device 100 or touch panel 355 of device 300). These components optionally communicate over one or more communication buses or signal lines 103.

As used in this specification and claims, the term "haptic output" refers to a physical displacement of a device relative to a previous position of the device, a physical displacement of a component of the device (e.g., a touch-sensitive surface) relative to another component of the device (e.g., a housing), or a displacement of a component relative to a center of mass of the device that is to be detected by a user with the user's sense of touch. For example, where the device or component of the device is in contact with a surface of the user that is sensitive to touch (e.g., a finger, palm, or other portion of the user's hand), the tactile output generated by the physical displacement will be interpreted by the user as a tactile sensation corresponding to a perceived change in a physical characteristic of the device or component of the device. For example, movement of the touch-sensitive surface (e.g., a touch-sensitive display or trackpad) is optionally interpreted by the user as a "down click" or "up click" of a physical actuation button. In some cases, the user will feel a tactile sensation, such as a "press click" or "release click," even when the physical actuation button associated with the touch-sensitive surface that is physically pressed (e.g., displaced) by the user's movement is not moving. As another example, movement of the touch sensitive surface may optionally be interpreted or sensed by the user as "roughness" of the touch sensitive surface even when there is no change in the smoothness of the touch sensitive surface. While such interpretation of touch by a user will be limited by the user's individualized sensory perception, the sensory perception of many touches is common to most users. Thus, when a haptic output is described as corresponding to a particular sensory perception of a user (e.g., "up click," "down click," "roughness"), unless otherwise stated, the generated haptic output corresponds to a physical displacement of the device or a component thereof that would generate the described sensory perception of a typical (or ordinary) user. Providing haptic feedback to a user using haptic output enhances the operability of the device and makes the user device interface more efficient (e.g., by helping the user provide appropriate input and reducing user error in operating/interacting with the device), thereby further reducing power usage and extending the battery life of the device by enabling the user to use the device more quickly and efficiently.

In some embodiments, the haptic output pattern specifies a characteristic of the haptic output, such as a magnitude of the haptic output, a shape of a motion waveform of the haptic output, a frequency of the haptic output, and/or a duration of the haptic output.

When the device generates haptic outputs having different haptic output modes (e.g., via one or more haptic output generators that move the movable mass to generate the haptic outputs), the haptic outputs may produce different tactile sensations in a user holding or touching the device. While the user's senses are based on the user's perception of the haptic output, most users will be able to recognize changes in the waveform, frequency, and amplitude of the haptic output generated by the device. Thus, the waveform, frequency, and amplitude may be adjusted to indicate to the user that a different operation has been performed. As such, having characteristics (e.g., size, material, weight, stiffness, smoothness, etc.) designed, selected, and/or arranged to simulate objects in a given environment (e.g., a user interface including graphical features and objects, a simulated physical environment having virtual boundaries and virtual objects, a real physical environment having physical boundaries and physical objects, and/or a combination of any of the above); behavior (e.g., oscillation, displacement, acceleration, rotation, stretching, etc.); and/or interaction (e.g., collision, adhesion, repulsion, attraction, friction, etc.) will in some cases provide helpful feedback to the user that reduces input errors and improves the efficiency of the user's operation of the device. Additionally, the haptic output is optionally generated to correspond to feedback unrelated to the simulated physical characteristic (such as an input threshold or object selection). Such tactile output will in some cases provide helpful feedback to the user, which reduces input errors and improves the efficiency of the user's operation of the device.

In some embodiments, the haptic output with the appropriate haptic output mode serves as a cue for an event of interest to occur in the user interface or behind the screen in the device. Examples of events of interest include activation of an affordance (e.g., a real or virtual button, or a toggle switch) provided on the device or in the user interface, success or failure of a requested operation, reaching or crossing a boundary in the user interface, entering a new state, switching input focus between objects, activating a new mode, reaching or crossing an input threshold, detecting or recognizing a type of input or gesture, and so forth. In some embodiments, a tactile output is provided to serve as a warning or cue as to an impending event or result that may occur unless a change of direction or an interrupt input is detected in time. Haptic output is also used in other contexts to enrich the user experience, improve accessibility to the device by users having visual or motor difficulties or other accessibility needs, and/or improve the efficiency and functionality of the user interface and/or device. Optionally comparing the tactile output with the audio input and/or visual user interface changes further enhances the user's experience when interacting with the user interface and/or device and facilitates better transfer of information about the state of the user interface and/or device, and this reduces input errors and improves the efficiency of the user's operation of the device.

Fig. 4F provides a set of sample haptic output patterns that can be used, individually or in combination, as is or through one or more transformations (e.g., modulation, magnification, truncation, etc.) to generate suitable haptic feedback in various scenarios for various purposes, such as those described above and those described for the user interfaces and methods discussed herein. This example of a control panel of haptic output shows how a set of three waveforms and eight frequencies can be used to generate an array of haptic output patterns. In addition to the haptic output modes shown in this figure, each of these haptic output modes is optionally adjusted in magnitude by changing the gain values of the haptic output modes, as shown, for example, for FullTap 80Hz, fullTap 200Hz, miniTap 80Hz, miniTap 200Hz, microTap 80Hz, and MicroTap 200Hz in fig. 4G, which are each illustrated with variants having gains of 1.0, 0.75, 0.5, and 0.25. Changing the gain of the haptic output pattern changes the amplitude of the pattern without changing the frequency of the pattern or changing the shape of the waveform, as shown in FIG. 4G. In some embodiments, changing the frequency of the tactile output pattern also results in a lower amplitude because some tactile output generators are limited in how much force can be applied to the movable mass, so the higher frequency movement of the mass is constrained to a lower amplitude to ensure that the acceleration required to generate the waveform does not require forces outside the operating force range of the tactile output generator (e.g., peak amplitudes of fulltaps at 230Hz, 270Hz, and 300Hz are lower than amplitudes of fulltaps at 80Hz, 100Hz, 125Hz, and 200 Hz).

In fig. 4F, each column displays a tactile output pattern having a particular waveform. Waveform representation of haptic output pattern relative to neutral position (e.g., x) _zero ) Through which the movable mass passes to generate a haptic output having the haptic output pattern. For example, the first set of haptic output patterns (e.g., the haptic output pattern of "FullTap") shown in the left column of fig. 4F each have a waveform (e.g., on) that includes an oscillation having two complete cyclesBeginning and ending in neutral position and passing through neutral position three times). The second set of haptic output patterns (e.g., the haptic output patterns of "MiniTap") shown in the middle column of fig. 4F each have a waveform that includes an oscillation having one full cycle (e.g., an oscillation that begins and ends at a neutral position and passes through the neutral position once). The third set of tactile output patterns shown in the right column of fig. 4F (e.g., the tactile output patterns of "MicroTap") each have a waveform that includes an oscillation having one-half full cycle (e.g., an oscillation that begins and ends at a neutral position and does not pass through the neutral position). The waveform of the haptic output pattern also includes a start buffer and an end buffer representing the gradual acceleration and deceleration of the movable mass at the beginning and end of the haptic output. The exemplary waveforms shown in FIGS. 4F-4G include x representing the maximum and minimum degrees of movement of the movable mass _min And x _max The value is obtained. For larger electronic devices where the movable mass is large, the minimum and maximum degrees of movement of the mass may be greater or less. The examples shown in fig. 4F-4G describe the movement of a mass in 1 dimension, but similar principles can be applied to the movement of a movable mass in two or three dimensions.

As shown in fig. 4F, each haptic output pattern also has a corresponding characteristic frequency that affects the "pitch" of the tactile sensation felt by the user from the haptic output having that characteristic frequency. For continuous haptic output, the characteristic frequency represents the number of cycles (e.g., cycles per second) that a movable mass of the haptic output generator completes within a given time period. For discrete haptic output, a discrete output signal (e.g., having 0.5, 1, or 2 cycles) is generated, and the characteristic frequency value specifies how fast the movable mass needs to move to generate the haptic output having the characteristic frequency. As shown in fig. 4F, for each type of haptic output (e.g., defined by a respective waveform, such as FullTap, miniTap, or MicroTap), a higher frequency value corresponds to faster movement of the movable mass, and thus, in general, a shorter haptic output completion time (e.g., including the time to complete the desired number of cycles of discrete haptic output plus start and end buffer times). For example, a FullTap with a characteristic frequency of 80Hz takes longer to complete than a FullTap with a characteristic frequency of 100Hz (e.g., 35.4ms vs.28.3ms in FIG. 4F). Further, for a given frequency, a haptic output having more cycles in its waveform at the corresponding frequency takes longer to complete than a haptic output having fewer cycles in its waveform at the same corresponding frequency. For example, a 150Hz FullTap takes longer to complete than a 150Hz MiniTap (e.g., 19.4ms vs.12.8 ms), and a 150Hz MiniTap takes longer to complete than a 150Hz MicroTap (e.g., 12.8ms vs.9.4 ms). However, for haptic output patterns having different frequencies, this rule may not apply (e.g., a haptic output with more cycles but a higher frequency may take a shorter amount of time to complete than a haptic output with fewer cycles but a lower frequency, or vice versa). For example, at 300Hz, fullTap takes as long as MiniTap (e.g., 9.9 ms).

As shown in fig. 4F, the haptic output pattern also has a characteristic magnitude that affects the amount of energy contained in the haptic signal, or the "intensity" of the tactile sensation that the user can feel through the haptic output having the characteristic magnitude. In some embodiments, the characteristic magnitude of the tactile output pattern refers to an absolute or normalized value representing the maximum displacement of the movable mass relative to a neutral position when generating the tactile output. In some embodiments, the characteristic magnitude of the haptic output pattern may be adjusted according to various conditions (e.g., customized based on user interface context and behavior) and/or preconfigured metrics (e.g., input-based metrics, and/or user interface-based metrics), such as by a fixed or dynamically determined gain factor (e.g., a value between 0 and 1). In some embodiments, a characteristic of an input (e.g., a rate of change of a characteristic intensity of a contact in a press input or a rate of movement of a contact on a touch-sensitive surface) during the input that triggers generation of a tactile output is measured based on a metric of the input (e.g., an intensity change metric or an input speed metric). In some embodiments, a characteristic of a user interface element (e.g., the speed of movement of the element through a bug or visible boundary in the user interface) during a user interface change that triggers generation of a tactile output is measured based on a metric of the user interface (e.g., a cross-boundary speed metric). In some embodiments, the characteristic amplitude of the tactile output pattern may be "envelope" modulated, and the peaks of adjacent cycles may have different amplitudes, with one of the waveforms shown above being further modified by multiplying with an envelope parameter that varies over time (e.g., from 0 to 1) to gradually adjust the amplitude of portions of the tactile output over time as the tactile output is generated.

Although specific frequencies, amplitudes, and waveforms are shown in the sample haptic output pattern for illustrative purposes in FIG. 4F, haptic output patterns having other frequencies, amplitudes, and waveforms may be used for similar purposes. For example, a waveform having between 0.5 and 4 cycles may be used. Other frequencies in the range of 60Hz-400Hz may also be used. Table 1 provides examples of specific tactile feedback behaviors, configurations, and examples of their use.

TABLE 1

It should be understood that device 100 is only one example of a portable multifunction device, and that device 100 optionally has more or fewer components than shown, optionally combines two or more components, or optionally has a different configuration or arrangement of these components. The various components shown in fig. 1A are implemented in hardware, software, firmware, or any combination thereof, including one or more signal processing circuits and/or application specific integrated circuits.

The memory 102 optionally includes high-speed random access memory, and also optionally includes non-volatile memory, such as one or more magnetic disk storage devices, flash memory devices, or other non-volatile solid-state memory devices. Access to memory 102 by other components of device 100, such as one or more CPUs 120 and peripheral interface 118, is optionally controlled by a memory controller 122.

Peripheral interface 118 may be used to couple the input and output peripherals of the device to memory 102 and one or more CPUs 120. The one or more processors 120 run or execute various software programs and/or sets of instructions stored in the memory 102 to perform various functions of the device 100 and to process data.

In some embodiments, peripherals interface 118, one or more CPUs 120, and memory controller 122 are optionally implemented on a single chip, such as chip 104. In some other embodiments, they are optionally implemented on separate chips.

RF (radio frequency) circuitry 108 receives and transmits RF signals, also called electromagnetic signals. The RF circuitry 108 converts electrical signals to/from electromagnetic signals and communicates with communication networks and other communication devices via electromagnetic signals. RF circuitry 108 optionally includes well-known circuitry for performing these functions, including but not limited to an antenna system, an RF transceiver, one or more amplifiers, a tuner, one or more oscillators, a digital signal processor, a codec chipset, a Subscriber Identity Module (SIM) card, memory, and so forth. RF circuitry 108 optionally communicates with networks, such as the internet, also known as the World Wide Web (WWW), intranets, and/or wireless networks, such as a cellular telephone network, a wireless Local Area Network (LAN), and/or a Metropolitan Area Network (MAN), as well as other devices through wireless communication. The wireless communication optionally uses any of a number of communication standards, protocols, and technologies, including but not limited to global system for mobile communications (GSM), enhanced Data GSM Environment (EDGE), high Speed Downlink Packet Access (HSDPA), high Speed Uplink Packet Access (HSUPA), evolution data only (EV-DO), HSPA +, dual cell HSPA (DC-HSPDA), long Term Evolution (LTE), near Field Communication (NFC), wideband code division multiple access (W-CDMA), code Division Multiple Access (CDMA), time Division Multiple Access (TDMA), bluetooth, wireless fidelity (Wi-Fi) (e.g., IEEE 802.11a, IEEE 802.11ac, IEEE 802.11ax, IEEE 802.11b, IEEE 802.11g, and/or IEEE 802.11 n), voice over internet protocol (VoIP), wi-MAX, email protocols (e.g., internet Message Access Protocol (IMAP) and/or Post Office Protocol (POP)), instant messages (e.g., extensible messaging and presence protocol (XMPP), session initiation protocol with extensions for instant messaging and presence (SIMPLE), instant Messaging and Presence Service (IMPS)), and/or Short Message Service (SMS), or any other suitable communication protocol including communication protocols not yet developed at the filing date of this document.

Audio circuitry 110, speaker 111, and microphone 113 provide an audio interface between a user and device 100. The audio circuitry 110 receives audio data from the peripheral interface 118, converts the audio data to electrical signals, and transmits the electrical signals to the speaker 111. The speaker 111 converts the electric signal into a sound wave audible to a human. The audio circuit 110 also receives electrical signals converted by the microphone 113 from sound waves. The audio circuit 110 converts the electrical signals to audio data and transmits the audio data to the peripheral interface 118 for processing. Audio data is optionally retrieved from and/or transmitted to memory 102 and/or RF circuitry 108 by peripheral interface 118. In some implementations, the audio circuit 110 also includes a headset jack (e.g., 212 in fig. 2A). The headset jack provides an interface between the audio circuitry 110 and a removable audio input/output peripheral such as an output-only headphone or a headset having both an output (e.g., a monaural headphone or a binaural headphone) and an input (e.g., a microphone).

The I/O subsystem 106 couples input/output peripheral devices on the device 100, such as a touch-sensitive display system 112 and other input or control devices 116, to a peripheral interface 118. The I/O subsystem 106 optionally includes a display controller 156, an optical sensor controller 158, an intensity sensor controller 159, a haptic feedback controller 161, and one or more input controllers 160 for other input or control devices. The one or more input controllers 160 receive/transmit electrical signals from/to other input control devices 116. Other input control devices 116 optionally include physical buttons (e.g., push buttons, rocker buttons, etc.), dials, slide switches, joysticks, click wheels, and the like. In some alternative embodiments, one or more input controllers 160 are optionally coupled to (or not coupled to) any of: a keyboard, an infrared port, a USB port, a stylus, and/or a pointing device such as a mouse. The one or more buttons (e.g., 208 in fig. 2A) optionally include an up/down button for volume control of the speaker 111 and/or microphone 113. The one or more buttons optionally include a push button (e.g., 206 in fig. 2A).

Touch-sensitive display system 112 provides an input interface and an output interface between the device and the user. Display controller 156 receives electrical signals from touch-sensitive display system 112 and/or transmits electrical signals to touch-sensitive display system 112. Touch-sensitive display system 112 displays visual output to a user. The visual output optionally includes graphics, text, icons, video, and any combination thereof (collectively "graphics"). In some embodiments, some or all of the visual output corresponds to a user interface object. As used herein, the term "affordance" refers to a user-interactive graphical user interface object (e.g., a graphical user interface object configured to respond to input directed to the graphical user interface object). Examples of user interactive graphical user interface objects include, but are not limited to, buttons, sliders, icons, selectable menu items, switches, hyperlinks, or other user interface controls.

Touch-sensitive display system 112 has a touch-sensitive surface, sensor, or group of sensors that accept input from a user based on haptic and/or tactile contact. Touch-sensitive display system 112 and display controller 156 (along with any associated modules and/or sets of instructions in memory 102) detect contact (and any movement or breaking of the contact) on touch-sensitive display system 112 and convert the detected contact into interaction with user interface objects (e.g., one or more soft keys, icons, web pages, or images) that are displayed on touch-sensitive display system 112. In some embodiments, the point of contact between touch-sensitive display system 112 and the user corresponds to a user's finger or a stylus.

Touch-sensitive display system 112 optionally uses LCD (liquid crystal display) technology, LPD (light emitting polymer display) technology, or LED (light emitting diode) technology, although other display technologies are used in other embodiments. Touch-sensitive display system 112 and display controller 156 optionally detect contact and any movement or breaking thereof using any of a variety of touch sensing technologies now known or later developed, including but not limited to capacitive, resistive, infrared, and surface acoustic wave technologies, as well as other proximity sensor arrays or other elements for determining one or more points of contact with touch-sensitive display system 112. In some embodiments, a projected mutual capacitance sensing technique is used, such as that from Apple inc. (Cupertino, california)

iPod

And

the technique found in (1).

Touch-sensitive display system 112 optionally has a video resolution in excess of 100 dpi. In some embodiments, the touchscreen video resolution exceeds 400dpi (e.g., 500dpi, 800dpi, or greater). The user optionally makes contact with touch-sensitive display system 112 using any suitable object or appendage, such as a stylus, a finger, and so forth. In some embodiments, the user interface is designed to work with finger-based contacts and gestures, which may not be as accurate as stylus-based input due to the larger contact area of the finger on the touch screen. In some embodiments, the device translates the rough finger-based input into a precise pointer/cursor position or command for performing the action desired by the user.

In some embodiments, in addition to a touch screen, device 100 optionally includes a trackpad (not shown) for activating or deactivating particular functions. In some embodiments, the touchpad is a touch-sensitive area of the device that, unlike a touch screen, does not display visual output. The touchpad is optionally a touch-sensitive surface separate from touch-sensitive display system 112, or an extension of the touch-sensitive surface formed by the touch screen.

The device 100 also includes a power system 162 for powering the various components. The power system 162 optionally includes a power management system, one or more power sources (e.g., batteries, alternating Current (AC)), a recharging system, power failure detection circuitry, a power converter or inverter, a power source status indicator (e.g., a Light Emitting Diode (LED)), and any other components associated with the generation, management, and distribution of power in the portable device.

The device 100 optionally further includes one or more optical sensors 164. FIG. 1A shows an optical sensor coupled to an optical sensor controller 158 in the I/O subsystem 106. The one or more optical sensors 164 optionally include Charge Coupled Devices (CCDs) or Complementary Metal Oxide Semiconductor (CMOS) phototransistors. The one or more optical sensors 164 receive light projected through the one or more lenses from the environment and convert the light into data representing an image. In conjunction with imaging module 143 (also called a camera module), one or more optical sensors 164 optionally capture still images and/or video. In some embodiments, an optical sensor is located on the back of device 100 opposite touch-sensitive display system 112 on the front of the device, enabling the touch screen to be used as a viewfinder for still and/or video image capture. In some embodiments, another optical sensor is located on the front of the device to capture images of the user (e.g., for self-timer shooting, for video conferencing while the user is viewing other video conference participants on a touch screen, etc.).

Device 100 optionally further comprises one or more contact intensity sensors 165. FIG. 1A shows a contact intensity sensor coupled to an intensity sensor controller 159 in the I/O subsystem 106. The one or more contact intensity sensors 165 optionally include one or more piezoresistive strain gauges, capacitive force sensors, electrical force sensors, piezoelectric force sensors, optical force sensors, capacitive touch-sensitive surfaces, or other intensity sensors (e.g., sensors for measuring the force (or pressure) of a contact on a touch-sensitive surface). One or more contact intensity sensors 165 receive contact intensity information (e.g., pressure information or a proxy for pressure information) from the environment. In some implementations, at least one contact intensity sensor is collocated with or proximate to a touch-sensitive surface (e.g., touch-sensitive display system 112). In some embodiments, at least one contact intensity sensor is located on the back of device 100 opposite touch-sensitive display system 112, which is located on the front of device 100.

The device 100 optionally further includes one or more proximity sensors 166. Fig. 1A shows a proximity sensor 166 coupled to the peripheral interface 118. Alternatively, the proximity sensor 166 is coupled with the input controller 160 in the I/O subsystem 106. In some embodiments, the proximity sensor turns off and disables touch-sensitive display system 112 when the multifunction device is placed near the user's ear (e.g., when the user is making a phone call).

Device 100 optionally further comprises one or more tactile output generators 167. FIG. 1A shows a haptic output generator coupled to a haptic feedback controller 161 in I/O subsystem 106. One or more tactile output generators 167 optionally include one or more electro-acoustic devices such as speakers or other audio components; and/or an electromechanical device such as a motor, solenoid, electroactive aggregator, piezoelectric actuator, electrostatic actuator, or other tactile output generating component for converting energy into linear motion (e.g., a component for converting an electrical signal into a tactile output on the device). Tactile output generator 167 receives tactile feedback generation instructions from tactile feedback module 133 and generates tactile outputs on device 100 that can be felt by a user of device 100. In some embodiments, at least one tactile output generator is juxtaposed or adjacent to a touch-sensitive surface (e.g., touch-sensitive display system 112), and optionally generates tactile output by moving the touch-sensitive surface vertically (e.g., into/out of the surface of device 100) or laterally (e.g., back and forth in the same plane as the surface of device 100). In some embodiments, at least one tactile output generator sensor is located on the back of device 100 opposite touch-sensitive display system 112 located on the front of device 100.

Device 100 optionally also includes one or more accelerometers 168. Fig. 1A shows accelerometer 168 coupled with peripheral interface 118. Alternatively, accelerometer 168 is optionally coupled with input controller 160 in I/O subsystem 106. In some embodiments, information is displayed in a portrait view or a landscape view on the touch screen display based on an analysis of data received from the one or more accelerometers. Device 100 optionally includes a magnetometer (not shown) and a GPS (or GLONASS or other global navigation system) receiver (not shown) in addition to accelerometer 168 for obtaining information about the position and orientation (e.g., portrait or landscape) of device 100.

In some embodiments, the software components stored in memory 102 include an operating system 126, a communication module (or set of instructions) 128, a contact/motion module (or set of instructions) 130, a graphics module (or set of instructions) 132, a haptic feedback module (or set of instructions) 133, a text input module (or set of instructions) 134, a Global Positioning System (GPS) module (or set of instructions) 135, and an application program (or set of instructions) 136. Further, in some embodiments, memory 102 stores device/global internal state 157, as shown in fig. 1A and 3. The device/global internal state 157 includes one or more of: an active application state indicating which applications (if any) are currently active; display state, which indicates what applications, views, or other information occupy various areas of touch-sensitive display system 112; sensor status, including information obtained from various sensors of the device and other input or control devices 116; and position and/or orientation information regarding the position and/or attitude of the device.

The operating system 126 (e.g., iOS, darwin, RTXC, LINUX, UNIX, OSX, WINDOWS, or embedded operating systems such as VxWorks) includes various software components and/or drivers for controlling and managing general system tasks (e.g., memory management, storage device control, power management, etc.) and facilitates communication between various hardware and software components.

The communication module 128 facilitates communication with other devices through one or more external ports 124 and also includes various software components for processing data received by the RF circuitry 108 and/or the external ports 124. External port 124 (e.g., universal Serial Bus (USB), firewire, etc.) is adapted to couple directly to other devices or indirectly through a network (e.g., the internet, wireless LAN, etc.). In some embodiments, the external port is some of that of Apple inc

iPod

A multi-pin (e.g., 30-pin) connector the same as or similar to and/or compatible with the 30-pin connectors used in iPod devices. In some embodiments, the external port is some of that of Apple inc

iPod

A Lightning connector that is the same as or similar and/or compatible with the Lightning connector used in the iPod device.

Contact/motion module 130 optionally detects contact with touch-sensitive display system 112 (in conjunction with display controller 156) and other touch-sensitive devices (e.g., a touchpad or a physical click wheel). The contact/motion module 130 includes various software components for performing various operations related to contact detection (e.g., by a finger or stylus), such as determining whether contact has occurred (e.g., detecting a finger-down event), determining the intensity of contact (e.g., the force or pressure of the contact, or a substitute for the force or pressure of the contact), determining whether there is movement of the contact and tracking movement across the touch-sensitive surface (e.g., detecting one or more finger-dragging events), and determining whether contact has ceased (e.g., detecting a finger-lift event or a contact-break). The contact/motion module 130 receives contact data from the touch-sensitive surface. Determining movement of the point of contact optionally includes determining velocity (magnitude), velocity (magnitude and direction), and/or acceleration (change in magnitude and/or direction) of the point of contact, the movement of the point of contact being represented by a series of contact data. These operations are optionally applied to single point contacts (e.g., single finger contacts or stylus contacts) or multiple simultaneous contacts (e.g., "multi-touch"/multi-finger contacts). In some embodiments, the contact/motion module 130 and the display controller 156 detect contact on the touch panel.

The contact/motion module 130 optionally detects gesture input by the user. Different gestures on the touch-sensitive surface have different contact patterns (e.g., different motions, timings, and/or intensities of detected contacts). Thus, the gesture is optionally detected by detecting a particular contact pattern. For example, detecting a single-finger tap gesture includes detecting a finger-down event, and then detecting a finger-up (lift-off) event at the same location (or substantially the same location) as the finger-down event (e.g., at an icon location). As another example, detecting a finger swipe gesture on the touch-sensitive surface includes detecting a finger-down event, then detecting one or more finger-dragging events, and then subsequently detecting a finger-up (lift-off) event. Similarly, taps, swipes, drags, and other gestures of the stylus are optionally detected by detecting a particular contact pattern of the stylus.

In some embodiments, detecting a finger tap gesture (e.g., on touch-sensitive display system 112) is dependent on a length of time between detecting a finger-down event and a finger-up event, but independent of a finger contact intensity between detecting a finger-down event and a finger-up event. In some embodiments, a flick gesture is detected based on a determination that the length of time between a finger down event and a finger up event is less than a predetermined value (e.g., less than 0.1, 0.2, 0.3, 0.4, or 0.5 seconds), regardless of whether the intensity of the finger contact during the flick reaches a given intensity threshold (greater than a nominal contact detection intensity threshold), such as a light press or a deep press intensity threshold. Thus, the finger tap gesture may satisfy an input criterion that is configured to be satisfied even when the characteristic intensity of the contact does not satisfy a given intensity threshold. For clarity, a finger contact in a flick gesture generally requires that a nominal contact detection intensity threshold below which no contact is detected be met in order to detect a finger press event. A similar analysis applies to detecting flick gestures by a stylus or other contact. Where the device is configured to detect a finger or stylus contact hovering over the touch-sensitive surface, the nominal contact detection intensity threshold optionally does not correspond to a physical contact between the finger or stylus and the touch-sensitive surface.

The same concept applies in a similar manner to other types of gestures. For example, a swipe gesture, a pinch gesture, a zoom-in gesture, and/or a long press gesture are optionally detected (e.g., on touch-sensitive display system 112) based on satisfying criteria independent of the intensity of the contact included in the gesture. For example, a swipe gesture is detected based on the amount of movement of one or more contacts; a zoom gesture is detected based on movement of two or more contacts toward each other; a magnification gesture is detected based on movement of two or more contacts away from each other; the long press gesture is detected based on a duration of contact on the touch-sensitive surface having less than a threshold amount of movement. Thus, an expression that the gesture recognition criteria are configured to be met by the criteria when the contact in the gesture has an intensity below the respective intensity threshold means that the gesture recognition criteria can be met even if the contact in the gesture does not reach the respective intensity threshold. It should be understood, however, that this expression does not preclude the gesture recognition criteria from being met if one or more contacts in the gesture meet or exceed respective intensity thresholds. For example, a flick gesture is configured to be detected if finger-down and finger-up events are detected within a predefined time period, regardless of whether the contact is above or below a respective intensity threshold during the predefined time period, and a swipe gesture is configured to be detected if the contact movement is greater than a predefined magnitude, even if the contact is above the respective intensity threshold at the end of the contact movement.

In some cases, the contact intensity threshold, the duration threshold, and the movement threshold are combined in various different combinations in order to create a heuristic algorithm to distinguish two or more different gestures for the same input element or region, such that multiple different interactions with the same input element can provide a richer set of user interactions and responses. A particular set of gesture recognition criteria is configured to qualify as satisfying such an expression when a contact in the gesture has an intensity below a respective intensity threshold and to not exclude other intensity-related gesture recognition criteria from being evaluated at the same time to identify other gestures that do qualify as satisfying the criteria when the gesture includes a contact having an intensity above the respective intensity threshold. For example, in some cases, a first gesture recognition criterion of a first gesture (which is configured to be satisfied when the gesture has an intensity below a respective intensity threshold) competes with a second gesture recognition criterion of a second gesture (which depends on the gesture reaching the respective intensity threshold). In such a competition, if the second gesture recognition criteria of the second gesture are first satisfied, the gesture is optionally not recognized as satisfying the first gesture recognition criteria of the first gesture. For example, if the contact reaches a respective intensity threshold before the contact moves by a predefined amount of movement, a deep press gesture is detected instead of a swipe gesture. Conversely, if the contact moves by a predefined amount of movement before the contact reaches the respective intensity threshold, a swipe gesture is detected instead of a deep press gesture. Even in this case, the first gesture recognition criteria of the first gesture are still configured to be satisfied when the contact in the gesture has an intensity below the respective intensity threshold, because if the contact remains below the respective intensity threshold until the gesture ends (e.g., a swipe gesture with a contact that does not increase to an intensity above the respective intensity threshold), the gesture will have been recognized by the first gesture recognition criteria as a swipe gesture. Thus, a particular gesture recognition criterion that is configured to be satisfied when the intensity of the contact remains below a respective intensity threshold will (a) in some cases ignore the intensity of the contact relative to the intensity threshold (e.g., for a flick gesture) and/or (B) in some cases still depend on the intensity of the contact relative to the intensity threshold in the sense that: if a competing set of intensity-related gesture recognition criteria (e.g., for a deep press gesture) recognizes an input as corresponding to an intensity-related gesture before the particular gesture recognition criteria recognizes a gesture corresponding to the input, the particular gesture recognition criteria (e.g., for a long press gesture) will fail.

Graphics module 132 includes various known software components for rendering and displaying graphics on touch-sensitive display system 112 or other displays, including components for changing the visual impact (e.g., brightness, transparency, saturation, contrast, or other visual properties) of displayed graphics. As used herein, the term "graphic" includes any object that may be displayed to a user, including without limitation text, web pages, icons (such as user interface objects including soft keys), digital images, videos, animations and the like.

In some embodiments, graphics module 132 stores data representing graphics to be used. Each graphic is optionally assigned a corresponding code. The graphic module 132 receives one or more codes for specifying a graphic to be displayed, if necessary, coordinate data and other graphic attribute data from an application program or the like, and then generates screen image data to output to the display controller 156.

The haptic feedback module 133 includes various software components for generating instructions for use by the haptic output generator 167 to produce haptic outputs at one or more locations on the device 100 in response to user interaction with the device 100.

Text input module 134, which is optionally a component of graphics module 132, provides a soft keyboard for entering text in various applications such as contacts 137, email 140, IM 141, browser 147, and any other application that requires text input.

The GPS module 135 determines the location of the device and provides such information for use in various applications (e.g., to the phone 138 for location-based dialing; to the camera 143 as picture/video metadata; and to applications that provide location-based services such as weather desktop widgets, local yellow pages desktop widgets, and map/navigation desktop widgets).

Application 136 optionally includes the following modules (or sets of instructions), or a subset or superset thereof:

contacts module 137 (sometimes referred to as an address book or contact list);

a phone module 138;

a video conferencing module 139;

an email client module 140;

an Instant Messaging (IM) module 141;

fitness support module 142;

camera module 143 for still and/or video images;

an image management module 144;

a browser module 147;

a calendar module 148;

desktop applet module 149, optionally including one or more of: a weather desktop applet 149-1, a stock market desktop applet 149-2, a calculator desktop applet 149-3, an alarm desktop applet 149-4, a dictionary desktop applet 149-5 and other desktop applets obtained by the user, and a user created desktop applet 149-6;

A desktop applet creator module 150 for forming a user-created desktop applet 149-6;

a search module 151;

a video and music player module 152, optionally consisting of a video player module and a music player module;

a notepad module 153;

a map module 154; and/or

Online video module 155.

Examples of other applications 136 optionally stored in memory 102 include other word processing applications, other image editing applications, drawing applications, rendering applications, JAVA-enabled applications, encryption, digital rights management, voice recognition, and voice replication.

In conjunction with touch-sensitive display system 112, display controller 156, contact module 130, graphics module 132, and text input module 134, contacts module 137 includes executable instructions for managing contact lists or contact lists (e.g., stored in memory 102 or in an application internal state 192 of contacts module 137 in memory 370), including: adding a name to the address book; deleting names from the address book; associating a phone number, email address, physical address, or other information with a name; associating the image with a name; classifying and classifying names; providing a telephone number and/or email address to initiate and/or facilitate communication via telephone 138, video conference 139, email 140, or instant message 141; and so on.

In conjunction with RF circuitry 108, audio circuitry 110, speaker 111, microphone 113, touch-sensitive display system 112, display controller 156, contact module 130, graphics module 132, and text input module 134, phone module 138 includes executable instructions for: entering a sequence of characters corresponding to a telephone number, accessing one or more telephone numbers in the address book 137, modifying the entered telephone number, dialing a corresponding telephone number, conducting a conversation, and disconnecting or hanging up when the conversation is completed. As noted above, the wireless communication optionally uses any of a variety of communication standards, protocols, and technologies.

In conjunction with RF circuitry 108, audio circuitry 110, speaker 111, microphone 113, touch-sensitive display system 112, display controller 156, one or more optical sensors 164, optical sensor controller 158, contact module 130, graphics module 132, text input module 134, contact list 137, and telephony module 138, video conference module 139 includes executable instructions to initiate, conduct, and terminate video conferences between the user and one or more other participants according to user instructions.

In conjunction with RF circuitry 108, touch-sensitive display system 112, display controller 156, contact module 130, graphics module 132, and text input module 134, email client module 140 includes executable instructions for creating, sending, receiving, and managing emails in response to user instructions. In conjunction with the image management module 144, the email client module 140 makes it very easy to create and send an email with a still image or a video image captured by the camera module 143.

In conjunction with RF circuitry 108, touch-sensitive display system 112, display controller 156, contact module 130, graphics module 132, and text input module 134, instant messaging module 141 includes executable instructions for: entering a sequence of characters corresponding to an instant message, modifying previously entered characters, sending a corresponding instant message (e.g., using a Short Message Service (SMS) or Multimedia Message Service (MMS) protocol for telephone-based instant messages or using XMPP, SIMPLE, apple Push Notification Services (APNs) or IMPS for internet-based instant messages), receiving an instant message, and viewing the received instant message. In some embodiments, the transmitted and/or received instant messages optionally include graphics, photos, audio files, video files, and/or MMS and/or other attachments supported in an Enhanced Messaging Service (EMS). As used herein, "instant messaging" refers to both telephony-based messages (e.g., messages sent using SMS or MMS) and internet-based messages (e.g., messages sent using XMPP, SIMPLE, APNs, or IMPS).

In conjunction with RF circuitry 108, touch-sensitive display system 112, display controller 156, contact module 130, graphics module 132, text input module 134, GPS module 135, map module 154, and music player module 152, workout support module 142 includes executable instructions for creating a workout (e.g., having time, distance, and/or calorie burning goals); communicating with fitness sensors (in sports equipment and smart watches); receiving fitness sensor data; calibrating a sensor for monitoring fitness; selecting and playing music for fitness; and displaying, storing and transmitting fitness data.

In conjunction with touch-sensitive display system 112, display controller 156, one or more optical sensors 164, optical sensor controller 158, contact module 130, graphics module 132, and image management module 144, camera module 143 includes executable instructions to: capturing still images or video (including video streams) and storing them in the memory 102, modifying features of the still images or video, and/or deleting the still images or video from the memory 102.

In conjunction with touch-sensitive display system 112, display controller 156, contact module 130, graphics module 132, text input module 134, and camera module 143, image management module 144 includes executable instructions for arranging, modifying (e.g., editing), or otherwise manipulating, labeling, deleting, presenting (e.g., in a digital slide or album), and storing still and/or video images.

In conjunction with RF circuitry 108, touch-sensitive display system 112, display system controller 156, contact module 130, graphics module 132, and text input module 134, browser module 147 includes executable instructions to browse the internet (including searching, linking to, receiving, and displaying web pages or portions thereof, and attachments and other files linked to web pages) in accordance with user instructions.

In conjunction with RF circuitry 108, touch-sensitive display system 112, display system controller 156, contact module 130, graphics module 132, text input module 134, email client module 140, and browser module 147, calendar module 148 includes executable instructions for creating, displaying, modifying, and storing calendars and data associated with calendars (e.g., calendar entries, to-do items, etc.) according to user instructions.

In conjunction with RF circuitry 108, touch-sensitive display system 112, display system controller 156, contact module 130, graphics module 132, text input module 134, and browser module 147, the desktop applet module 149 is a mini-application (e.g., weather desktop applet 149-1, stock market desktop applet 149-2, calculator desktop applet 149-3, alarm clock desktop applet 149-4, and dictionary desktop applet 149-5) that is optionally downloaded and used by the user, or a mini-application created by the user (e.g., user-created desktop applet 149-6). In some embodiments, the desktop applet includes an HTML (hypertext markup language) file, a CSS (cascading style sheet) file, and a JavaScript file. In some embodiments, the desktop applet includes an XML (extensible markup language) file and a JavaScript file (e.g., yahoo! desktop applet).

In conjunction with RF circuitry 108, touch-sensitive display system 112, display system controller 156, contact module 130, graphics module 132, text input module 134, and browser module 147, desktop applet creator module 150 includes executable instructions for creating a desktop applet (e.g., turning a user-specified portion of a web page into a desktop applet).

In conjunction with touch-sensitive display system 112, display system controller 156, contact module 130, graphics module 132, and text input module 134, search module 151 includes executable instructions for searching memory 102 for text, music, sound, images, video, and/or other files that match one or more search criteria (e.g., one or more user-specified search terms) in accordance with user instructions.

In conjunction with touch-sensitive display system 112, display system controller 156, contact module 130, graphics module 132, audio circuitry 110, speakers 111, RF circuitry 108, and browser module 147, video and music player module 152 includes executable instructions to allow a user to download and play back recorded music and other sound files stored in one or more file formats, such as MP3 or AAC files, as well as executable instructions for displaying, rendering, or otherwise playing back video (e.g., on touch-sensitive display system 112 or on an external display wirelessly connected via external port 124). In some embodiments, the device 100 optionally includes the functionality of an MP3 player, such as an iPod (trademark of Apple inc.).

In conjunction with touch-sensitive display system 112, display controller 156, contact module 130, graphics module 132, and text input module 134, notepad module 153 includes executable instructions for creating and managing notepads, to-do-things, and the like, according to user instructions.

In conjunction with RF circuitry 108, touch-sensitive display system 112, display system controller 156, contact module 130, graphics module 132, text input module 134, GPS module 135, and browser module 147, map module 154 includes executable instructions for receiving, displaying, modifying, and storing maps and data associated with maps (e.g., driving routes; data for stores and other points of interest at or near a particular location; and other location-based data) according to user instructions.

In conjunction with touch-sensitive display system 112, display system controller 156, contact module 130, graphics module 132, audio circuit 110, speaker 111, RF circuit 108, text input module 134, email client module 140, and browser module 147, online video module 155 includes executable instructions that allow a user to access, browse, receive (e.g., by streaming and/or downloading), play back (e.g., on touch screen 112 or on an external display that is wirelessly connected or connected via external port 124), send an email with a link to a particular online video, and otherwise manage online video in one or more file formats, such as h.264. In some embodiments, the link to the particular online video is sent using the instant messaging module 141 instead of the email client module 140.

Each of the modules and applications identified above corresponds to a set of executable instructions for performing one or more of the functions described above as well as methods described in this application (e.g., computer-implemented methods and other information processing methods described herein). These modules (i.e., sets of instructions) need not be implemented as separate software programs, procedures or modules, and thus various subsets of these modules are optionally combined or otherwise rearranged in various embodiments. In some embodiments, memory 102 optionally stores a subset of the modules and data structures described above. Further, memory 102 optionally stores additional modules and data structures not described above.

In some embodiments, device 100 is a device in which the operation of a predefined set of functions on the device is performed exclusively through a touch screen and/or a touchpad. By using a touch screen and/or trackpad as the primary input control device for operating the device 100, the number of physical input control devices (e.g., push buttons, dials, etc.) on the device 100 is optionally reduced.

The set of predefined functions performed exclusively by the touchscreen and/or trackpad optionally includes navigating between user interfaces. In some embodiments, the touchpad, when touched by a user, navigates device 100 from any user interface displayed on device 100 to a main, home, or root menu. In such embodiments, a touchpad is used to implement a "menu button". In some other embodiments, the menu button is a physical push button or other physical input control device rather than a touchpad.

Fig. 1B is a block diagram illustrating exemplary components for event processing, according to some embodiments. In some embodiments, memory 102 (in FIG. 1A) or memory 370 (FIG. 3) includes event classifier 170 (e.g., in operating system 126) and corresponding application 136-1 (e.g., any of the aforementioned applications 136, 137-155, 380-390).

Event sorter 170 receives the event information and determines application 136-1 and application view 191 of application 136-1 to which the event information is to be delivered. The event sorter 170 includes an event monitor 171 and an event dispatcher module 174. In some embodiments, application 136-1 includes an application internal state 192 that indicates one or more current application views that are displayed on touch-sensitive display system 112 when the application is active or executing. In some embodiments, device/global internal state 157 is used by event classifier 170 to determine which application(s) are currently active, and application internal state 192 is used by event classifier 170 to determine the application view 191 to which to deliver event information.

In some embodiments, the application internal state 192 includes additional information, such as one or more of: resume information to be used when the application 136-1 resumes execution, user interface state information indicating information being displayed by the application 136-1 or information that is ready for display by the application, a state queue for enabling a user to return to a previous state or view of the application 136-1, and a repeat/undo queue of previous actions taken by the user.

Event monitor 171 receives event information from peripheral interface 118. The event information includes information about a sub-event (e.g., a user touch on touch-sensitive display system 112 as part of a multi-touch gesture). Peripherals interface 118 transmits information that it receives from I/O subsystem 106 or sensors such as proximity sensor 166, accelerometer 168, and/or microphone 113 (through audio circuitry 110). Information received by peripherals interface 118 from I/O subsystem 106 includes information from touch-sensitive display system 112 or a touch-sensitive surface.

In some embodiments, event monitor 171 sends requests to peripheral interface 118 at predetermined intervals. In response, peripheral interface 118 transmits the event information. In other embodiments, peripheral interface 118 transmits event information only when there is a significant event (e.g., receiving input above a predetermined noise threshold and/or receiving input for more than a predetermined duration).

In some embodiments, event classifier 170 further includes hit view determination module 172 and/or active event recognizer determination module 173.

When touch-sensitive display system 112 displays more than one view, hit view determination module 172 provides a software process for determining where within one or more views a sub-event has occurred. The view consists of controls and other elements that the user can see on the display.

Another aspect of the user interface associated with an application is a set of views, sometimes referred to herein as application views or user interface windows, in which information is displayed and touch-based gestures occur. The application view (of the respective application) in which the touch is detected optionally corresponds to a programmatic level within a programmatic or view hierarchy of applications. For example, the lowest level view in which a touch is detected is optionally referred to as a hit view, and the set of events identified as correct inputs is optionally determined based at least in part on the hit view of the initial touch that initiated the touch-based gesture.

Hit view determination module 172 receives information related to sub-events of the touch-based gesture. When the application has multiple views organized in a hierarchy, the hit view determination module 172 identifies the hit view as the lowest view in the hierarchy that should handle the sub-event. In most cases, the hit view is the lowest level view in which the initiating sub-event (i.e., the first sub-event in the sequence of sub-events that form an event or potential event) occurs. Once a hit view is identified by the hit view determination module, the hit view typically receives all sub-events related to the same touch or input source for which it was identified as the hit view.

The active event recognizer determination module 173 determines which view or views within the view hierarchy should receive a particular sequence of sub-events. In some implementations, the activity event identifier determination module 173 determines that only the hit view should receive a particular sequence of sub-events. In other embodiments, active event recognizer determination module 173 determines that all views that include the physical location of the sub-event are actively involved views, and thus determines that all actively involved views should receive a particular sequence of sub-events. In other embodiments, even if the touch sub-event is completely confined to the area associated with a particular view, the higher views in the hierarchy will remain actively participating views.

The event dispatcher module 174 dispatches the event information to an event recognizer (e.g., event recognizer 180). In embodiments that include active event recognizer determination module 173, event dispatcher module 174 delivers event information to event recognizers determined by active event recognizer determination module 173. In some embodiments, the event dispatcher module 174 stores event information in an event queue, which is retrieved by the respective event receiver module 182.

In some embodiments, the operating system 126 includes an event classifier 170. Alternatively, application 136-1 includes event classifier 170. In further embodiments, the event classifier 170 is a stand-alone module or is part of another module stored in the memory 102 (such as the contact/motion module 130).

In some embodiments, the application 136-1 includes a plurality of event handlers 190 and one or more application views 191, where each application view includes instructions for handling touch events occurring within a respective view of the application's user interface. Each application view 191 of the application 136-1 includes one or more event recognizers 180. Typically, the respective application view 191 includes a plurality of event recognizers 180. In other embodiments, one or more of the event recognizers 180 are part of a separate module, such as a user interface toolkit (not shown) or a higher level object from which the application 136-1 inherits methods and other properties. In some embodiments, the respective event handlers 190 comprise one or more of: data updater 176, object updater 177, GUI updater 178, and/or event data 179 received from event sorter 170. Event handler 190 optionally utilizes or calls data updater 176, object updater 177 or GUI updater 178 to update application internal state 192. Alternatively, one or more of the application views 191 include one or more respective event handlers 190. Additionally, in some embodiments, one or more of data updater 176, object updater 177, and GUI updater 178 are included in a respective application view 191.

The corresponding event recognizer 180 receives event information (e.g., event data 179) from the event classifier 170 and recognizes events from the event information. The event recognizer 180 includes an event receiver 182 and an event comparator 184. In some embodiments, event recognizer 180 also includes metadata 183 and at least a subset of event delivery instructions 188 (which optionally include sub-event delivery instructions).

The event receiver 182 receives event information from the event sorter 170. The event information includes information about a sub-event such as a touch or touch movement. According to the sub-event, the event information further includes additional information, such as the location of the sub-event. When the sub-event relates to motion of a touch, the event information optionally also includes the velocity and direction of the sub-event. In some embodiments, the event comprises rotation of the device from one orientation to another (e.g., from a portrait orientation to a landscape orientation, or vice versa), and the event information comprises corresponding information about the current orientation of the device (also referred to as the device pose).

Event comparator 184 compares the event information to predefined event or sub-event definitions and determines an event or sub-event or determines or updates the state of an event or sub-event based on the comparison. In some embodiments, event comparator 184 includes event definitions 186. Event definition 186 contains definitions of events (e.g., predefined sub-event sequences), such as event 1 (187-1), event 2 (187-2), and other events. In some embodiments, sub-events in event 187 include, for example, touch start, touch end, touch move, touch cancel, and multi-touch. In one example, the definition for event 1 (187-1) is a double click on the displayed object. For example, a double tap includes a first touch (touch start) on the displayed object for a predetermined length of time, a first lift-off (touch end) for a predetermined length of time, a second touch (touch start) on the displayed object for a predetermined length of time, and a second lift-off (touch end) for a predetermined length of time. In another example, the definition for event 2 (187-2) is a drag on the displayed object. For example, dragging includes a predetermined length of time of touch (or contact) on a displayed object, movement of the touch across touch-sensitive display system 112, and liftoff of the touch (touch end). In some embodiments, an event also includes information for one or more associated event handlers 190.

In some embodiments, event definition 187 includes definitions of events for respective user interface objects. In some embodiments, event comparator 184 performs a hit test to determine which user interface object is associated with a sub-event. For example, in an application view that displays three user interface objects on touch-sensitive display system 112, when a touch is detected on touch-sensitive display system 112, event comparator 184 performs a hit-test to determine which of the three user interface objects is associated with the touch (sub-event). If each displayed object is associated with a corresponding event handler 190, the event comparator uses the results of the hit test to determine which event handler 190 should be activated. For example, event comparator 184 selects the event handler associated with the sub-event and the object that triggered the hit test.

In some embodiments, the definition of the respective event 187 further includes a delay action that delays delivery of the event information until it has been determined whether the sequence of sub-events does or does not correspond to the event type of the event recognizer.

When the respective event recognizer 180 determines that the sequence of sub-events does not match any event in the event definition 186, the respective event recognizer 180 enters an event not possible, event failed, or event ended state, after which subsequent sub-events of the touch-based gesture are ignored. In this case, the other event recognizers (if any) that remain active for the hit view continue to track and process sub-events of the ongoing touch-based gesture.

In some embodiments, the respective event recognizer 180 includes metadata 183 with configurable attributes, tags, and/or lists that indicate how the event delivery system should perform sub-event delivery to actively participating event recognizers. In some embodiments, metadata 183 includes configurable attributes, flags, and/or lists that indicate how event recognizers interact or are able to interact with each other. In some embodiments, metadata 183 includes configurable attributes, flags, and/or lists that indicate whether a sub-event is delivered to a different level in the view or programmatic hierarchy.

In some embodiments, when one or more particular sub-events of an event are identified, the respective event identifier 180 activates an event handler 190 associated with the event. In some embodiments, the respective event identifier 180 delivers event information associated with the event to event handler 190. Activating event handler 190 is different from sending (and deferring) sub-events to the corresponding hit view. In some embodiments, the event recognizer 180 throws a flag associated with the recognized event, and the event handler 190 associated with the flag catches the flag and performs a predefined process.

In some embodiments, the event delivery instructions 188 include sub-event delivery instructions that deliver event information about sub-events without activating an event handler. Instead, the sub-event delivery instructions deliver event information to event handlers associated with the series of sub-events or to actively participating views. Event handlers associated with the series of sub-events or with actively participating views receive the event information and perform a predetermined process.

In some embodiments, data updater 176 creates and updates data used in application 136-1. For example, the data updater 176 updates a phone number used in the contacts module 137 or stores a video file used in the video player module 152. In some embodiments, object updater 177 creates and updates objects used in application 136-1. For example, the object updater 177 creates a new user interface object or updates the location of a user interface object. GUI updater 178 updates the GUI. For example, GUI updater 178 prepares display information and sends the display information to graphics module 132 for display on the touch-sensitive display.

In some embodiments, event handler 190 includes, or has access to, data updater 176, object updater 177, and GUI updater 178. In some embodiments, data updater 176, object updater 177, and GUI updater 178 are included in a single module of a respective application 136-1 or application view 191. In other embodiments, they are included in two or more software modules.

It should be understood that the above discussion of event processing with respect to user touches on a touch sensitive display also applies to other forms of user input utilizing an input device to operate multifunction device 100, not all of which are initiated on a touch screen. For example, mouse movements and mouse button presses, optionally in conjunction with single or multiple keyboard presses or holds; contact movements on the touchpad, such as taps, drags, scrolls, and the like; inputting by a stylus; movement of the device; verbal instructions; detected eye movement; inputting biological characteristics; and/or any combination thereof, is optionally used as input corresponding to sub-events defining the event to be identified.

Fig. 1C is a block diagram illustrating a haptic output module according to some embodiments. In some embodiments, the I/O subsystem 106, such as the tactile feedback controller 161 (fig. 1A) and/or other input controller 160 (fig. 1A), includes at least some of the exemplary components shown in fig. 1C. In some embodiments, peripheral interface 118 includes at least some of the exemplary components shown in fig. 1C.

In some embodiments, the haptic output module includes a haptic feedback module 133. In some embodiments, the tactile feedback module 133 aggregates and combines tactile output from user interface feedback from software applications on the electronic device (e.g., feedback responsive to user input corresponding to a displayed user interface and prompts and other notifications indicating performance of operations or occurrence of events in the user interface of the electronic device). The haptic feedback module 133 includes one or more of a waveform module 123 (for providing a waveform for generating a haptic output), a mixer 125 (for mixing waveforms, such as waveforms in different channels), a compressor 127 (for reducing or compressing the dynamic range of the waveform), a low pass filter 129 (for filtering out high frequency signal components in the waveform), and a thermal controller 131 (for adjusting the waveform according to thermal conditions). In some embodiments, the tactile feedback module 133 is included in the tactile feedback controller 161 (fig. 1A). In some embodiments, a separate unit of the haptic feedback module 133 (or a separate implementation of the haptic feedback module 133) is also included in the audio controller (e.g., the audio circuit 110, fig. 1A) and used to generate the audio signal. In some embodiments, a single haptic feedback module 133 is used to generate the audio signal and to generate the waveform of the haptic output.

In some embodiments, the haptic feedback module 133 also includes a trigger module 121 (e.g., a software application, operating system, or other software module that determines that a haptic output is to be generated and initiates a process for generating a corresponding haptic output). In some embodiments, the trigger module 121 generates a trigger signal for causing (e.g., by the waveform module 123) the generation of the waveform. For example, the trigger module 121 generates a trigger signal based on a preset timing criterion. In some embodiments, the trigger module 121 receives a trigger signal from outside the tactile feedback module 133 (e.g., in some embodiments, the tactile feedback module 133 receives a trigger signal from a hardware input processing module 146 located outside the tactile feedback module 133) and relays the trigger signal to other components within the tactile feedback module 133 (e.g., the waveform module 123) or software applications that trigger operations (with the trigger module 121) based on activation of a hardware input device (e.g., a primary button). In some embodiments, the trigger module 121 also receives haptic feedback generation instructions (e.g., from the haptic feedback module 133, fig. 1A and 3). In some embodiments, the trigger module 121 generates the trigger signal in response to the haptic feedback module 133 (or the trigger module 121 in the haptic feedback module 133) receiving a haptic feedback instruction (e.g., from the haptic feedback module 133, fig. 1A and 3).

Waveform module 123 receives as input a trigger signal (e.g., from trigger module 121) and provides waveforms for generating one or more tactile outputs (e.g., waveforms selected from a predefined set of waveforms assigned for use by waveform module 123, such as the waveforms described in more detail below with reference to fig. 4F-4G) in response to receiving the trigger signal.

The mixer 125 receives waveforms as input (e.g., from the waveform module 123) and mixes the waveforms together. For example, when the mixer 125 receives two or more waveforms (e.g., a first waveform in a first channel and a second waveform in a second channel that at least partially overlaps the first waveform), the mixer 125 outputs a combined waveform corresponding to the sum of the two or more waveforms. In some embodiments, the mixer 125 also modifies one or more of the two or more waveforms to emphasize a particular waveform relative to the rest of the two or more waveforms (e.g., by increasing the scale of the particular waveform and/or decreasing the scale of the other of the waveforms). In some cases, mixer 125 selects one or more waveforms to remove from the combined waveform (e.g., when waveforms from more than three sources have been requested to be output by tactile output generator 167 simultaneously, the waveform from the oldest source is discarded).

The mixer 127 receives as input waveforms, such as a combined waveform from the mixer 125, and modifies the waveforms. In some embodiments, compressor 127 reduces the waveforms (e.g., according to the physical specifications of tactile output generator 167 (fig. 1A) or 357 (fig. 3)) such that the tactile outputs corresponding to the waveforms are reduced. In some embodiments, the compressor 127 limits the waveform, such as by imposing a predefined maximum amplitude for the waveform. For example, the compressor 127 reduces the amplitude of the waveform portions that exceed a predefined amplitude threshold, while maintaining the amplitude of the waveform portions that do not exceed the predefined amplitude threshold. In some implementations, the compressor 127 reduces the dynamic range of the waveform. In some embodiments, compressor 127 dynamically reduces the dynamic range of the waveform such that the combined waveform remains within the performance specifications (e.g., force and/or movable mass displacement limits) of tactile output generator 167.

Low pass filter 129 receives as input a waveform (e.g., a compressed waveform from compressor 127) and filters (e.g., smoothes) the waveform (e.g., removes or reduces high frequency signal components in the waveform). For example, in some cases, compressor 127 includes extraneous signals (e.g., high frequency signal components) in the compressed waveform that prevent haptic output generation and/or exceed the performance specifications of haptic output generator 167 in generating haptic output from the compressed waveform. Low pass filter 129 reduces or removes such extraneous signals in the waveform.

Thermal controller 131 receives as input a waveform (e.g., a filtered waveform from low pass filter 129) and adjusts the waveform according to a thermal condition of apparatus 100 (e.g., based on an internal temperature detected within apparatus 100, such as a temperature of tactile feedback controller 161, and/or an external temperature detected by apparatus 100). For example, in some cases, the output of the tactile feedback controller 161 varies as a function of temperature (e.g., in response to receiving the same waveform, the tactile feedback controller 161 generates a first tactile output when the tactile feedback controller 161 is at a first temperature and a second tactile output when the tactile feedback controller 161 is at a second temperature different from the first temperature). For example, the magnitude of the haptic output may vary as a function of temperature. To reduce the effects of temperature variations, the waveform is modified (e.g., the amplitude of the waveform is increased or decreased based on temperature).

In some embodiments, the haptic feedback module 133 (e.g., trigger module 121) is coupled to the hardware input processing module 146. In some embodiments, the other input controller 160 in fig. 1A includes a hardware input processing module 146. In some embodiments, the hardware input processing module 146 receives input from a hardware input device 145 (e.g., the other input or control device 116 in fig. 1A, such as a home button). In some embodiments, the hardware input device 145 is any input device described herein, such as the touch-sensitive display system 112 (fig. 1A), the keyboard/mouse 350 (fig. 3), the touchpad 355 (fig. 3), one of the other input or control devices 116 (fig. 1A), or an intensity-sensitive primary button (e.g., a primary button with a mechanical actuator as shown in fig. 2B or fig. 2C). In some embodiments, hardware input device 145 is comprised of an intensity-sensitive primary button (e.g., a primary button with a mechanical actuator as shown in fig. 2B, or fig. 2C) rather than a touch-sensitive display system 112 (fig. 1A), a keyboard/mouse 350 (fig. 3), or a touchpad 355 (fig. 3). In some embodiments, in response to input from the hardware input device 145, the hardware input processing module 146 provides one or more trigger signals to the tactile feedback module 133 to indicate that a user input has been detected that meets predefined input criteria, such as an input corresponding to a primary button "click" (e.g., "press click" or "release click"). In some embodiments, the tactile feedback module 133 provides a waveform corresponding to the primary button "click" in response to an input corresponding to the primary button "click" to simulate tactile feedback of pressing a physical primary button.

In some embodiments, the haptic output module includes a haptic feedback controller 161 (e.g., haptic feedback controller 161 in fig. 1A) that controls the generation of the haptic output. In some embodiments, the tactile feedback controller 161 is coupled to a plurality of tactile output generators and selects one or more of the plurality of tactile output generators and sends a waveform to the selected one or more tactile output generators for generating a tactile output. In some embodiments, the haptic feedback controller 161 coordinates haptic output requests corresponding to activating the hardware input device 145 and haptic output requests corresponding to software events (e.g., haptic output requests from the haptic feedback module 133) and modifies one or more of the two or more waveforms to emphasize a particular waveform relative to the rest of the two or more waveforms (e.g., by increasing the scale of the particular waveform and/or decreasing the scale of the rest of the waveforms to preferentially process haptic output corresponding to activating the hardware input device 145 over haptic output corresponding to software events).

In some embodiments, as shown in fig. 1C, the output of the tactile feedback controller 161 is coupled to an audio circuit (e.g., audio circuit 110, fig. 1A) of the device 100 and provides an audio signal to the audio circuit of the device 100. In some embodiments, the tactile feedback controller 161 provides both a waveform for generating a tactile output and an audio signal for providing an audio output in conjunction with generating the tactile output. In some embodiments, the tactile feedback controller 161 modifies the audio signal and/or waveform (used to generate the haptic output) such that the audio output and the haptic output are synchronized (e.g., by delaying the audio signal and/or waveform). In some embodiments, the tactile feedback controller 161 includes a digital-to-analog converter for converting a digital waveform to an analog signal, which is received by the amplifier 163 and/or the haptic output generator 167.

In some embodiments, the haptic output module includes an amplifier 163. In some embodiments, amplifier 163 receives a waveform (e.g., from tactile feedback controller 161) and amplifies the waveform and then sends the amplified waveform to tactile output generator 167 (e.g., either tactile output generator 167 (fig. 1A) or 357 (fig. 3)). For example, amplifier 163 amplifies the received waveform to a signal level that meets the physical specifications of tactile output generator 167 (e.g., to a voltage and/or current required by tactile output generator 167 to generate a tactile output such that the signal sent to tactile output generator 167 generates a tactile output corresponding to the waveform received from tactile feedback controller 161) and sends the amplified waveform to tactile output generator 167. In response, tactile output generator 167 generates a tactile output (e.g., by moving the movable mass back and forth in one or more dimensions relative to a neutral position of the movable mass).

In some embodiments, the haptic output module includes a sensor 169 coupled to a haptic output generator 167. Sensor 169 detects a state or change in state (e.g., mechanical position, physical displacement, and/or movement) of tactile output generator 167 or one or more components of tactile output generator 167 (e.g., one or more moving components, such as a membrane, used to generate tactile output). In some embodiments, the sensor 169 is a magnetic field sensor (e.g., a hall effect sensor) or other displacement and/or motion sensor. In some embodiments, sensor 169 provides information (e.g., the position, displacement, and/or movement of one or more components in tactile output generator 167) to tactile feedback controller 161, and tactile feedback controller 161 adjusts the waveform output from tactile feedback controller 161 (e.g., optionally the waveform sent to tactile output generator 167 via amplifier 163) based on the information provided by sensor 169 regarding the status of tactile output generator 167.

Fig. 2A illustrates a portable multifunction device 100 with a touch screen (e.g., touch-sensitive display system 112, fig. 1A) in accordance with some embodiments. The touch screen optionally displays one or more graphics within the User Interface (UI) 200. In these embodiments, as well as other embodiments described below, a user can select one or more of these graphics by making gestures on the graphics, for example, with one or more fingers 202 (not drawn to scale in the figure) or one or more styluses 203 (not drawn to scale in the figure). In some embodiments, selection of one or more graphics will occur when the user breaks contact with the one or more graphics. In some embodiments, the gesture optionally includes one or more taps, one or more swipes (left to right, right to left, up, and/or down), and/or a rolling of a finger (right to left, left to right, up, and/or down) that has made contact with device 100. In some implementations, or in some cases, inadvertent contact with a graphic does not select the graphic. For example, a swipe gesture that sweeps over an application icon optionally does not select the corresponding application when the gesture corresponding to the selection is a tap.

Device 100 optionally also includes one or more physical buttons, such as a "home" button, or menu button 204. As previously described, menu button 204 is optionally used to navigate to any application 136 in a set of applications that are optionally executed on device 100. Alternatively, in some embodiments, the menu button is implemented as a soft key in a GUI displayed on the touch screen display.

In some embodiments, device 100 includes a touch screen display, a menu button 204 (sometimes referred to as a main button 204), a push button 206 for powering the device on/off and for locking the device, a volume adjustment button 208, a Subscriber Identity Module (SIM) card slot 210, a headset jack 212, and a docking/charging external port 124. Pressing the button 206 optionally serves to turn the device on/off by pressing the button and holding the button in a pressed state for a predefined time interval; locking the device by depressing the button and releasing the button before the predefined time interval has elapsed; and/or unlocking the device or initiating an unlocking process. In some embodiments, device 100 also accepts voice input through microphone 113 for activating or deactivating certain functions. Device 100 also optionally includes one or more contact intensity sensors 165 for detecting the intensity of contacts on touch-sensitive display system 112, and/or one or more tactile output generators 167 for generating tactile outputs for a user of device 100.

Fig. 2B-2C illustrate exploded views of a first input device (e.g., as a primary button 204) suitable for use in the electronic device illustrated in fig. 1A, 2A, 3, and/or 4A. FIG. 2B illustrates an example of an intensity-sensitive primary button having a capacitive sensor for determining a range of intensity values corresponding to a force applied to the intensity-sensitive primary button. Fig. 2C shows an example of a main button having a mechanical switching element. Referring to fig. 2B, the input device stack 220 includes a cover member 222 and a trim 224. In the illustrated embodiment, the trim 224 completely surrounds the sides of the cover member 222 and the perimeter of the top surface of the cover member 222. Other embodiments are not limited to this configuration. For example, in one embodiment, the sides and/or top surface of cover member 222 can be partially surrounded by trim 224. Alternatively, selvedges 224 may be omitted in other embodiments.

Both cover element 222 and trim 224 can be made of any suitable opaque, transparent, and/or translucent material. For example, cover member 222 may be formed of glass, plastic, or sapphire, while trim 224 may be formed of metal or plastic. In some embodiments, one or more additional layers (not shown) may be located below the cover element 222. For example, when the cover element 222 is formed of a transparent material, an opaque ink layer may be disposed below the cover element 222. The opaque ink layer may hide other components in the input device stack 220 so that the other components cannot be seen through the transparent cover element.

A first circuit layer 226 may be disposed below the cover element 222. Any suitable circuit layer may be used. For example, the first circuit layer 226 may be a circuit board or a flexible circuit. The first circuit layer 226 may include one or more circuits, signal lines, and/or integrated circuits. In one embodiment, the first circuit layer 226 includes a biometric sensor 228. Any suitable type of biometric sensor may be used. For example, in one embodiment, the biometric sensor is a capacitive fingerprint sensor that captures at least one fingerprint when a user's finger approaches and/or contacts the cover element 222.

The first circuit layer 226 may be attached to the bottom surface of the cover element 222 with an adhesive layer 230. Any suitable adhesive may be used for the adhesive layer. For example, a pressure sensitive adhesive layer may be used as the adhesive layer 230.

The compliant layer 232 is disposed below the first circuit layer 226. In one embodiment, the compliant layer 232 includes openings 234 formed in the compliant layer 232. The opening 234 exposes the top surface of the first circuit layer 226 and/or the biometric sensor 228 when the device stack 220 is assembled. In the exemplified embodiment, compliant layer 232 is positioned around the inner perimeter of selvedges 224 and/or around the perimeter edges of cover element 222. Although illustrated in a circular shape, the compliant layer 232 may have any given shape and/or dimension, such as square or oval. The compliant layer 232 is illustrated in fig. 2B and 2C as a continuous compliant layer, but other embodiments are not limited to this configuration. In some embodiments, multiple discrete compliant layers may be used in device stack 220. Further in some embodiments, compliant layer 232 does not include openings 234, and compliant layer 232 extends across at least a portion of input device stack 220. For example, the compliant layer 232 may extend across a bottom surface of the cover element 222, a bottom surface of the first circuit layer 226, or a portion of the bottom surface of the cover element 222 (e.g., around a perimeter edge of the cover element) and the bottom surface of the first circuit layer 226.

The second circuit layer 238 is positioned below the first circuit layer 226. Flex circuits and circuit boards are examples of circuit layers that may be used in the second circuit layer 238. In some embodiments, the second circuit layer 238 may include a first circuit section 240 and a second circuit section 242. The first circuit section 240 and the second circuit section 242 may be electrically connected to each other.

The first circuit section 240 may include a first set of one or more intensity sensor components included in the intensity sensor. In some implementations, the first circuit section 240 can be electrically connected to the first circuit layer 226. For example, when the first circuit layer 226 includes the biometric sensor 228, the biometric sensor 228 may be electrically connected to the first circuit section 240 of the second circuit layer 238.

The second circuit section 242 may include additional circuitry, such as signal lines, circuit components, integrated circuits, and so forth. In one embodiment, the second circuit section 242 may include a board-to-board connector 244 for electrically connecting the second circuit layer 238 to other circuitry in the electronic device. For example, the second circuit layer 238 may be operatively connected to the processing device using a board-to-board connector 244. Additionally or alternatively, the second circuit layer 238 may be operatively connected to circuitry that transmits signals (e.g., sensing signals) received from the intensity sensor components in the first circuit section 240 to a processing device. Additionally or alternatively, the second circuit layer 238 may be operatively connected to circuitry that provides signals (e.g., drive signals, reference signals) to the one or more intensity sensor components in the first circuit section 240.

In some implementations, the first circuit section 240 of the second circuit layer 238 can be attached to the bottom surface of the first circuit layer 226 using an adhesive layer 236. In a non-limiting example, first circuit section 240 may be attached to the bottom surface of first circuit layer 226 using a die attach film.

The third circuit layer 246 is disposed below the first circuit section 240 of the second circuit layer 238. The third circuit layer 246 may include a second set of one or more intensity sensor components included in the intensity sensor. The third circuit layer 246 is supported by the support element 248 and/or attached to the support element 248. In one embodiment, support elements 248 are attached to selvedges 224 to create a housing for other components in device stack 220. Support element 248 may be attached to selvedges 224 using any suitable attachment mechanism.

Together, the first set of one or more intensity sensor components in first circuit section 240 and the second set of one or more intensity sensor components in third circuit layer 246 form an intensity sensor. The intensity sensor may use any suitable intensity sensing technology. Exemplary sensing technologies include, but are not limited to, capacitive, piezoelectric, piezoresistive, ultrasonic, and magnetic.

In the example shown in fig. 2B and 2C, the intensity sensor is a capacitive force sensor. For a capacitive force sensor, the first set of one or more intensity sensor components may comprise a first set of one or more electrodes 250 and the second set of one or more force sensor components comprises a second set of one or more electrodes 252. Although illustrated in a square shape in fig. 2B and 2C, each of the first and second sets of one or

more electrodes

250, 252 may have any given shape (e.g., rectangular, circular). Additionally, the one or more electrodes in the first set 250 and the second set 252 may be arranged in any given pattern (e.g., one or more rows and one or more columns).

Fig. 2B and 2C show two electrodes of the first and second sets of one or

more electrodes

250, 252. However, other embodiments are not limited to this configuration. The first and second sets of one or

more electrodes

250, 252 may each be a single electrode or a plurality of discrete electrodes. For example, if the first set of one or more electrodes is a single electrode, the second set of one or more electrodes includes a plurality of discrete electrodes. In some embodiments, the second set of one or more electrodes may be a single electrode, the first set comprising a plurality of discrete electrodes. Alternatively, the first and second sets of one or more electrodes may each comprise a plurality of discrete electrodes.

Each electrode of the first set of one or more electrodes 250 is aligned in at least one direction (e.g., vertically) with a corresponding electrode of the second set of one or more electrodes 252 to generate one or more capacitors. When a force input is applied to the cover element 222 (e.g., an input surface of an input device), at least one electrode in the first set 250 moves closer to a corresponding electrode in the second set 252, which changes the capacitance of the capacitor. The capacitance signal sensed from each capacitor represents a capacitance measurement for that capacitor. A processing device (not shown) is configured to receive the capacitance signal and correlate the capacitance signal to an amount of intensity applied to the cover element 222. In some embodiments, a force sensor may replace the switching element, and different intensity thresholds may be used to determine the activation event.

In some embodiments, such as the embodiment shown in fig. 2C, the switching element 254 may be located below the support element 248. The switch element 254 registers a user input when a force input applied to the cover element 222 exceeds a given amount of force (e.g., a force threshold from closing the distance between the first circuit section 240 and the third circuit layer 246). Any suitable switching element may be used. For example, the switch element 254 may be a dome switch that collapses when a force input applied to the cover element 222 exceeds a force threshold. Upon collapse, the dome switch completes a circuit that is detected by the processing device and recognized as a user input (e.g., selecting an icon, function, or application). In one embodiment, the dome switch is arranged such that the apex of the collapsible dome is adjacent to the bottom surface of the support plate 248. In another embodiment, the base of the collapsible dome may be adjacent to the bottom surface of the support plate 248.

Fig. 3 is a block diagram of an exemplary multifunction device with a display and a touch-sensitive surface in accordance with some embodiments. The device 300 need not be portable. In some embodiments, the device 300 is a laptop computer, a desktop computer, a tablet computer, a multimedia player device, a navigation device, an educational device (such as a child learning toy), a gaming system, or a control device (e.g., a home controller or an industrial controller). Device 300 typically includes one or more processing units (CPUs) 310, one or more network or other communication interfaces 360, memory 370, and one or more communication buses 320 for interconnecting these components. The communication bus 320 optionally includes circuitry (sometimes called a chipset) that interconnects and controls communication between system components. Device 300 includes an input/output (I/O) interface 330 with a display 340, typically a touch screen display. I/O interface 330 also optionally includes a keyboard and/or mouse (or other pointing device) 350 and a touchpad 355, a tactile output generator 357 (e.g., similar to one or more tactile output generators 167 described above with reference to fig. 1A) for generating tactile outputs on device 300, a sensor 359 (e.g., an optical sensor, an acceleration sensor, a proximity sensor, a touch-sensitive sensor, and/or a contact intensity sensor similar to one or more contact intensity sensors 165 described above with reference to fig. 1A). Memory 370 includes high-speed random access memory, such as DRAM, SRAM, DDR RAM or other random access solid state memory devices; and optionally includes non-volatile memory, such as one or more magnetic disk storage devices, optical disk storage devices, flash memory devices, or other non-volatile solid-state storage devices. Memory 370 optionally includes one or more storage devices located remotely from CPU 310. In some embodiments, memory 370 stores programs, modules, and data structures similar to, or a subset of, the programs, modules, and data structures stored in memory 102 of portable multifunction device 100 (FIG. 1A). Further, memory 370 optionally stores additional programs, modules, and data structures not present in memory 102 of portable multifunction device 100. For example, memory 370 of device 300 optionally stores drawing module 380, presentation module 382, word processing module 384, website creation module 386, disk editing module 388, and/or spreadsheet module 390, while memory 102 of portable multifunction device 100 (FIG. 1A) optionally does not store these modules.

Each of the above identified elements in fig. 3 is optionally stored in one or more of the previously mentioned memory devices. Each of the identified modules corresponds to a set of instructions for performing the functions described above. The above identified modules or programs (i.e., sets of instructions) need not be implemented as separate software programs, procedures or modules, and thus various subsets of these modules are optionally combined or otherwise rearranged in various embodiments. In some embodiments, memory 370 optionally stores a subset of the modules and data structures described above. Further, memory 370 optionally stores additional modules and data structures not described above.

Attention is now directed to embodiments of a user interface ("UI") optionally implemented on portable multifunction device 100.

Fig. 4A illustrates an exemplary user interface of an application menu on portable multifunction device 100 according to some embodiments. A similar user interface is optionally implemented on device 300. In some embodiments, the user interface 400 includes the following elements, or a subset or superset thereof:

signal strength indicators 402 for wireless communications (such as cellular signals and Wi-Fi signals);

Time 404;

a Bluetooth indicator;

a battery status indicator 406;

tray 408 with common application icons such as:

an icon 416 of the telephony module 138 labeled "telephony", optionally including an indicator 414 of the number of missed calls or voice messages;

icon 418 of email client module 140, labeled "mail", optionally including an indicator 410 of the number of unread emails;

icon 420 of browser module 147, labeled "browser"; and

an icon 422 labeled "iPod" of video and music player module 152 (also referred to as iPod (trademark of Apple inc.) module 152); and

icons for other applications, such as:

icon 424 of IM module 141 labeled "message";

icon 426 of calendar module 148 labeled "calendar";

icon 428 of image management module 144 labeled "photo";

icon 430 of the o-camera module 143 labeled "camera";

icon 432 of online video module 155 labeled "online video";

an icon 434 of the stock market desktop applet 149-2 labeled "stock market";

Icon 436 of map module 154 labeled "map";

icon 438 labeled "weather" of weather desktop applet 149-1;

icon 440 labeled "clock" of alarm clock desktop applet 149-4;

icon 442 labeled "fitness support" of fitness support module 142;

icon 444 of notepad module 153 labeled "notepad"; and

an icon 446 for setting applications or modules, the icon 446 providing access to settings of the device 100 and its various applications 136.

It should be noted that the icon labels shown in fig. 4A are merely exemplary. For example, in some embodiments, icon 422 of video and music player module 152 is labeled "music" or "music player". Other tabs are optionally used for various application icons. In some embodiments, the label of the respective application icon includes a name of the application corresponding to the respective application icon. In some embodiments, the label of a particular application icon is different from the name of the application corresponding to the particular application icon.

Fig. 4B illustrates an exemplary user interface on a device (e.g., device 300 in fig. 3) having a touch-sensitive surface 451 (e.g., tablet or trackpad 355 in fig. 3) separate from the display 450. Device 300 also optionally includes one or more contact intensity sensors (e.g., one or more of sensors 357) for detecting the intensity of contacts on touch-sensitive surface 451 and/or one or more tactile output generators 359 for generating tactile outputs for a user of device 300.

Although many of the examples that follow will be given with reference to input on touch screen display 112 (where the touch-sensitive surface and the display are combined), in some embodiments, the device detects input on a touch-sensitive surface that is separate from the display, as shown in fig. 4B. In some embodiments, the touch-sensitive surface (e.g., 451 in fig. 4B) has a major axis (e.g., 452 in fig. 4B) that corresponds to a major axis (e.g., 453 in fig. 4B) on the display (e.g., 450). In accordance with these embodiments, the device detects contacts (e.g., 460 and 462 in fig. 4B) with the touch-sensitive surface 451 at locations that correspond to corresponding locations on the display (e.g., in fig. 4B, 460 corresponds to 468 and 462 corresponds to 470). As such, user inputs (e.g.,

contacts

460 and 462, and movements thereof) detected by the device on the touch-sensitive surface (e.g., 451 in FIG. 4B) are used by the device to manipulate the user interface on the display (e.g., 450 in FIG. 4B) when the touch-sensitive surface is separate from the display of the multifunction device. It should be understood that similar methods are optionally used for the other user interfaces described herein.

Further, while the examples below are given primarily with reference to finger inputs (e.g., finger contact, finger tap gesture, finger swipe gesture, etc.), it should be understood that in some embodiments, one or more of these finger inputs are replaced by an input from another input device (e.g., a stylus input). Similarly, when multiple user inputs are detected simultaneously, it should be understood that multiple finger contacts, or a combination of finger contact and stylus input, are used simultaneously.

As used in this specification and claims, the term "intensity" of a contact on a touch-sensitive surface refers to the force or pressure (force per unit area) of the contact on the touch-sensitive surface (e.g., a finger contact or a stylus contact), or to a substitute for the force or pressure of the contact on the touch-sensitive surface (surrogate). The intensity of the contact has a range of values that includes at least four different values and more typically includes hundreds of different values (e.g., at least 256). The intensity of the contact is optionally determined (or measured) using various methods and various sensors or combinations of sensors. For example, one or more force sensors below or adjacent to the touch-sensitive surface are optionally used to measure forces at different points on the touch-sensitive surface. In some implementations, force measurements from multiple force sensors are combined (e.g., weighted average or sum) to determine an estimated contact force. Similarly, the pressure-sensitive tip of the stylus is optionally used to determine the pressure of the stylus on the touch-sensitive surface. Alternatively, the size of the contact area detected on the touch-sensitive surface and/or changes thereto, the capacitance of the touch-sensitive surface proximate the contact and/or changes thereto, and/or the resistance of the touch-sensitive surface proximate the contact and/or changes thereto are optionally used as a surrogate for the force or pressure of the contact on the touch-sensitive surface. In some implementations, the surrogate measure of contact force or pressure is used directly to determine whether an intensity threshold has been exceeded (e.g., the intensity threshold is described in units corresponding to the surrogate measure). In some implementations, the substitute measurement of contact force or pressure is converted to an estimated force or pressure, and the estimated force or pressure is used to determine whether an intensity threshold has been exceeded (e.g., the intensity threshold is a pressure threshold measured in units of pressure). The intensity of the contact is used as a property of the user input, allowing the user to access additional device functionality that the user would otherwise not have easily accessible on a smaller sized device with limited real estate for displaying affordances (e.g., on a touch-sensitive display) and/or receiving user input (e.g., via a touch-sensitive display, a touch-sensitive surface, or physical/mechanical controls such as knobs or buttons).

In some embodiments, the contact/motion module 130 uses a set of one or more intensity thresholds to determine whether an operation has been performed by the user (e.g., determine whether the user has "clicked" on an icon). In some embodiments, at least a subset of the intensity thresholds are determined as a function of software parameters (e.g., the intensity thresholds are not determined by the activation thresholds of particular physical actuators and may be adjusted without changing the physical hardware of device 100). For example, the mouse "click" threshold of the trackpad or touch screen display may be set to any one of a wide range of predefined thresholds without changing the trackpad or touch screen display hardware. Additionally, in some implementations, a user of the device is provided with software settings for adjusting one or more intensity thresholds in a set of intensity thresholds (e.g., by adjusting individual intensity thresholds and/or by adjusting multiple intensity thresholds at once with a system-level click on an "intensity" parameter).

As used in the specification and in the claims, the term "characteristic intensity" of a contact refers to a characteristic of the contact based on one or more intensities of the contact. In some embodiments, the characteristic intensity is based on a plurality of intensity samples. The characteristic intensity is optionally based on a predefined number of intensity samples or a set of intensity samples sampled during a predetermined time period (e.g., 0.05 seconds, 0.1 seconds, 0.2 seconds, 0.5 seconds, 1 second, 2 seconds, 5 seconds, 10 seconds) relative to a predefined event (e.g., after detecting contact, before detecting contact lift, before or after detecting contact start movement, before or after detecting contact end, before or after detecting an increase in intensity of contact, and/or before or after detecting a decrease in intensity of contact). The characteristic intensity of the contact is optionally based on one or more of: a maximum value of the contact strength, a mean value of the contact strength, a value at the first 10% of the contact strength, a half maximum value of the contact strength, a 90% maximum value of the contact strength, a value generated by low-pass filtering the contact strength over a predefined period of time or from a predefined time, etc. In some embodiments, the duration of the contact is used in determining the characteristic intensity (e.g., when the characteristic intensity is an average of the intensity of the contact over time). In some embodiments, the characteristic intensity is compared to a set of one or more intensity thresholds to determine whether the user has performed an operation. For example, the set of one or more intensity thresholds may include a first intensity threshold and a second intensity threshold. In this example, a contact whose characteristic intensity does not exceed the first threshold results in a first operation, a contact whose characteristic intensity exceeds the first intensity threshold but does not exceed the second intensity threshold results in a second operation, and a contact whose characteristic intensity exceeds the second threshold results in a third operation. In some embodiments, a comparison between the characteristic intensity and one or more intensity thresholds is used to determine whether to perform one or more operations (e.g., whether to perform a respective option or forgo performing a respective operation) rather than to determine whether to perform a first operation or a second operation.

In some implementations, a portion of the gesture is recognized for determining the feature intensity. For example, the touch-sensitive surface may receive a continuous swipe contact that transitions from a starting location and reaches an ending location (e.g., a drag gesture) where the intensity of the contact increases. In this embodiment, the characteristic strength of the contact at the end position may be based on only a portion of the continuous swipe contact, rather than the entire swipe contact (e.g., only a portion of the swipe contact at the end position). In some implementations, a smoothing algorithm may be applied to the intensity of the swipe gesture before determining the characteristic intensity of the contact. For example, the smoothing algorithm optionally includes one or more of: a non-weighted moving average smoothing algorithm, a triangular smoothing algorithm, a median filter smoothing algorithm, and/or an exponential smoothing algorithm. In some cases, these smoothing algorithms eliminate narrow spikes or dips in the intensity of the swipe contacts for determining the feature intensity.

The user interface diagrams described herein optionally include various intensity maps that illustrate contacts on a touch-sensitive surface relative to one or more intensity thresholds (e.g., a contact detection intensity threshold, IT) ₀ Light press pressure intensity threshold IT _L Deep compression strength threshold IT _D (e.g., initially at least above I) _L ) And/or one or more other intensity thresholds (e.g., ratio I) _L Lower intensity threshold I _H ) Current intensity). This intensity map is typically not part of the displayed user interface, but is provided to assist in interpreting the map. In some embodiments, the light press intensity threshold corresponds to an intensity that: at this intensity, the device will perform the operations typically associated with clicking a button of a physical mouse or trackpad. In some embodiments, the deep press intensity threshold corresponds to an intensity that: at which intensity the device will perform a different operation than that typically associated with clicking a button of a physical mouse or trackpad. In some embodiments, when the characteristic intensity is detected to be below the light press intensity threshold (e.g., and above the nominal contact detection intensity threshold IT) ₀ A contact that is lower than the nominal contact detection intensity threshold is no longer detected), the device will move the focus selector in accordance with movement of the contact across the touch-sensitive surface without performing the operations associated with the light press intensity threshold or the deep press intensity threshold. Generally, unless otherwise stated, these intensity thresholds are consistent between different sets of user interface drawings.

In some embodiments, the device's response to an input detected by the device depends on criteria based on the intensity of contact during the input. For example, for some "tap" inputs, the intensity of the contact that exceeds a first intensity threshold during the input triggers a first response. In some embodiments, the response of the device to an input detected by the device depends on criteria that include both the intensity of contact during the input and time-based criteria. For example, for some "deep press" inputs, the intensity of a contact that exceeds a second intensity threshold, greater than the first intensity threshold of a light press, triggers a second response during the input as long as a delay time elapses between the first intensity threshold being met and the second intensity threshold being met. The delay time is typically less than 200ms (milliseconds) in duration (e.g., 40ms, 100ms, or 120ms, depending on the magnitude of the second intensity threshold, wherein the delay time increases as the second intensity threshold increases). This delay time helps avoid accidentally recognizing a deep press input. As another example, for some "deep press" inputs, a period of reduced sensitivity will occur after the first intensity threshold is reached. During this period of reduced sensitivity, the second intensity threshold is increased. This temporary increase in the second intensity threshold also helps to avoid accidental deep press inputs. For other deep press inputs, the response to detecting a deep press input does not depend on time-based criteria.

In some embodiments, one or more of the input intensity thresholds and/or corresponding outputs vary based on one or more factors (such as user settings, contact motion, input timing, application execution, rate at which intensity is applied, number of simultaneous inputs, user history, environmental factors (e.g., environmental noise), focus selector position, etc.. Exemplary factors are described in U.S. patent application Ser. Nos. 14/399,606 and 14/624,296, which are incorporated by reference herein in their entireties.

For example, fig. 4C illustrates a dynamic intensity threshold 480 that varies over time based in part on the intensity of the touch input 476 over time. The dynamic intensity threshold 480 is the sum of two components: a first component 474 that decays over time after a predefined delay time p1 from when the touch input 476 was initially detected, and a second component 478 that tracks the intensity of the touch input 476 over time. The initial high intensity threshold of the first component 474 reduces accidental triggering of a "deep press" response while still allowing an immediate "deep press" response if the touch input 476 provides sufficient intensity. The second component 478 reduces the gradual intensity fluctuations through the touch input without unintentionally triggering a "deep press" response. In some embodiments, a "deep press" response is triggered when the touch input 476 meets a dynamic intensity threshold 480 (e.g., at point 481 in fig. 4C).

FIG. 4D illustrates another dynamic intensity threshold 486 (e.g., intensity threshold I) _D ). Fig. 4D also shows two other intensity thresholds: first intensity threshold I _H And a second intensity threshold I _L . In FIG. 4D, although touch input 484 satisfies first intensity threshold I before time p2 _H And a second intensity threshold I _L But no response is provided until a delay time p2 elapses at time 482. Also in FIG. 4D, the dynamic intensity threshold 486 decays over time, with decay at slave time 482 (triggering and second intensity threshold I) _L Time of associated response) has elapsed a time 488 after a predefined delay time p 1. This type of dynamic intensity threshold reduction is immediately triggered with a lower threshold intensity (such as the first intensity threshold I) _H Or a second intensity threshold I _L ) Unexpected triggering of and dynamic intensity threshold I after or concurrently with an associated response _D An associated response.

FIG. 4E shows yet another dynamic intensity threshold 492 (e.g., intensity threshold I) _D ). In FIG. 4E, the trigger and intensity threshold I is set after a delay time p2 has elapsed since the time when the touch input 490 was initially detected _L An associated response. Meanwhile, the dynamic intensity threshold 492 decays after a predefined delay time p1 has elapsed from the time the touch input 490 was initially detected. Thus, at the trigger and intensity threshold I _L Decreasing the intensity of the touch input 490 after the associated response, then increasing the intensity of the touch input 490 without releasing the touch input 490 may trigger the intensity threshold I _D The associated response (e.g., at time 494), even when the intensity of the touch input 490 is below another intensity threshold (e.g., intensity threshold I) _L ) The same is true of the case.

The contact characteristic intensity is lower than the light pressing intensity threshold IT _L To be between the light press intensity threshold IT _L And deep press intensity threshold IT _D The intensity in between is sometimes referred to as a "light press" input. Characteristic intensity of contact from below deep press intensity threshold IT _D Is increased to above the deep press strength threshold IT _D Is sometimes referred to as a "deep press" input. Contact characteristic intensity from below contact detection intensity threshold IT ₀ To be in-between the contact detection intensity threshold IT ₀ And light press intensity threshold IT _L Is sometimes referred to as detecting contact on the touch surface. Characteristic intensity of contact is higher than contact detection intensity threshold IT ₀ Is reduced to below a contact detection intensity threshold IT ₀ Is sometimes referred to as detecting lift-off of the contact from the touch surface. In some embodiments, IT ₀ Is zero. In some embodiments, IT ₀ Greater than zero in some illustrations, a shaded circle or ellipse is used to represent the intensity of a contact on the touch-sensitive surface. In some illustrations, circles or ellipses without shading are used to represent respective contacts on the touch-sensitive surface without specifying intensities of the respective contacts.

In some embodiments, the dynamic intensity thresholds shown in FIGS. 4C-4E are used for input on touch-sensitive display system 112. In some embodiments, different criteria (e.g., the criteria described with reference to fig. 5A-5 II) are used for input on an intensity-sensitive input device, such as the intensity-sensitive primary button shown in fig. 2B and 2C. In some embodiments, the criteria described with reference to FIGS. 5A-5II are used for all inputs, rather than the dynamic intensity thresholds shown in FIGS. 4C-4E.

In some embodiments described herein, one or more operations are performed in response to detecting a gesture that includes a respective press input or in response to detecting a respective press input performed with a respective contact (or contacts), wherein the respective press input is detected based at least in part on detecting an increase in intensity of the contact (or contacts) above a press input intensity threshold. In some embodiments, the respective operation is performed in response to detecting that the intensity of the respective contact increases above the press input intensity threshold (e.g., performing the respective operation on a "down stroke" of the respective press input). In some embodiments, the press input includes an increase in intensity of the respective contact above a press input intensity threshold and a subsequent decrease in intensity of the contact below the press input intensity threshold, and the respective operation is performed in response to detecting the subsequent decrease in intensity of the respective contact below the press input threshold (e.g., the respective operation is performed on an "up stroke" of the respective press input).

In some embodiments, the device employs intensity hysteresis to avoid accidental input sometimes referred to as "jitter," where the device defines or selects a hysteresis intensity threshold having a predefined relationship to the press input intensity threshold (e.g., the hysteresis intensity threshold is X intensity units less than the press input intensity threshold, or the hysteresis intensity threshold is 75%, 90%, or some reasonable proportion of the press input intensity threshold). Thus, in some embodiments, a press input includes an increase in intensity of the respective contact above a press input intensity threshold and a subsequent decrease in intensity of the contact below a hysteresis intensity threshold corresponding to the press input intensity threshold, and a respective operation is performed in response to detecting the subsequent decrease in intensity of the respective contact below the hysteresis intensity threshold (e.g., the respective operation is performed on an "up stroke" of the respective press input). Similarly, in some embodiments, a press input is only detected when the device detects an increase in intensity of the contact from an intensity at or below the hysteresis intensity threshold to an intensity at or above the press input intensity threshold and optionally a subsequent decrease in intensity of the contact to an intensity at or below the hysteresis intensity, and a corresponding operation is performed in response to detecting the press input (e.g., an increase in intensity of the contact or a decrease in intensity of the contact, depending on the circumstances).

For ease of explanation, the description of operations performed in response to a press input associated with a press input intensity threshold or in response to a gesture that includes a press input is optionally triggered in response to detecting: the intensity of the contact is increased above the press input intensity threshold, the intensity of the contact is increased from an intensity below the hysteresis intensity threshold to an intensity above the press input intensity threshold, the intensity of the contact is decreased below the press input intensity threshold, or the intensity of the contact is decreased below the hysteresis intensity threshold corresponding to the press input intensity threshold. Additionally, in examples in which operations are described as being performed in response to detecting that the intensity of the contact decreases below the press input intensity threshold, the operations are optionally performed in response to detecting that the intensity of the contact decreases below a hysteresis intensity threshold that corresponds to and is less than the press input intensity threshold. As described above, in some embodiments, the triggering of these operations also depends on the time-based criteria being met (e.g., a delay time has elapsed between the first intensity threshold being met and the second intensity threshold being met).

Although only certain frequencies, amplitudes, and waveforms are shown in the sample haptic output pattern for illustrative purposes in FIG. 4F, haptic output patterns having other frequencies, amplitudes, and waveforms may be used for similar purposes. For example, a waveform having between 0.5 and 4 cycles may be used. Other frequencies in the range of 60Hz-400Hz may also be used.

User interface and associated process

Attention is now directed to implementations of a user interface ("UI") and associated processes that may be implemented on an electronic device, such as portable multifunction device 100 or device 300, having a display, a touch-sensitive surface, and one or more sensors for detecting intensity of contacts with the touch-sensitive surface.

5A-5II illustrate an exemplary user interface and a plurality of timeout periods and intensity thresholds for detecting gestures. According to some embodiments, the intensity thresholds and some of the timeout periods are based on previous input intensities (e.g., during gesture input, some intensity thresholds are based on a characteristic or representative intensity of the input during the same gesture). The user interfaces in these figures are used to illustrate the processes described below, including the processes in fig. 6A-6F, 7A-7E, and 8A-8C. For ease of explanation, some of the embodiments will be discussed with reference to operations performed on a device having touch-sensitive display system 112. In such embodiments, the focus selector is optionally: a respective finger or stylus contact, a representative point corresponding to the finger or stylus contact (e.g., a center of gravity of or a point associated with the respective contact), or a center of gravity of two or more contacts detected on touch-sensitive display system 112. However, similar operations are optionally performed on a device having a display 450 and a separate touch-sensitive surface 451 in response to detecting a contact on the touch-sensitive surface 451 when the user interface shown in the figures is displayed on the display 450 along with a focus selector.

5A-5N illustrate a user interface, user input intensity, and corresponding intensity thresholds for distinguishing between multiple gestures (such as single-tap, double-tap, and long-press gestures). 5A-5C illustrate a single tap gesture on the primary button 204 of an electronic device that also includes touch-sensitive display 112. In some embodiments, the primary button 204 is separate from the display, and optionally includes a set of one or more intensity sensors separate from the intensity sensor used to detect the intensity of the input on the display. In some embodiments, the master button 204 is a virtual master button displayed on the display (e.g., having a set of one or more intensity sensors separate from the intensity sensor used to detect the intensity of the input on the display, or optionally using an intensity sensor integrated into the display to determine the intensity of the input with the virtual master button). The primary button 204 is associated with an intensity sensor for measuring the intensity of the user input on the primary button. FIG. 5A illustrates the electronic device 100, the display 112, and the primary button 204, further illustrating a first down click intensity threshold I _D . As shown in FIG. 5B, the touch input 505 on the primary button 204 has a time-varying intensity, including a first increase in intensity 520 (sometimes referred to as a first down-click) to a peak intensity I _Peak, The peak intensity is higher than the first click intensity threshold I _D (e.g., because the user will typically exceed the down-click intensity threshold when performing down-click operations). Thus, the electronic device or a module thereof (e.g., contact/motion module 130, FIG. 1A) detects that the increase in intensity satisfies a down-click detection criterion that requires the input's intensity to increase above a first down-click intensity thresholdValue I _D So that the press click detection criteria are met. In some embodiments, when the down-click intensity threshold is reached, the device provides feedback (e.g., audible and/or tactile feedback) indicating that the down-click intensity threshold has been reached.

As shown in FIG. 5C, after the first intensity is increased, the intensity of the touch input on the primary button 204 decreases, and the electronic device detects 522 a first decrease in intensity of the contact, sometimes referred to as an up-click. 5B-5C, a first decrease in intensity of the input satisfies an up-click detection criterion that requires the intensity of the input to decrease below a first up-click intensity threshold I _U So that the up-click detection criteria are met. As will be explained in more detail below, the first up-click intensity threshold I _U Based on the intensity of the input during the first intensity increase 520 of the contact. Since the first intensity reduction 522 of the contact satisfies the up-click detection criteria, the electronic device 100 provides first feedback. In the example shown in fig. 5B-5C, the first feedback is or includes switching from displaying a user interface of the first application (e.g., a timer application) to displaying an application launch user interface. In some embodiments, the user interface transitions shown in FIGS. 5B-5C are implemented by closing an application (e.g., a timer application) or ceasing to display the user interface of the application in response to a single-tap gesture on the primary button 204, the single-tap gesture being performed when the first decrease in intensity of the input meets the release-tap detection criteria. In some embodiments, when the up-click intensity threshold is reached, the device provides feedback (e.g., audible and/or tactile feedback) indicating that the up-click intensity threshold has been reached. In some embodiments, the tactile output for the up-click is different from the tactile output for the down-click (e.g., the tactile output for the up-click has a reduced magnitude relative to the down-click tactile output, such as a MicroTap (270 Hz) tactile output mode with a gain of 0.5 relative to a MicroTap (270 Hz) tactile output mode with a gain of 1.0 relative to the down-click).

5B and 5F correspond to input as compared to the touch input based gestures represented by FIGS. 5B-5CThe first intensity reduction in intensity does not satisfy the up-click detection criteria (e.g., does not decrease below the first up-click intensity threshold I) _U ) So the electronic device forgoes providing the first feedback (e.g., visual, audible, and/or tactile feedback).

Fig. 5D-5E illustrate a second single click input. As shown in FIG. 5D, the second touch input 507 on the primary button 204 detected after the first touch input 505 (shown in FIG. 5C) ends has an intensity that changes over time, including an increase in intensity 524 to a peak intensity I _Peak, The peak intensity is higher than the first click intensity threshold I _D . Thus, the electronic device or a module thereof (e.g., the contact/motion module 130, fig. 1A) detects that the intensity increase 524 meets the down-click detection criteria. In some embodiments, when the down-click intensity threshold is reached, the device provides feedback (e.g., audible and/or tactile feedback) indicating that the down-click intensity threshold has been reached.

As shown in FIG. 5E, after the intensity of the second input 507 increases 524, the intensity of the touch input on the primary button 204 decreases, and the electronic device detects a decrease 526 in the intensity of the input. 5D-5E, the decrease in intensity of the input satisfies an up-click detection criterion that requires the intensity of the input to decrease below a first up-click intensity threshold I _U So that the up-click detection criteria are met. The electronic device 100 provides feedback because the decrease in intensity of the input satisfies the up-click detection criteria. In the example shown in fig. 5D-5E, the feedback is or includes scrolling from one icon screen (including the first set of application launch icons) in an application launch user interface (as shown in fig. 5D) to another icon screen (including the second set of application launch icons, which includes application launch icons that are not in the first set of application launch icons) of the application launch user interface. In some embodiments, when the up-click intensity threshold is reached, the device provides feedback (e.g., audible and/or tactile feedback) indicating that the up-click intensity threshold has been reached.

In FIGS. 5C, 5D, and 5E, indicator 510 indicates a point in time when the down click detection criteria are met, indicator 512 indicates a point in time when the up click detection criteria are met, indicator 514 indicates a point in time when the down click detection criteria are met a second time, and indicator 516 indicates a point in time when the up click detection criteria are met a second time. In some embodiments, the electronic device or a module thereof (e.g., an application independent module such as contact/motion module 130, fig. 1A) generates an event (e.g., a press-click event) in response to press-click detection criteria being met, or generates an event (e.g., a release-click event) in response to press-click detection criteria being met, or both. In some embodiments, an event (e.g., a press-click event or a release-click event) is delivered to one or more targets, such as an application, or a web page for instruction processing in the web page, or to a web browser (which is a special case of an application) and/or a tactile feedback module, such as module 133, FIG. 1A. For example, as shown in FIG. 5C, in some embodiments, tactile output 502 (e.g., tactile output having a MicroTap (270 Hz) tactile output mode, FIG. 4F) is generated in conjunction with the electronic device detecting that a decrease in intensity of the input satisfies the up-click criteria (sometimes referred to as detecting an up-click or detecting a single-click input). Further, referring to fig. 5B, 5BB, 5DD, 5FF, 5HH, and 5II, in some embodiments, the

tactile output

502 or 504 is generated in conjunction with the electronic device detecting that the increase in intensity of the input satisfies the down-click criteria (sometimes referred to as detecting a down-click or long press), e.g., as discussed above with reference to fig. 5Z-5 II.

5G-5I illustrate two examples of double-click input and providing corresponding feedback. In particular, fig. 5G shows that the peak intensity I is reached including a first intensity increase 532 _Peak The peak intensity is higher than the first down-click intensity threshold I _D . After reaching the peak intensity I _Peak Then, the touch input includes a first intensity reduction 534 of the input to a low intensity I _Valley The low intensity is below a first up-click intensity threshold I _U So that the up-click detection criteria are met. As described above, the first up-click intensity threshold I _U Intensity of input during contact-based first intensity increase 532To be determined. After the first intensity decrease 534, the touch input includes a second intensity increase 536 to above the second down-click intensity threshold I _D2 The strength of (2). As will be explained in more detail below, the second down-click intensity threshold I _D2 The determination is based on the intensity of the input during the first decrease in intensity of the contact. For example, in some embodiments, the second down click intensity threshold I _D2 Input minimum or minimum intensity I during first intensity reduction based on contact _Valley To be determined.

In some embodiments, the input shown in FIG. 5G includes a first increase in intensity 532 as indicated by indicator 510 that satisfies the down-click detection criteria, followed by a first decrease in intensity 534 as indicated by indicator 512 that satisfies the up-click detection criteria, followed by a second increase in intensity 536 as indicated by indicator 518 that satisfies the down-click detection criteria, where the down-click detection criteria includes a first down-click intensity threshold for the first increase in intensity and includes a second down-click intensity threshold for the second increase in intensity.

In some embodiments, represented by the transition from fig. 5G to fig. 5I, when the electronic device 101 determines that the first increase in intensity 532, the first decrease in intensity 534, and the second increase in intensity 536 of the input shown in fig. 5G have satisfied the down-click detection criteria, the up-click detection criteria, and the down-click detection criteria, respectively, the electronic device generates a second feedback (e.g., visual, audible, and/or tactile feedback) indicating that the second increase in intensity is identified as part of the double-click input, such as displaying the multitasking user interface shown in fig. 5H. In some embodiments, the second feedback is generated or initiated at or immediately after the time indicated by the indicator 518 in fig. 5G and 5I at which the second increase in intensity satisfies the down-click detection criteria. It is noted that in some embodiments or in some cases, "one-half click" (e.g., input comprising a first down click, a first up click, and a second down click in sequence) is treated as a double click, which triggers execution of an action, such as generation of a second feedback. In some embodiments, the second feedback is or includes a transition to a multitasking user interface, as shown in fig. 5I. In some embodiments, the second feedback is or includes generating a haptic output 503, as shown in fig. 5H and 5I. In some embodiments, the haptic output 503 is a haptic output having a MiniTap (270 Hz) haptic output mode (fig. 4F).

In some embodiments, represented by the transition from FIG. 5G to FIG. 5H, the input shown in FIG. 5G continues to fall below the up-click intensity threshold I _U And thus the up-click detection criteria is met at the time indicated by

indicator

522, 538. Although FIG. 5H illustrates the same up-click intensity threshold I for the first and second intensity reductions _U However, in some embodiments or in some cases, the up-click intensity threshold for the second up-click (e.g., the second decrease in intensity) is different than the up-click intensity threshold for the first up-click (e.g., the first decrease in intensity). It is noted that in some embodiments or in some cases, a "two full click" (e.g., an input comprising, in sequence, a first down click 532, a first up click 534, a second down click 536, and a second up click 538) is treated as a double click that triggers performance of an action, such as generating a second feedback (e.g., visual, audible, and/or tactile feedback). In some embodiments, the second feedback is a transition to a multitasking user interface, as shown in fig. 5H.

FIG. 5J graphically illustrates the up-click intensity threshold I _D (see fig. 5C, and 5E-5I) how the ratio of the intensity value representing the contact strength to the intensity value representing the contact strength changes based on the intensity value representing the contact strength. As shown, the ratio is greater for input intensity than the first down click intensity threshold I _D Large (or high) I _D1 When it has a maximum value of a ₂ At an input intensity of ratio I _D1 Large (or high) I _D2 Has a minimum value of a ₁ . This ratio has a value between 0 and 1. In some embodiments, the maximum value of the ratio, a ₂ Equal to 0.73, minimum value a of this ratio ₁ Equal to 0.6. In one example, the ratio is 0.6 when the low-pass filtered present strength of the contact (as discussed below with reference to FIG. 5K) is 500g, and the ratio is 0.73 when the low-pass filtered present strength of the contact is 300g。

More generally, when the up-click intensity is based on a first intensity value representing the contact intensity, a ratio of the up-click intensity threshold to the first intensity value has a first value; when the up-click intensity is based on a second intensity value representing the contact intensity that is greater than the first intensity value, the ratio of the up-click intensity threshold to the second intensity value has a second value that is different from (e.g., smaller or larger than) the first value.

In some embodiments, the ratio shown in FIG. 5J is applied to (e.g., multiplied by) a characteristic intensity of the input (e.g., a peak intensity of the input during a first increase in intensity, or an intensity value obtained by low-pass filtering the intensity during a first decrease in intensity) to determine the up-click intensity threshold I _U Is used as a multiplier of (1).

FIG. 5K illustrates an up-click intensity threshold I that dynamically changes as the intensity of the input changes during a first intensity decrease 542 after the electronic device has detected a first intensity increase 540 _U(t) 546. More specifically, during the first intensity reduction 542, the intensity of the input is low pass filtered, thereby generating a first time-varying value I _LPup 544. The first time-varying value is then multiplied by a fixed value, such as 0.7, or an intensity-based value, such as the ratio shown in FIG. 5J, to generate a time-varying up-click intensity threshold I _U(t) Wherein the "(t)" symbol indicates that the value is time-varying. When the intensity of the input matches or decreases below a time-varying up-click intensity threshold I _U(t) When the up-click detection criteria are satisfied, as shown by indicator 548.

FIG. 5L shows a down-click intensity threshold I that dynamically changes as the intensity of the input changes during a second increase in intensity 550 after the electronic device has detected a first increase in intensity 540 and a first decrease in intensity 542 _D(t) 554. More specifically, during the second intensity increase 550, the intensity of the input is low pass filtered, thereby generating a second time-varying value I _LPdown 552. In some implementations, the second intensity of the contact is increased 550 by the low pass filtered intensity (I) of the detected intensity input _LPdown 552 Initially at the beginning of the second intensity increase 550 of the contactLowest intensity I of input during first intensity reduction 534 set to contact _Valley 。

A second time-varying value I _LPdown 552 then multiplied by a fixed value such as 1.4, (or alternatively, divided by a fixed value such as 0.7) or an intensity-based value such as the ratio shown in FIG. 5J, to generate a time-varying up-click intensity threshold I _U(t) 554, wherein the "(t)" symbol indicates that the value is time-varying. When the intensity of the input matches or increases above a time-varying down-click intensity threshold I _D(t) 554, the click detection criteria are met as indicated by indicator 556.

FIG. 5M shows an up-click intensity threshold I that dynamically changes as the intensity of the input changes during a second intensity decrease 558 (sometimes referred to as a second up-click) after the electronic device has detected a second intensity increase 550 _U2(t) 561. More specifically, during the second intensity reduction 558, the intensity of the input is low pass filtered, thereby generating a second time-varying value I _LP2up 560. The second time-varying value is then multiplied by a fixed value, such as 0.7, or an intensity-based value, such as the ratio shown in FIG. 5J, to generate a time-varying up-click intensity threshold I _U2(t) 561 where the "(t)" symbol indicates that the value is time-varying. When the intensity of the input matches or decreases below a time-varying up-click intensity threshold I _U2(t) 561, the up-click detection criteria is met as shown by indicator 562.

FIG. 5N is similar to FIG. 5M, except that the minimum up-click intensity threshold I _UM Time-varying up-click intensity threshold I applied to FIG. 5M _U2(t) 561, thereby generating a modified time-varying up-click intensity threshold I _U2M(t) 563. In other words, the time-varying up-click intensity threshold I _U2M(t) 563 equal at each point in time a time-varying up-click intensity threshold I _U2(t) 561 and minimum Un-click intensity threshold I _UM The larger of them.

5O-5Y illustrate a user interface, user input intensity, and corresponding intensity thresholds for distinguishing between single-tap inputs or gestures and double-tap inputs or gestures, where recognition of single-tap gestures is accelerated or expedited (e.g., based onDetermine that a double-click input will not be performed). 5O-5P illustrate a single tap gesture, sometimes referred to as a single tap or single tap gesture, on the primary button 204 of an electronic device that also includes a touch-sensitive display 112. The main button 204 includes an intensity sensor for measuring the intensity of the user input on the main button. FIG. 5O illustrates the electronic device 100, the display 112, and the primary button 204, further illustrating the first down click intensity threshold I _D First up click intensity threshold I _U And early confirmation threshold I _A Sometimes referred to as an accelerated acknowledgement threshold. As shown in FIG. 5O, the touch input 523 on the primary button 204 has a time-varying intensity, including a first increase in intensity 532 (sometimes referred to as a first down click) to a peak intensity I _Peak, The peak intensity is higher than the first click intensity threshold I _D . Thus, the electronic device or a module thereof (e.g., contact/motion module 130, FIG. 1A) detects that the increase in intensity satisfies the down-click detection criteria, such as at time T ₁ Is indicated by a point indicator 510 that requires the intensity of the input to increase above the first down-click intensity threshold I _D So that the press click detection criteria are met.

After the first increase in intensity 532, the intensity of the touch input 523 on the primary button 204 decreases and the electronic device detects a first decrease in intensity of the contact 534, sometimes referred to as a first up-click. In the example shown in FIG. 5O, the first decrease in intensity of the input satisfies the up-click detection criteria, such as at time T _2a At indicator 512, where the input satisfying the up-click detection criteria requires that the intensity of the input decrease below a first up-click intensity threshold I _U So that the up-click detection criteria are met. If the first increase in intensity satisfies the down-click detection criteria and the first decrease in intensity satisfies the up-click detection criteria, the device identifies at least a portion of the change in intensity of the input as a first event, such as a click event, sometimes referred to as a first click.

If the current context of the electronic device 100 allows both a single click and a double click on the primary button 204, the performance of the first operation associated with identifying the single click is delayed until the device determines that the user is not entering a double click, or equivalently, the first click is not the first part of a double click. In some embodiments, the delay of the execution of the first operation is limited to a default delay time, such as 300ms or 500ms. However, if the electronic device is able to determine that the input satisfies the early-confirmation criteria before the default delay time expires, indicating that the input will not be a double-click, the electronic device may cause performance of the first operation as long as that determination is made.

In some embodiments, the early validation criterion requires that the intensity input remain below the validation intensity threshold I during the first intensity reduction _A Continuously exceeding the early acknowledgement time threshold. For example, in some embodiments, the confirmation intensity threshold I _A Is 100g, and the up-click detection threshold I _U Is 150g or 200g or more, and therefore confirms the intensity threshold I _A Below the up-click detection threshold.

In some embodiments, the confirmation intensity threshold I _A According to the peak characteristic intensity I of the input detected during the detected increase in intensity of the input before the decrease in intensity of the input on the input element is detected _Peak To be determined. For example, in some embodiments, the up-click detection threshold I _U According to the peak characteristic intensity I of the input detected during the detected increase of the intensity of the input before the decrease of the intensity of the input on the input element is detected _Peak To determine and confirm the intensity threshold I _A According to the detection threshold I of the click-off _U To set it. In some such embodiments, the confirmation intensity threshold I _A Is set to be greater than an up click detection threshold I _U A predefined amount of level that is small (e.g., 50g smaller), while in other such embodiments, the intensity threshold I is confirmed _A Is set to the up-click detection threshold I _U E.g., 0.90 times or 90%. In some embodiments, the up-click threshold is the dynamically determined up-click threshold described above with reference to FIGS. 5A-5N.

Optionally, tactile output 502 (e.g., tactile output having a MicroTap (270 Hz) tactile output mode, fig. 4F) is generated in conjunction with the electronic device detecting that an increase in intensity of the input meets a down-click criterion (sometimes referred to as detecting or identifying a down-click) or in conjunction with the electronic device detecting that a decrease in intensity of the input meets an up-click criterion (sometimes referred to as detecting or identifying an up-click or a single-click input). In some embodiments, the up-click haptic output is different from the down-click haptic output (e.g., the up-click haptic output has a reduced magnitude relative to the down-click haptic output, such as a MicroTap (270 Hz) haptic output mode with a gain of 0.5 relative to a MicroTap (270 Hz) haptic output mode with a gain of 1.0 relative to the down-click haptic output mode).

In FIG. 5P, the first reduction 534 of the input shown in FIG. 5O continues at time T ₂ Is passed through the confirmation intensity threshold I _A As indicated by indicator 564. However, in this example, the input ends at time T _off Thereby ending the gesture. In some embodiments, terminating the input by liftoff of the contact during the first decrease in intensity is handled as a confirmation gesture being a single tap (or equivalently, a single tap). Thus, the electronic device performs a first operation, which in this example comprises ending the display of the user interface of the application displayed before the single tap was received, so the display 112 of the electronic device transitions from displaying the user interface of the first application (e.g., the time application) to displaying a set of application launch icons in the application launch user interface.

In FIG. 5Q, a touch input 525 detected on the primary button 204 (e.g., during display of an application user interface similar to that shown in FIG. 5O on the touch-sensitive display 112) has a time-varying intensity, including a first increase in intensity 532 (sometimes referred to as a first down-click) followed by a first decrease in intensity 534. Indicator 512 indicates a point in time when the up-click detection criteria are met, and indicator 564 indicates that the first intensity reduction reached the confirmation intensity threshold I _A The time point of (c). In the example shown in FIG. 5Q, the confirmation intensity threshold I is reached _A Thereafter, the input remains below the confirmation intensity threshold I _A From time T ₂ Beginning for a period of time that lasts at least for the acknowledgment time threshold. Referred to as "fast timeout periodThe timeout period starts at time T ₂ When the intensity of the input decreases below the validation threshold I _A Then (c) is performed. When the duration of the fast timeout period reaches the acknowledgment time threshold, at time T ₃ The input is confirmed as a single click, as indicated by indicator 565. Since the fast timeout period reaches the acknowledgment time threshold, at time T in the example of FIG. 5Q ₃ So a first operation (e.g., an operation associated with the recognized single click) is performed or the execution of the first operation is caused. As described above, in these examples, the first operation includes ending the display of the user interface of the application (e.g., the application user interface shown in FIG. 5O) that was displayed before the single tap was received, and thus the display 112 of the electronic device transitions from displaying the user interface of the first application (e.g., the time application) to displaying a set of application launch icons in the application launch user interface.

Fig. 5R shows a continuation of the inputs shown in fig. 5Q. In this example, at (e.g., at time T) ₃ Where) confirming that the initial input is a single tap input and performing or causing performance of the first operation, a second single tap is received by the electronic device. In particular, the second single tap includes a second increase in intensity 566 (as indicated by indicator 567) that satisfies the down-click detection criteria, and a second decrease in intensity 568 (as indicated by indicator 569) that satisfies the up-click detection criteria. In this example, another operation is performed when a second single tap is detected, for example, when the second intensity reduction 568 satisfies the up-click detection criteria. In the example shown in fig. 5Q and 5R, the operation includes scrolling from one icon screen in the application launch user interface (as shown in fig. 5D) (including the first set of application launch icons) to another icon screen in the application launch user interface (as shown in fig. 5R) (including the second set of application launch icons, including application launch icons that are not in the first set of application launch icons).

It is noted that in the example shown in FIG. 5R, during the second intensity decrease 568, the intensity also drops below the confirmation intensity threshold I _A As indicated by indicator 570. However, unless electrically poweredThe sub-device is configured to detect single, double, and triple tap inputs, otherwise the intensity has dropped below the confirmation intensity threshold I during the second intensity reduction _A Any additional actions of the electronic device may not be triggered. For further consideration in detecting the multi-tap gesture, see the discussion of FIG. 5Y below.

In FIG. 5S, the touch input 527 on the primary button 204 has a time-varying intensity, including a first increase in intensity 532 and at time T _2A Decrease below the up-click intensity threshold I _U As indicated by

indicator

512, 534. After the intensity decreases below the up-click intensity threshold I _U (this may be considered a first event) the intensity of the touch input 527 remains below the up-click intensity threshold I _U But above the confirmation intensity threshold I _A The strength of (2). Execution of the first operation in response to the first event is delayed until a default delay period has elapsed, e.g., at time T ₄ As indicated by indicator 571. As described above, in these examples, the first operation includes ending the display of the user interface of the application displayed before the single tap was received, and thus the display 112 of the electronic device transitions from displaying the user interface of the first application (e.g., the time application) to displaying a set of application launch icons in the application launch user interface.

In some embodiments, the delay time is the time from when the first down click is detected (e.g., time T in FIG. 5S) ₁ Indicated) is monitored or measured. In some other embodiments, the delay time is the time from when the first up-click was detected (e.g., time T in FIG. 5S) _2A Indicated) is measured. In some embodiments, if the intensity of the input increases above the up-click intensity threshold I before the delay time reaches the default delay time period _U The measurement of the delay time is stopped and the measured delay time is reset to zero. The default delay period is typically significantly longer than the early acknowledgement time threshold. For example, in some embodiments, the default delay period is a value between 300ms and 500ms, while the early acknowledgement time threshold is a value between 100ms and 200 ms.

As shown in FIGS. 5T, 5U, and 5V, in some embodiments, the early-confirmation criterion is that the intensity of the input is below the early-confirmation threshold I _A The criterion is satisfied when the cumulative amount of time of early acknowledgment reaches the early acknowledgment time threshold. In the example shown in FIG. 5T, after the first intensity increase 532 and the first intensity decrease 534, the intensity of the input is at time T ₂ Decrease below the up-click intensity threshold I _U And early confirmation threshold I _A As indicated by indicator 568, but then at time T _2-1 Increase above early acknowledgement threshold I _A But does not meet the early validation criteria. As shown in FIG. 5U, the intensity of the input is at time T _2-2 Decreases again below the early confirmation threshold I _A As indicated by indicator 572, then remains below early acknowledgement threshold I, as shown in fig. 5V _A Until the input intensity has remained below the early-confirmation threshold I _A At time T _3-2 The early acknowledgement time threshold is reached as indicated by indicator 573. Thus, at time T _3-2 The electronic device performs or causes performance of a first operation. As described above, in these examples, the first operation includes ending the display of the user interface for the application displayed before the single tap was received, and thus the display 112 of the electronic device transitions from displaying the user interface for the first application (e.g., the temporal application) to displaying a set of application launch icons in the application launch user interface.

In FIG. 5W, the touch input 531 on the primary button 204 has a time-varying intensity including a first intensity increase 532 and decrease below the up-click intensity threshold I _U As indicated by indicator 512, and then below early acknowledgement threshold I at time T2 _A As indicated by indicator 568. The input then continues with a second intensity increase 536 above the down click intensity threshold I _D Or alternatively, to above a second down-click intensity threshold I _D2 As shown in fig. 5G), as indicated by indicator 574. The second intensity increase 536 occurs before the default delay period expires and also before the intensity of the input meets the early validation criteria, so the electronic deviceA second input event (e.g., a double tap) is identified and the first operation is not performed. In the example shown in FIG. 5W, the intensity drops below the early acknowledgment intensity threshold I _A Time T of ₂ To intensity rise above early confirmation threshold I _A Time T of _2-1 Is shorter than the early acknowledgement time threshold.

Conversely, in response to the second intensity increase 536 to above the press click intensity threshold I _D From displaying a previous user interface, such as a user interface of an application (as shown in fig. 5O), to displaying a multitasking user interface, as shown in fig. 5W.

Optionally, the tactile output 503 is generated in conjunction with the electronic device recognizing the second input event (e.g., detecting the second increase in intensity 536 to continue to an intensity above the down-click intensity threshold before the default delay period expires). In some embodiments, the haptic output 503 is a haptic output having a MiniTap (270 Hz) haptic output mode (fig. 4F).

As shown in FIG. 5X, the touch input 533 on the main button 204 has a time-varying intensity, including a first intensity increase 532 and decrease below the up-click intensity threshold I _U Is detected, as indicated by indicator 512, and then falls below the early acknowledgement threshold I _A . In some embodiments, the measurement of the delay time (for delaying performance of the first operation corresponding to the single click input) begins with a decrease in the intensity of the input below the up-click intensity threshold I _U And as long as the input remains below the up-click intensity threshold I _U The measurement is continued. Alternatively, in some embodiments, the measurement of the delay time begins with the intensity of the input increasing above the down-click intensity threshold I _D As shown in fig. 5S, and once the up-click is detected, as long as the intensity of the input 533 remains below the up-click intensity threshold I _U The measurement is continued.

In FIG. 5Y, the electronic device determines whether the intensity of the touch input 531 meets the double-tap criterion or the triple-tap criterion, and thus whether the input is a double-tap or a triple-tap. Touch input 531 on primary button 204 (also shown in FIG. 5W) has a follow-upThe time-varying intensity, including the first intensity increase 532 and decrease below the up-click intensity threshold I _U As indicated by indicator 512, and then below early acknowledgement threshold I at time T2 _A As indicated by indicator 568. Input 531 then continues with a second intensity increase 536 above the down-click intensity threshold I _D Or alternatively, to above a second down-click intensity threshold I _D2 As shown in fig. 5G), as indicated by indicator 574, and a second intensity decrease 538 below the up-click intensity threshold I _U As indicated by indicator 522.

At this point, the intensity of input 531 has followed the change required for the double click. After this point, four possible successions of input 531 are shown in FIG. 5Y. One possibility shown in FIG. 5Y is lift-off or suspension of the touch input 531, as indicated by indicator 576, which in some embodiments is sufficient to meet double-tap detection criteria.

A second possibility, shown in FIG. 5Y, is for the intensity of the touch input 531 to drop below and remain below the early-confirmation threshold I _A For at least the early confirmation time threshold, as indicated by indicator 577, and thereby satisfies the double-click detection criteria. A third possibility, shown in FIG. 5Y, is for the intensity of the touch input 531 to drop below and remain below the up-click intensity threshold I _U (and above early acknowledgement threshold I) _A ) For at least a default delay time threshold, as indicated by indicator 578, and thereby satisfies the double-click detection criteria. The electronic device performs or causes performance of a second operation, such as a transition from a previous user interface to a multitasking user interface, in response to detecting that the input 531 meets the double-click detection criteria, as shown in FIG. 5Y.

A fourth possibility, shown in FIG. 5Y, is that after the second intensity decrease 538 and in the event that the touch input 531 does not satisfy the double-tap detection criterion, the intensity of the touch input 531 has a third intensity increase 580, during which the intensity is increased above the down-click intensity threshold I _D (or alternatively, increase to a corresponding down-click intensity threshold I above the historical intensity based on the touch input 531 _D ) Such as indicators579 and thus meets the three-tap detection criteria. The electronic device performs or causes performance of a third operation, such as transitioning from a previous user interface to a predefined user interface (e.g., a device settings interface) or to an easy-access mode, in response to detecting that the input 531 satisfies the triple-click detection criteria. When the presence of a triple-tap operation is configured to be detected in response to a change in intensity of the contact on the primary button, the device optionally imposes a delay in detecting the double-tap operation after detecting the second up-click to ensure that a triple-tap is not detected. The delay in detecting double clicks optionally operates in a similar manner to the delay in detecting single clicks described above with reference to fig. 5O-5Y.

5Z-5II illustrate touch input scenarios in which the length of time it takes to recognize a "long press" depends on the intensity of the touch input. For example, if the default time for identifying a long press input is assumed to be 500ms, then according to some embodiments, the time for identifying the input as a long press input is reduced to 400ms or even 300ms if the intensity of the input reaches a predefined or corresponding intensity level.

Fig. 5Z shows a graph in which the rate (sometimes referred to as the recognition rate or timer rate) varies with the intensity of the input. More specifically, when the intensity is below the first down-click intensity threshold I _D Then the rate is the default rate r ₀ . When the intensity is between the first click intensity threshold I _D And a second click intensity threshold I _D+ In time between, the rate is from a first rate r ₁ Becomes the second rate r ₂ . In some embodiments, the intensity as the input is varied from I _D To change to I _D+ The velocity is from a first velocity r ₁ Linearly to a second rate r ₂ . However, in some embodiments, the intensity of the input is varied from I _D To change to I _D+ The rate is increased from a first rate r in a plurality of discrete steps, such as one or more steps ₁ To a second rate r ₂ 。

In some embodiments, the timeout timer or counter is updated at periodic intervals according to the current rate determined by the current or most recently measured intensity of the input. For example, in some embodiments, the timeout timer or counter is updated as indicated by the pseudo-code representation in table 1 of the timeout timer update function.

In table 1, "timeout _ value" is the current value of the timer; start _ value is a default timeout period, such as 500ms; current _ intensity is the current intensity of the input or the last measured intensity; rate (current _ intensity) is a rate function that maps the current strength to rate, an example of which is shown in FIG. 5Z; timeout _ event is an event delivered to a respective software module, such as either the contact/motion module 130 or the application 136, when the timer "times out" (i.e., when timeout value reaches or falls below 0). In some embodiments, the timeout timer update function first reaches a predefined intensity threshold, such as a first down click intensity threshold I, when the intensity of the input reaches a predefined intensity threshold _D Time is called, so timeout value of the timeout timer is initialized to the default timeout period when the strength of the input reaches the predefined strength threshold.

The amount of time it takes for a timeout timer or counter to expire and then to issue a timeout event varies depending on the strength of the input. Also, in some embodiments, if the intensity of the input meets (e.g., reaches or falls below) the up-click intensity threshold, the timeout function is aborted without issuing a timeout event. For example, in some embodiments, if there is liftoff of the contact, the intensity of the input meets the up-click intensity threshold. In some embodiments, the up-click intensity threshold is the dynamic intensity threshold described above with reference to FIGS. 5A-5N.

In FIG. 5AA, the electronic device 100 receives a touch input 535 on the primary button 204 and displays a user interface on the display 112, the touch input having a measured or detected intensity. In this example, the user interface displayed is a user interface of an application. Fig. 5AA includes a first graph showing the intensity of the input 535 over a period of time, a second graph showing the value of the aforementioned timeout timer over the same period of time, and a third graph showing the timer rate during the same period of time, the timer rate being a function of the intensity of the input.

In FIG. 5AA, when the intensity of the input 535 reaches or rises above the first down-click intensity threshold I _D At time T indicated by indicator 582 ₁ At this point, a timeout timer is initialized to a starting value, such as a default timeout time for detecting a long press. Further, the intensity of the input 535 remains above the first down-click intensity threshold I _D And the value of the timeout timer is decreased at a rate determined by the strength of the input 535. Since in this example the intensity of input 535 remains constant during this time, the timeout timer decreases at a constant rate determined by the intensity of input 535.

In the example shown in FIG. 5BB, the intensity of input 543 is from time T ₁ To time T ₂ Remains constant, as indicated by indicator 583, and at time T ₂ The value of the timeout reaches zero, indicating that the timeout period for detecting a long press has expired, sometimes referred to as detecting a long press. Further, as shown in fig. 5BB, once the period for detecting a long press has expired, the electronic device 100 performs an operation (e.g., transitioning to a user help user interface) according to the respective determination. In some embodiments, the aforementioned determining comprises determining that the intensity of the input does not satisfy the up-click detection criteria before the expiration of the timeout period. Further, in some embodiments, the operations performed when the long press is detected include generating a haptic output 504. In some embodiments, the tactile output 504 is a tactile output having a FullTap (125 Hz) tactile output mode (fig. 4F).

Unlike fig. 5BB, in fig. 5II, the electronic device 100 receives the touch input 543 on the primary button 204 and displays a user interface (not shown in fig. 5II, but visible in fig. 5 AA) on the display 112, the touch input having a measured or detected intensity. Fig. 5II includes a first graph showing the intensity of the input 543 over a period of time, a second graph showing the value of the aforementioned timeout timer over the same period of time, and a third graph showing the timer rate during the same period of time, the timer rate being a function of the input intensity.

In FIG. 5II, the intensity of the input 543 remains below the first down-click intensity threshold I _D Passing through the slave time T ₁ To time T ₆ As indicated by indicator 587. The timeout timer is initialized to a starting value, such as a default timeout time for detecting a long press. Further, since the intensity of the input 535 remains below the first down-click intensity threshold I _D The value of the timeout timer is decreased at a default rate. Since in this example, the intensity of the input 535 remains below the first down-click intensity threshold I during this time period _D So the value of the timeout timer decreases at a constant default rate and therefore expires after a default period of time, in fig. 5II by the slave T ₁ To T ₆ Is shown.

Further, as shown in FIG. 5II, once the default time period for detecting a long press has expired, electronic device 100 performs an operation (e.g., transitioning to a user help user interface) according to the respective determination. In some embodiments, the aforementioned determining comprises determining that the intensity of the input does not meet the up-click detection criteria before expiration of the default timeout period. Further, in some embodiments, the operation performed when the long press is detected includes generating a haptic output 504. In some embodiments, the haptic output 504 is a haptic output having a FullTap (125 Hz) haptic output mode (fig. 4F).

In FIG. 5CC, the electronic device 100 receives a touch input 537 on the primary button 204 with a measured or detected intensity and displays a user interface on the display 112. In this example, the user interface displayed is a user interface of an application. FIG. 5CC is similar to FIG. 5AA, except that touch input 537 has a higher or greater magnitude than touch input 535 of FIG. 5 AA. Thus, the timer rate in the example shown in fig. 5CC is higher or greater than the timer rate in the example shown in fig. 5AA, and the value of the timeout timer is decreased in fig. 5CC at a faster rate than in fig. 5 AA.

In the example shown in FIG. 5DD, the intensity of input 537 is from time T ₁ To time T ₃ Is kept constant, this being at time T ₂ Before, and the value of the timeout timer is at time T ₃ Reaching zero, as indicated by indicator 584, indicates that the timeout period for detecting a long press has expired. Therefore, the timeout period (T) for detecting a long press ₁ To T ₃ ) The timeout period (T) in the example shown in FIGS. 5CC-5DD is greater than that in the example shown in FIGS. 5AA-5BB ₁ To T ₂ ) Short because the intensity of the touch input is higher in the example shown in fig. 5CC-5DD than in the example shown in fig. 5AA-5BB, and the corresponding timer rate is higher in the example shown in fig. 5CC-5DD than in the example shown in fig. 5AA-5 BB.

Further, as shown in fig. 5DD, once the period for detecting a long press has expired, the electronic device 100 performs an operation (e.g., transitioning to a user help user interface) according to the respective determination. In some embodiments, the aforementioned determining comprises determining that the intensity of the input does not meet the up-click detection criteria before the expiration of the timeout period.

In FIG. 5EE, the electronic device 100 receives a touch input 539 on the primary button 204 and displays a user interface on the display 112, the touch input having a measured or detected intensity. In this example, the user interface displayed is a user interface of an application. FIG. 5EE is similar to FIGS. 5AA and 5CC, except that touch input 539 has a higher or greater magnitude than touch input 535 in FIG. 5AA and touch input 537 in FIG. 5 CC. Thus, the timer rate in the example shown in fig. 5EE is higher or greater than the timer rate in the examples shown in fig. 5AA and 5CC, and the value of the timeout timer is decreased in fig. 5EE at a faster rate than in fig. 5AA and 5 CC.

In the example shown in FIG. 5FF, the intensity of input 539 is from time T ₁ To time T ₄ Is kept constant, this being at time T ₃ And time T ₂ Before, and the value of the timeout timer is at time T ₄ Reaching zero, as indicated by indicator 585, indicates that the timeout period for detecting a long press has expired. Therefore, the timeout period (T) for detecting a long press ₁ To T ₄ ) The timeout period (T) in the example shown in FIGS. 5EE-5FF is greater than that in the example shown in FIGS. 5AA-5BB ₁ To T ₂ ) And the timeout period (T) in the example shown in FIGS. 5CC-5DD ₁ To T ₃ ) Short because the intensity of the touch input is higher in the example shown in FIG. 5EE-5FF than in the examples shown in FIGS. 5AA-5BB and 5CC-5DD, and the corresponding timer rate is higher in the example shown in FIG. 5EE-5FF than in the examples shown in FIGS. 5AA-5BB and 5CC-5 DD.

Further, as shown in fig. 5FF, once the period for detecting a long press has expired, the electronic device 100 performs an operation (e.g., transitioning to a user help user interface) according to the respective determination. In some embodiments, the aforementioned determining comprises determining that the intensity of the input does not meet the up-click detection criteria before the expiration of the timeout period.

In FIG. 5GG, the electronic device 100 receives a touch input 541 on the main button 204 and displays a user interface on the display 112, the touch input having a measured or detected intensity. In this example, the user interface displayed is a user interface of an application. FIG. 5GG is similar to FIG. 5AA, and touch input 541 has the same or similar intensity as touch input 535 in FIG. 5 AA. Thus, the timer rate in the example shown in FIG. 5GG is the same as or similar to the timer rate in the example shown in FIG. 5AA, and the value of the timeout timer is decreased in FIG. 5GG at the same or similar rate as in FIG. 5 AA.

In the example shown in FIG. 5HH, the intensity of input 541 is at time T ₁ And time T ₅ To change or change. In this example, time T is indicated by indicator 586 ₅ The intensity of the input 541 is at or about the intensity level associated with the maximum timer rate of the timeout timer. Due to time T ₁ And time T ₅ At a timer rate of time T ₁ And time T ₅ According to the strength of the input 541, so that the rate of decrease of the value of the timeout timer is at time T ₁ And time T ₅ To change between. In this example, the rate of decrease of the value of the timeout timer is at the intensity of the input at time T _3b And accelerates when increasing. Alternatively, if the intensity of the input is at time T _3b Having decreased, the rate of decrease of the value of the timeout timer may have slowed.

In this example, the value of the timeout timer is at time T ₅ Reaching zero indicates that the timeout period for detecting a long press has expired. The timeout period is time T in this example ₁ To a time T ₅ The period of (c).

Further, as shown in FIG. 5HH, once the period for detecting the long press has expired, electronic device 100 performs an operation (e.g., transitioning to a user help user interface) according to the respective determination. In some embodiments, the aforementioned determining comprises determining that the intensity of the input does not meet the up-click detection criteria before the expiration of the timeout period.

6A-6F are flow diagrams illustrating a method 600 of monitoring input on an intensity-sensitive input element and detecting an up-click and/or a down-click in the monitored input using one or more intensity thresholds based on a previous input intensity of the input. Method 600 is performed at an electronic device (e.g., device 300, FIG. 3; or portable multifunction device 100, FIG. 1A) having a display, a touch-sensitive surface, and one or more sensors 165 for detecting intensity of contacts with the touch-sensitive surface. In some embodiments, the electronic device performing method 600 has a main button 204 that includes one of the sensors 165 in addition to the touch-sensitive surface. In some embodiments, the primary button 204 is separate from the display, and optionally includes a set of one or more intensity sensors separate from the intensity sensor used to detect the intensity of the input on the display. In some embodiments, the master button 204 is a virtual master button displayed on the display (e.g., having a set of one or more intensity sensors separate from the intensity sensor used to detect the intensity of the input on the display, or optionally using an intensity sensor integrated into the display to determine the intensity of the input with the virtual master button). In some embodiments, the display is a touch screen display and the touch sensitive surface is on or integrated with the display. In some embodiments, the display is separate from the touch-sensitive surface. Some operations in method 600 are optionally combined, and/or the order of some operations is optionally changed.

As described below, the method 600 provides a way to accurately determine user intent for whether a touch input includes an up click or a down click by considering the intensity of the user's input during the portion of the input immediately prior to the intensity decrease or immediately prior to the intensity increase. The method 600 reduces "positive misrecognitions," such as inputs incorrectly detected as including an up click or a down click, and "negative misrecognitions," such as inputs incorrectly detected as not including a corresponding up click or down click, thereby generating a more efficient human-machine interface. For battery-driven electronic devices, taking into account the priority strength of the user's touch input enables the user to input gestures, such as one or more of single, double, and triple tap gestures, more quickly and efficiently, which saves power and increases the time between battery charges.

The device detects (602) a first increase in intensity of the input on the input element that satisfies the down-click detection criteria, and detects (608) a decrease in first intensity of the contact after detecting the first increase in intensity of the input on the input element. For example, as shown in FIG. 5B, the touch input 505 on the primary button 204 has a time-varying intensity, including a first increase in intensity 520 (sometimes referred to as a first down-click) to a peak intensity I _Peak, The peak intensity is higher than the first click intensity threshold I _D . As shown in FIG. 5C, after the first intensity increase, the intensity of the touch input on the primary button 204 decreases, and the electronic device detects a first intensity decrease 522 of the contact, sometimes referred to as a first up-click.

In some embodiments, the input on the input element comprises (604) an input on a touch-sensitive surface. In the example shown in fig. 5B and 5C, the input on the input element is input 505 on the primary button 204. Further, for a first increase in intensity, the down-click detection criteria require (606) that the intensity of the input increase above a first down-click intensity threshold, such as the first down-click intensity threshold I shown in FIGS. 5B and 5C _D So that the click detection criteria are met. In some embodiments, and in the examples shown in fig. 5B and 5C, the down-click intensity threshold is a fixed, predefined value, and thus is neither time-varying nor based on detecting an inputIncreases (602) the intensity of the immediately preceding input.

In response to detecting (610) a first decrease in intensity of the input (e.g., decrease 552, FIG. 5C), the method 600 includes determining (612) whether the first decrease in intensity of the input satisfies up-click detection criteria, wherein (A) for the first decrease in intensity, the up-click detection criteria require that the intensity of the input decrease below a first up-click intensity threshold (e.g., threshold I) _U FIG. 5C) such that the up-click detection criteria are met, (B) a first up-click intensity threshold (e.g., threshold I) _U Fig. 5C) is selected based on the intensity of the input during an increase in intensity of the detected contact (e.g., first increase 520, fig. 5B-5C) prior to detecting the first decrease in intensity of the input. As described in more detail below, the first up-click intensity threshold is based on the peak intensity I entered prior to the first reduction 522 _Peak (see fig. 5C), or another characteristic intensity or representative intensity of the input. In some embodiments, the peak intensity, or the characteristic intensity or the representative intensity, is multiplied by a multiplier, such as a value between 0.6 and 0.75, to determine a first up-click intensity threshold.

In some embodiments, the first up-click intensity threshold is time-varying according to a low-pass filtering of a detected intensity input during the first decrease in intensity of the contact (632). For example, FIG. 5K illustrates an up-click intensity threshold I that dynamically changes as the intensity of the input changes during a first intensity reduction 542 _U(t) 546. In the example described above with reference to fig. 5K, during the first intensity reduction 542, the intensity of the input is low pass filtered, generating a first time-varying value I _LPup 544. The first time-varying value is then multiplied by a fixed value, such as 0.7, or an intensity-based value, such as the ratio shown in FIG. 5J, to generate a time-varying up-click intensity threshold I _U(t) Wherein the "(t)" symbol indicates that the value is time-varying.

In some embodiments, the up-click intensity threshold I is described above with reference to FIGS. 5J-5N _U The ratio to the intensity value representing the intensity of the contact is varied (642) based on the intensity value representing the intensity of the contact (e.g., the input low pass filtered peak intensity value)When the up-click intensity is based on a first intensity value (e.g., intensity I) representing the intensity of the contact _D1 FIG. 5J), the ratio of the up-click intensity threshold to the first intensity value has a first value (e.g., a) ₂ Fig. 5J); when the up-click intensity is based on a second intensity value (e.g., intensity I) representing the intensity of the contact that is greater than the first intensity value _D2 Fig. 5J), the ratio of the up-click intensity threshold to the second intensity value has a second value (e.g., a 1) that is different from (e.g., lower or higher than) the first value (e.g., the ratio is 0.6 when the low-pass filtered present intensity of the contact is 500g, and the ratio is 0.73 when the low-pass filtered present intensity of the contact is 300 g). In some embodiments, the rate of change of intensity based on contact is to account for increased thermal drift at higher intensity levels.

In some embodiments, as described above with reference to fig. 5J-5N, the magnitude of the up-click intensity threshold is set (644) by multiplying an intensity value representative of the intensity of the contact (e.g., the peak intensity of the contact before the decrease in intensity of the contact is detected, or the low-pass filtered current intensity of the contact) by an adjustment value (e.g., a value between 0 and 1) determined based at least in part on the magnitude of the intensity value representative of the intensity of the contact (e.g., the peak intensity of the contact before the decrease in intensity of the contact is detected, or the low-pass filtered current intensity of the contact).

In some embodiments, as described above with reference to fig. 5J-5N, the ratio of the up-click intensity threshold to the intensity value representing the intensity of the contact varies according to the maximum characteristic intensity of the input (646). For example, the multiplier varies from a predefined maximum value when the detected intensity is below a first intensity value (e.g., 300 g) to a predefined minimum value when the detected intensity is above a second intensity value (e.g., 500 g).

In some embodiments, as shown in FIG. 5J, the ratio of the up-click intensity threshold to the intensity value representing the intensity of the contact smoothly varies (648) from a predefined maximum value to a predefined minimum value as the intensity value representing the intensity of the contact varies between a first intensity value and a second intensity value, where the first intensity value is less than the second intensity value. In some embodiments, "smoothly varying" means that the ratio changes in two or more steps, or three or more steps, from a predefined maximum value to a predefined minimum value as the intensity value representing the intensity of the contact changes between the first intensity value and the second intensity value. More generally, the ratio of the up-click intensity threshold to the intensity value representing the intensity of the contact monotonically varies from a predefined maximum value to a predefined minimum value as the intensity value representing the intensity of the contact varies between the first intensity value and the second intensity value.

In some embodiments, the up-click intensity threshold is not less than a predefined minimum up-click intensity threshold (649). For example, where the aforementioned ratio or multiplier is applied to an intensity value representing the intensity of the contact, the up-click intensity threshold is set to be the greater of a predefined minimum up-click intensity threshold (e.g., 130 g) and an up-click intensity threshold determined using the ratio or multiplier. In the form of a formula, imposing a predefined minimum up-click intensity threshold may be expressed as:

I _U ＝max(IT _min ,I _{representative} *β)

wherein IT _min Is a predefined minimum up-click intensity threshold, I _{representative} Is an intensity value representing the intensity of the contact, and β is the aforementioned ratio or multiplier.

In some embodiments, examples of which are discussed above with reference to FIGS. 5J-5N, the up-click intensity threshold is determined according to a multiplier (634) applied to the characteristic intensity of the input having a value greater than zero and less than one. This multiplier is effectively the aforementioned ratio, multiplied by an intensity value representing the intensity of the contact. As described in more detail elsewhere, the intensity value representing the intensity of the contact is, in some embodiments, the peak intensity prior to the up-click or the lowest intensity immediately preceding the down-click. In some other embodiments, the intensity values representing the intensity of the contact are low-pass filtered intensity values of the respective portions of the input, as shown in fig. 5L for the second down click and in fig. 5K and 5N for the first or second up click.

In some embodiments, the multiplier varies according to the characteristic intensity of the input (636). For example, in some embodiments, the multiplier smoothly varies (638) from a predefined maximum value to a predefined minimum value as the characteristic intensity of the input varies between a first intensity value and a second intensity value, wherein the first intensity value is less than the second intensity value. In some embodiments, "smoothly varying" means that the multiplier varies from a predefined maximum value to a predefined minimum value in two or more steps, or three or more steps, as the intensity value representative of the intensity of the contact varies between the first intensity value and the second intensity value. For example, the multiplier varies from a predefined maximum value when the detected intensity is below a first intensity value (e.g., 300 g) to a predefined minimum value when the detected intensity is above a second intensity value (e.g., 500 g). More generally, the multiplier monotonically varies from a predefined maximum value to a predefined minimum value as the intensity value representing the intensity of the contact varies between the first intensity value and the second intensity value.

In some implementations, the up-click intensity threshold is the greater of a predefined minimum up-click intensity threshold and a value determined from a multiplier (e.g., a value greater than zero and less than one) of the characteristic intensity applied to the input (640). In the form of a formula, imposing a predefined minimum up-click intensity threshold may be expressed as:

I _U ＝max(IT _min ,I _char *β)

Wherein IT _min Is a predefined minimum up-click intensity threshold, I _char Is the characteristic intensity of the contact, and β is the multiplier.

Further, in accordance with a determination (616) that the first decrease in intensity of the input satisfies the up-click detection criteria, the electronic device provides first feedback indicating that the first decrease in intensity of the input is identified as an up-click, and in accordance with a determination (616) that the decrease in intensity of the input does not satisfy the up-click detection criteria, the electronic device foregoes providing the first feedback. For example, as shown in FIG. 5C, when the decrease in intensity 522 of the input satisfies the up-click detection criteria (indicated by indicator 512), the device provides a tactile output 502 to indicate that the decrease in intensity 522 of the input is identified as an up-click, and as shown in FIG. 5F, when the decrease in intensity 530 of the input does not satisfy the up-click detection criteria, the electronic device does not generate a tactile output (e.g., tactile output 502) to indicate that the decrease in intensity (e.g., decrease 530) of the input is identified as an up-click.

Having now considered various aspects of detecting a first click, consider now various aspects of detecting a second click. In particular, the electronic device detects (614) a second increase in intensity of the input after determining that the first decrease in intensity satisfies the up-click detection criteria. Various scenarios in which a second increase in intensity is detected are discussed above with reference to FIGS. 5D-5E, 5G-5I, and 5L-5N.

In response to detecting the second increase in intensity of the input (620), in accordance with a determination that the second increase in intensity of the input satisfies the down-click detection criteria, the electronic device provides second feedback indicating that the second increase in intensity was identified as part of the double-click input. For example, as described above with reference to FIG. 5G, the second tactile output 503 is generated in conjunction with detecting a release click of the second click of the input 511 on the primary button 204. For example, in some embodiments, the second (press-and-click) tactile output is generated with one or more tactile output generators 167 (fig. 1A) of the electronic device. As another example, providing the second feedback includes generating audio output with one or more speakers (e.g., speaker 111, fig. 1A), and/or displaying a change in a graphical user interface displayed on a display (e.g., display 112, fig. 5G, 5H) of the electronic device according to a double-click operation, such as displaying a multitasking user interface, as shown in fig. 5H.

For a second increase in intensity, the down-click detection criteria require (622) that the intensity of the input increase above a second down-click intensity threshold (e.g., intensity threshold I) _D2 Fig. 5G) such that the down-click detection criteria are satisfied, and a second down-click intensity threshold for a second increase in intensity is selected based on the intensity entered during the first decrease in intensity of the contact (624). In the example shown in FIG. 5G, the second down-click intensity threshold for the second increase in intensity is based on the lowest intensity I entered during the first decrease in intensity 534 of the contact _Valley To select.

In some embodiments, the second down-click intensity threshold is time-varying, and the second down-click intensity threshold is based on a second number of inputsLow pass filtering of the detected intensity input during a second intensity reduction of the detected contact after the one intensity reduction is selected (626). For example, the decrease in intensity of the input satisfies the up-click detection criteria. FIG. 5L shows a time-varying second down-click intensity threshold I _D(t) An example of 554 that is based on low pass filtering (I) of the detected intensity input during the second intensity increase 550 of the contact _LPdown 552 ) to select or determine. In some implementations, the second intensity of the contact is increased 550 by the low pass filtered intensity (I) of the detected intensity input _LPdown 552 Initially set to the lowest intensity I of the input during the first intensity decrease 534 of the contact at the beginning of the second intensity increase 550 of the contact _Valley 。

In response to detecting the second increase in intensity of the input (620), in accordance with a determination (628) that the second increase in intensity of the input does not satisfy the down-click detection criteria, the electronic device foregoes providing the second feedback. For example, in FIG. 5G, if the input is to reach the second down-click intensity threshold I in intensity _D2 Previously stopped (e.g., if the contact is to be lifted off), the second increase in intensity of the input may not meet the down-click detection criteria, and the electronic device may forgo providing the second feedback.

In some embodiments, method 600 includes generating (650) a first tactile output in conjunction with detecting that an increase in intensity of an input satisfies a down-click detection criterion. For example, as described above with reference to FIG. 5C, the first tactile output 502 is generated in conjunction with detecting a press click during the decrease 522 in intensity of the input 505 on the primary button 204.

In some embodiments, the method 600 includes generating (652) a second tactile output in conjunction with detecting that the decrease in intensity of the input satisfies the up-click detection criteria. For example, as described above with reference to FIG. 5G, the second tactile output 503 is generated in conjunction with detecting a release click of the second click of the input 511 on the primary button 204.

In some embodiments, method 600 includes generating (654) a response for display on a display of the electronic device in response to detecting an increase in intensity of an input on the input element that satisfies the down-click detection criteria. In one example, the displayed response is a response that visually distinguishes an object or region of the user interface in the user interface whose location corresponds to the input.

In some embodiments, providing the first feedback includes generating (656) a response that is displayed on a display of the electronic device. In the example shown in fig. 5B-5C, for example, the response, which may be referred to as a first click or single click response, is a switch from displaying the user interface of the first application (e.g., a timer application) to displaying the application launch user interface.

In some embodiments, method 600 includes generating (658) the same first tactile output in conjunction with detecting multiple instances of an increase in intensity of an input detected on an input element that satisfies a down-click detection criterion, including instances in which the down-click detection criterion is associated with a different down-click intensity threshold. For example, even when the first and second down clicks are detected at different intensity thresholds, the same tactile output is generated for both the first and second down clicks (e.g., tactile output having a MicroTap (270 Hz) tactile output mode, fig. 4F).

In some embodiments, method 600 includes generating (660) the same second tactile output in conjunction with detecting multiple instances in which the decrease in intensity of the input detected on the input element satisfies the up-click detection criteria, including instances in which the up-click detection criteria are associated with different up-click intensity thresholds. For example, even when the first and second up-clicks are detected at different intensity thresholds, the same tactile output is generated for both the first and second up-clicks (e.g., a tactile output having a MiniTap (270 Hz) tactile output mode, FIG. 4F). In some embodiments, the up-click haptic output mode has a lower gain than the down-click haptic output mode (e.g., the up-click haptic output mode is a reduced magnitude version of the down-click haptic output mode).

In some embodiments, method 600 includes, after detecting a second increase in intensity of the input on the input element (e.g., increase in intensity 536 of input 513, fig. 5H), detecting 662 a second decrease in intensity of the contact (e.g., decrease in intensity 538 of input 513, fig. 5H), and providing a third feedback (e.g., performing a double-tap operation, an example of which is described above with reference to fig. 5H) in response to detecting the second decrease in intensity of the input indicating that the second decrease in intensity is identified as a release click input. In such embodiments, for the second intensity reduction, the up-click detection criteria require that the intensity of the input be reduced below a second up-click intensity threshold so that the up-click detection criteria are met; and the second up-click intensity threshold is selected based on the intensity of the input during the second increase in intensity of the contact. In addition, in accordance with a determination that the second decrease in intensity of the input does not satisfy the up-click detection criteria, the electronic device forgoes providing the third feedback (e.g., by forgoing performing a double-click operation).

In some embodiments, the third feedback is generated or initiated at or immediately after the time indicated by the indicator 522 in FIG. 5H at which time the second intensity reduction satisfies the up-click detection criteria. In some embodiments, the third feedback is or includes a transition to a multitasking user interface, as shown in fig. 5H. In some embodiments, the third feedback is or includes generating a haptic output 503, as shown in fig. 5H. In some embodiments, the haptic output 503 is a haptic output having a MiniTap (270 Hz) haptic output mode (fig. 4F).

It should be appreciated that the particular order in which the operations in fig. 6A-6F are described is merely exemplary and is not intended to suggest that the order is the only order in which the operations may be performed. One of ordinary skill in the art will recognize a variety of ways to reorder the operations described herein. Additionally, it should be noted that details of other processes described herein in connection with other methods described herein (e.g., method 700 and method 800) also apply in a similar manner to method 600 described above in connection with fig. 6A-6F. For example, the method 600 of monitoring inputs on intensity-sensitive input elements and detecting up-clicks and/or down-clicks in the monitored inputs using one or more intensity thresholds based on previous input intensities of the inputs, as described above with reference to fig. 6A-6E, optionally has one or more characteristics of the single-click acceleration detection method described herein with reference to method 700 and/or the long-press input acceleration detection method described herein with reference to method 800. For the sake of brevity, these details are not repeated here.

According to some embodiments, fig. 9 illustrates a functional block diagram of an electronic device 900 configured in accordance with the principles of various described embodiments. The functional blocks of the device are optionally implemented by hardware, software, or a combination of hardware and software that perform the principles of the various described embodiments. Those skilled in the art will understand that the functional blocks described in fig. 9 are optionally combined or separated into sub-blocks in order to implement the principles of the various described embodiments. Thus, the description herein optionally supports any possible combination or separation or further definition of the functional blocks described herein.

As shown in fig. 9, electronic device 900 includes a display unit 902 (e.g., corresponding to display 112) configured to display a user interface, an intensity-sensitive input unit 904 (e.g., corresponding to main button 204) configured to detect an intensity of a user input with the input element, and a processing unit 910 coupled to display unit 902 and intensity-sensitive input unit 904. In some embodiments, electronic device 900 also includes a touch-sensitive surface unit 906 for receiving touch inputs on a surface (such as a display surface of display unit 902), and one or more haptic output units 908 for generating haptic outputs, also coupled to processing unit 910. In some embodiments, processing unit 910 includes one or more of the following sub-units: an intensity monitoring unit 912, an up click determination unit 914, a down click determination unit 916, and a feedback unit 918. In some embodiments, feedback unit 918 includes a responsive display unit 920, and a haptic output unit 922.

The processing unit 910 is configured to: a first intensity increase of the input on the input element that satisfies the down-click detection criteria is detected (e.g., with intensity monitoring unit 912), and a first intensity of the contact is detected to decrease after the first intensity increase of the input on the input element is detected. In response to detecting a first decrease in intensity of the input, the processing unit 910 determines (e.g., with the up-click determination unit) whether the first decrease in intensity of the input satisfies up-click detection criteria, where for the first decrease in intensity, the up-click detection criteria require that the intensity of the input decrease below a first up-click intensity threshold selected based on the intensity of the input during the increase in intensity of the detected contact before the first decrease in intensity of the input is detected so that the up-click detection criteria are satisfied.

Processing unit 910 provides first feedback indicating that the first decrease in intensity of the input is identified as an up-click (e.g., using feedback unit 918) in accordance with a determination that the first decrease in intensity of the input satisfies up-click detection criteria, and forgoes providing the first feedback in accordance with a determination that the first decrease in intensity of the input does not satisfy up-click detection criteria.

In some embodiments, for a first increase in intensity, the down-click detection criteria require that the intensity of the input increase above a first down-click intensity threshold in order for the down-click detection criteria to be satisfied. In some such embodiments, the processing unit 910 is further configured to, upon determining that the first decrease in intensity satisfies the up-click detection criteria, detect (e.g., with intensity monitoring unit 912) a second increase in intensity of the input; and in response to detecting the second increase in intensity of the input, in accordance with a determination (e.g., with the down-click determination unit 916) that the second increase in intensity of the input satisfies the down-click detection criteria, providing (e.g., with the feedback unit 918) second feedback indicating that the second increase in intensity is identified as part of the double-click input. For a second increase in intensity, the down-click detection criteria require that the intensity of the input increase above a second down-click intensity threshold so that the down-click detection criteria are met. In these embodiments, the second down-click intensity threshold for the second intensity increase is selected based on the intensity of the input during the first intensity decrease of the contact, and the processing unit 910 foregoes providing the second feedback in accordance with a determination (e.g., with the down-click determination unit 916) that the second intensity increase of the input does not satisfy the down-click detection criteria.

In some embodiments, the second down-click intensity threshold is time-varying and is selected by the processing unit 912 (e.g., with the down-click determination unit 916) based on low-pass filtering of the detected intensity of the input during the second increase in intensity of the detected contact after the first decrease in intensity of the input.

In some embodiments, for a first increase in intensity, the down-click detection criteria require that the intensity of the input increase above a first down-click intensity threshold in order for the down-click detection criteria to be met.

In some embodiments, an input on intensity-sensitive input unit 904 comprises an input on a touch-sensitive surface.

In some embodiments, the first up-click intensity threshold is time-varying based on a low-pass filtering of the detected intensity of the input during the first decrease in intensity of the contact.

In some embodiments, a ratio of the up-click intensity threshold to the intensity value representative of the intensity of the contact varies based on the intensity value representative of the intensity of the contact such that the ratio of the up-click intensity threshold to the first intensity value has a first value when the up-click intensity is based on the first intensity value representative of the intensity of the contact; when the up-click intensity is based on a second intensity value representing the contact intensity that is greater than the first intensity value, a ratio of the up-click intensity threshold to the second intensity value has a second value that is different from the first value.

In some embodiments, the magnitude of the up-click intensity threshold is set (e.g., by the processing unit 910 or the up-click determination unit 914) by multiplying the intensity value representing the intensity of the contact by an adjustment value determined based at least in part on the magnitude of the intensity value representing the intensity of the contact. In some embodiments, the ratio of the up-click intensity threshold to the intensity value representing the intensity of the contact varies according to the maximum characteristic intensity of the input. In some embodiments, the ratio of the up-click intensity threshold to the intensity value representing the intensity of the contact smoothly changes from a predefined maximum value to a predefined minimum value as the intensity value representing the intensity of the contact changes between a first intensity value and a second intensity value, where the first intensity value is less than the second intensity value.

In some embodiments, the up-click intensity threshold is not less than a predefined minimum up-click intensity threshold.

In some implementations, the up-click intensity threshold (by the processing unit 910 or the up-click determination unit 914) is determined from a multiplier applied to the characteristic intensity of the input that has a value greater than zero and less than one. In some embodiments, the multiplier varies according to the characteristic intensity of the input. In some implementations, the multiplier smoothly changes from a predefined maximum value to a predefined minimum value as the characteristic intensity of the input changes between a first intensity value and a second intensity value, where the first intensity value is less than the second intensity value.

In some implementations, the up-click intensity threshold is the greater of a predefined minimum up-click intensity threshold and a value determined from a multiplier applied to the characteristic intensity of the input having a value greater than zero and less than one.

In some embodiments, the processing unit 910 is further configured to generate the first tactile output (e.g., with the tactile output unit 922 of the feedback unit 918) in conjunction with detecting that the increase in intensity of the input satisfies the down-click detection criteria.

In some embodiments, the processing unit 910 is further configured to generate the second tactile output (e.g., with the tactile output unit 922 of the feedback unit 918) in conjunction with detecting that the decrease in intensity of the input meets the up-click detection criteria.

In some embodiments, the processing unit is further configured to generate a response (e.g., with response display unit 920 of feedback unit 918) for display by display unit 902 of the electronic device in response to detecting an increase in intensity of an input on the input element that satisfies the down-click detection criteria.

In some embodiments, providing the first feedback includes generating a response (e.g., with response display unit 920 of feedback unit 918) for display by display unit 902 of the electronic device.

In some embodiments, the processing unit is further configured to generate the same first tactile output in conjunction with detecting a plurality of instances in which an increase in intensity of an input detected on the input element satisfies the down-click detection criteria (including instances in which the down-click detection criteria are associated with different down-click intensity thresholds).

In some embodiments, the processing unit is further configured to, after detecting an increase in the second intensity of the input on the input element, detect a decrease in the second intensity of the contact; and in response to detecting the second decrease in intensity of the input, in accordance with a determination that the second decrease in intensity of the input satisfies the up-click detection criteria, provide third feedback indicating that the second decrease in intensity is identified as the up-click input. In such embodiments, for the second intensity reduction, the up-click detection criteria require that the intensity of the input be reduced below a second up-click intensity threshold so that the up-click detection criteria are met; and the second up-click intensity threshold is selected based on the intensity of the input during the second increase in intensity of the contact. In such embodiments, the processing unit is configured to forgo providing the third feedback in accordance with (e.g., the up-click determination unit 914) a determination that the second decrease in intensity of the input does not satisfy the up-click detection criteria.

In some embodiments, the processing unit is further configured to generate the same second tactile output (e.g., with tactile output unit 922 of feedback unit 918) in conjunction with multiple instances of detecting that the decrease in intensity of the input detected on the input element satisfies the up-click detection criteria (including instances in which the up-click detection criteria are associated with different up-click intensity thresholds).

The operations in the information processing methods described above with reference to fig. 6A-6F are optionally implemented by running one or more functional modules in an information processing apparatus, such as a general-purpose processor (e.g., as described above with respect to fig. 1A and 3) or an application-specific chip.

The operations described above with reference to fig. 6A-6F are optionally implemented by components depicted in fig. 1A-1B or fig. 3. For example, the detection operations 602, 608, 614, etc. and the determination operation 612, etc. are optionally implemented by the contact/motion module 130, the feedback operation providing the haptic output is implemented by the haptic feedback module 133, and some other operations are optionally implemented by the event classifier 170, the event recognizer 180, and the event handler 190. Event monitor 171 in event sorter 170 detects a contact on touch-sensitive display 112 and event dispatcher module 174 communicates the event information to application 136-1. A respective event recognizer 180 of application 136-1 compares the event information to respective event definitions 186, and determines whether a first contact at a first location on the touch-sensitive surface (or whether rotation of the device) corresponds to a predefined event or sub-event, such as selection of an object on a user interface, or rotation of the device from one orientation to another. When a respective predefined event or sub-event is detected, the event recognizer 180 activates an event handler 190 associated with the detection of the event or sub-event. Event handler 190 optionally uses or calls data updater 176 or object updater 177 to update application internal state 192. In some embodiments, event handler 190 accesses a respective GUI updater 178 to update the content displayed by the application. Similarly, it will be clear to those skilled in the art how other processes may be implemented based on the components depicted in fig. 1A-1B.

7A-7E are flow diagrams illustrating a method 700 of monitoring an input on an intensity sensitive input element and detecting whether the input is a single tap or a double tap. Method 700 is performed at an electronic device (e.g., device 300, FIG. 3; or portable multifunction device 100, FIG. 1A) having a display, a touch-sensitive surface, and one or more sensors 165 for detecting intensity of contacts with the touch-sensitive surface. In some embodiments, the electronic device performing method 700 has a main button 204 that includes one of the sensors 165 in addition to the touch-sensitive surface. In some embodiments, the primary button 204 is separate from the display, and optionally includes a set of one or more intensity sensors separate from the intensity sensor used to detect the intensity of the input on the display. In some embodiments, the home button 204 is a virtual home button displayed on the display (e.g., having a set of one or more intensity sensors separate from the intensity sensor used to detect the intensity of the input on the display, or optionally using an intensity sensor integrated into the display to determine the intensity of the input utilizing the virtual home button). In some embodiments, the display is a touch screen display and the touch sensitive surface is on or integrated with the display. In some embodiments, the display is separate from the touch-sensitive surface. Some operations in method 700 are optionally combined, and/or the order of some operations is optionally changed.

As described below, the method 700 provides a way to accurately determine user intent for whether a touch input is a single or double click, expeditiously, by considering the intensity of the user's input during the portion of the input immediately following the recognition of the first event (e.g., the first decrease in intensity of the input has dropped below the up-click intensity threshold). The method 700 reduces latency by recognizing a single click faster than would otherwise be possible, while avoiding "false positives" (such as inputs being incorrectly detected as single clicks), thereby generating a more efficient human-machine interface. For battery-driven electronic devices, taking into account the priority strength of the user's touch input enables the user to input gestures, such as single or double clicks, more quickly and efficiently, which saves power and increases the time between battery charges.

The device detects (702) a change in intensity of an input on the input element (e.g., input 523 on primary button 204, fig. 5O), including detecting (702) an increase in intensity of the input on the input element (e.g., increase 532, fig. 5O), followed by a decrease in intensity of the input on the input element (e.g., decrease 534, fig. 5O). Depending on the situation, an increase in intensity followed by a decrease in intensity is sometimes referred to as a click, or a first click or click. In some embodiments, the input on the input element is or includes an input on a touch-sensitive surface (704), such as a surface of the main button 204 (fig. 5O) or a touch-sensitive surface of a touch-sensitive display 112 of the electronic device. In some embodiments, detecting a change in intensity of an input on the input element includes continuously detecting (706) the input on the input element.

The method 700 also includes identifying (708) at least a portion of the change in intensity of the input as a first input event associated with the first operation. As described above, the first input event is sometimes referred to as a click or a first click. One example is described above with reference to fig. 5O.

After identifying the first input event, the method 700 includes delaying (710) execution of the first operation while monitoring for a subsequent change in intensity of the input for a second input event, wherein the delaying is limited by a default delay time period. Thus, the performance of the first operation (such as changing the displayed user interface from the currently displayed application user interface to the home screen or the application launch user interface) is delayed or deferred while the device continues to monitor for changes in the intensity of the input. As described above with reference to fig. 5P, in some embodiments, monitoring of the change in intensity of the input ceases if there is liftoff of the contact, so performance of the first operation is not deferred after liftoff is detected.

The method 700 further includes, after delaying execution of the first operation (712): in accordance with a determination that a second input event (e.g., has been identified before the default delay period has elapsed), the second operation is performed (714) and execution of the first operation is aborted. As described above with reference to fig. 5W, this situation corresponds to the detection of a double click, sometimes by detecting a "half" click before the default delay time expires.

The method 700 further includes, after delaying execution of the first operation (712): in accordance with a determination that the early-confirmation criteria for the first input event have been met before the default delay time period has elapsed while the second input event is not recognized, a first operation is performed (716) before the default delay time period has elapsed. As described above with reference to fig. 5Q-5V, when the early-confirmation criterion is met before the default delay period has elapsed and the second input event is not recognized, a single click is recognized and acted upon by performing the first action at an accelerated rate (e.g., before the default delay period has elapsed). This reduces the delay of the device recognizing the user input as a single click. Various ways in which the early-confirmation criteria for the first input event may be met are described in more detail below with reference to operations 722-738, and are also explained in some detail above with reference to FIGS. 5Q-5V.

The method 700 further includes, after delaying execution of the first operation (712): in accordance with a determination that the default delay time period has elapsed without the early-confirmation criteria for the first input event being satisfied and the second input event being unrecognized, the first operation is performed (718) once the default delay time period has elapsed. As described above with reference to fig. 5S, this situation occurs when the early-confirmation criteria are not met before the default delay time period has elapsed without the second input event being recognized, so that only after the default delay time period has elapsed is a single click recognized and acted upon by performing the first action. This is effectively a backup or default, where detecting that the input is a single click and not a double click takes a normal or default amount of time.

In some embodiments, the early-validation criteria for the first input event (used in operation 716, described above) includes a criterion that the intensity of the input remains below a validation intensity threshold for longer than an early-validation time threshold (720). For example, in FIG. 5Q, the early-confirmation criteria for the first input event include the intensity of the input remaining below a specific up-click intensity threshold I _U Low confirmation intensity threshold I _A Lasting longer than the early determination time threshold, which in fig. 5Q and 5R corresponds to T ₂ And T ₃ The length of time in between. For comparison, the default delay period is illustrated in fig. 5S as corresponding to T ₁ (click detection) with T ₄ The length of time in between, or alternatively, T _2A (click-off detection) with T ₄ The length of time in between, as described in more detail above with reference to fig. 5S.

In some embodiments, the early acknowledgement time threshold is less than half of the default delay period (722). For example, in some embodiments, the default delay period is 500ms, while the early acknowledgement time threshold is 150ms, or 200ms, or a value between 150ms and 240 ms.

In some embodiments, the confirmation intensity threshold (e.g., confirmation intensity threshold I in FIGS. 5O-5Y) _A ) Below an up-click intensity threshold for identifying a second input event (e.g., up-click intensity threshold I in fig. 5O-5Y) _U ) (724). For example, in some embodiments, the confirmation intensity threshold is 150g, while the up-click intensity threshold is 200g. In some other embodiments, the acknowledgementThe intensity threshold is less than 150g, while the up-click intensity threshold is greater than 150g. Further in some such embodiments, the up-click intensity threshold is determined (726) from a characteristic intensity of the input during the detected increase in intensity of the input in which the input reached the peak intensity before the decrease in intensity of the input on the input element is detected. Examples of up-click intensity thresholds (sometimes referred to as time-varying intensity thresholds) determined in this manner are described above with reference to fig. 5K and 5M. FIGS. 5K and 5M illustrate examples of determining an up-click intensity threshold using the low-pass filtered intensity of a contact.

In some embodiments, the method 700 includes monitoring the duration of the fast timeout period from when the intensity of the input decreases below the confirmation intensity threshold (728). This is illustrated, for example, in FIGS. 5Q, 5R, and 5T-5V, where monitoring for a fast timeout period begins at time T ₂ In those examples it is the intensity of the input that decreases below the confirmation intensity threshold I _A Then (c) is performed.

In some embodiments, the duration of the fast timeout period is a cumulative amount of time that the intensity of the input is below the confirmation intensity threshold after the start of the fast timeout period (730). Measuring the duration of the fast timeout period as the cumulative amount of time that the input intensity is below the confirmation intensity threshold after the fast timeout period begins is discussed in more detail above with reference to fig. 5T-5V.

In some embodiments, method 700 includes, prior to detecting a decrease in intensity of an input on an input element, determining a peak characteristic intensity (e.g., I) from inputs detected during a detected increase in intensity of the input _Peak Fig. 5O) to determine (732) an acknowledgement strength threshold (e.g., threshold I in fig. 5O-5Y) _A ). Further explanation and examples of how the confirmation intensity threshold is determined are provided above with reference to fig. 5O.

In some implementations, the confirmation intensity threshold is independent of a peak characteristic intensity of the input detected during the detected increase in intensity of the input (734). For example, the confirmation intensity threshold is set to a fixed intensity threshold, such as 100g or 150g, regardless of the peak characteristic intensity of the input detected during the detected increase in intensity of the input.

In some embodiments, the method 700 includes monitoring (736) a duration of a default timeout period from when the intensity of the input increases to when the click intensity threshold is pressed. In some other embodiments, the method 700 includes monitoring (738) a duration of a default timeout period beginning when the intensity of the input decreases to the up-click intensity threshold. Both of these two options for monitoring the duration of the default timeout period are described above with reference to fig. 5S. In FIG. 5S, time T ₁ Is that the intensity of the input has increased to the down-click intensity threshold I _D Time of, and time T _2A Is that the intensity of the input has decreased to the up-click intensity threshold I _U Time of (d).

In some embodiments, the method 700 includes, after delaying the execution of the first operation, in accordance with a determination that the second input event has been identified after the default delay period has elapsed, executing (739) a third operation. For example, referring to FIG. 5S, after the default time period has expired, at time T ₄ If the second input event is recognized, it will be treated as a separate event, such as a separate click, and the third operation will correspond to an operation performed by the electronic device in response to the click. A similar example is shown in FIG. 5R, where the input includes an early confirmation criterion at time T ₃ A click after the criterion is met. In this example, the resulting third operation is scrolling from one screen of icons in the application launch user interface (including the first set of application launch icons) to another screen of icons in the application launch user interface.

In some embodiments, method 700 includes detecting (741) a series of different inputs on the input element separated by a period of time during which no input is detected on the input element, and repeating the identifying and delaying for a plurality of inputs in the series of different inputs. For example, as shown in FIG. 5E and described above with reference to FIG. 5E, the device may detect a touch down of a first input (e.g., touch input 505, FIGS. 5B-5C) on the input element followed by a lift-off of the first input and a touch down of a second input (e.g., touch input 507, FIG. 5E) on the input element. In response, the electronic device repeats the identification of the first input event for each input in the series of different inputs and delays performance of the first operation.

In some embodiments, the method 700 includes, after delaying execution of the first operation, in accordance with a determination that the first input event satisfies (740) the long press criterion before the second input event has been identified, executing a third operation and aborting execution of the first operation and the second operation. For example, the first input event may be a press-and-click, in which case the third operation may be a long press operation, such as invoking a virtual assistant or operating a dictation mode. The recognition of and response to a long press input is discussed in more detail with reference to fig. 5Z-5 II and 8A-8C.

In some embodiments, method 700 includes identifying a first input event (e.g., at time T) in conjunction with ₁ Identifying a click-down, or at time T _2a Identify an up click, fig. 5O) to generate (742) a first tactile output (e.g., first tactile output 502, fig. 5O).

In some embodiments, method 700 includes generating (744) a second tactile output in conjunction with identifying a second input event, as described above with reference to fig. 5W.

In some embodiments, the first operation is or includes (746) ceasing to display the user interface of the application (and optionally returning to a home screen of the display device or an application launch screen), for example as shown in the transition of fig. 5O to 5Q, and the second operation includes a multitasking operation (e.g., switching between applications or displaying a user interface that provides options to switch between multiple different applications, such as concurrently open applications or recently used applications, as shown in fig. 5W).

In some embodiments, the first operation is or includes (748) scrolling from one icon screen of the application launch user interface (e.g., an application launch user interface that includes a first set of application launch icons) to another icon screen of the application launch user interface (e.g., an application launch icon that includes a second set of application launch icons that includes application launch icons that are not in the first set of application launch icons), as shown in the transition from fig. 5Q to 5R, and the second operation includes a multitasking operation (e.g., switching between applications or displaying a user interface that provides options for switching between a plurality of different applications, such as concurrently open applications or recently used applications, as shown in fig. 5W).

In some embodiments, the identification of the first input event is based on detecting a change in the characteristic intensity of the input relative to a first intensity threshold (e.g., an up-click intensity threshold); the identification of the second input event is based on detecting a change in the characteristic intensity of the input relative to a second intensity threshold (e.g., a down-click intensity threshold) that is different from the first intensity threshold. For example, referring to FIG. 5Y, the up-click intensity threshold I used to identify the first event _U Different from (e.g., below) the down-click intensity threshold I used to identify the second event _D 。

In some embodiments, the method 700 includes, after identifying a second input event (e.g., a second up-click event, corresponding to indicator 522 in fig. 5Y), delaying (760) performance of the second operation while monitoring for a subsequent change in intensity of the input for a third input event (e.g., a third down-click, sometimes referred to as a triple-click event, corresponding to indicator 579), wherein the delay is limited by a second default delay period of time. The method 700 further includes, after delaying (762) execution of the second operation, in accordance with a determination that a third event (e.g., corresponding to indicator 579, fig. 5Y) has been identified before the second default delay period has elapsed, executing (764) the third operation (e.g., a triple-click operation) and aborting execution of the second operation (e.g., a double-click operation). Otherwise, after delaying (762) the execution of the second operation, the method 700 includes, in accordance with a determination that the early-confirmation criteria for the second input event have been met before the second default delay time period has elapsed (e.g., as indicated by indicator 577, fig. 5Y) and that the third input event is not recognized, executing (766) the second operation (e.g., a double-click operation) before the second default delay time period has elapsed. Additionally, after delaying (762) the execution of the second operation, the method 700 includes, in accordance with a determination that the second default delay time period has elapsed while the early-confirmation criteria for the second input event is not met (e.g., as indicated by indicator 578, fig. 5Y) and the third input event is not recognized, executing (768) the second operation (e.g., a double-click operation) once the second default delay time period has elapsed. In some embodiments, the second default delay period is the same as the first default delay period. In some other embodiments, the second default delay period is longer or shorter than the first default delay period.

According to some embodiments, fig. 10 illustrates a functional block diagram of an electronic device 1000 configured according to the principles of various described embodiments. The functional blocks of the device are optionally implemented by hardware, software, or a combination of hardware and software that perform the principles of the various described embodiments. Those skilled in the art will understand that the functional blocks described in fig. 10 are optionally combined or separated into sub-blocks in order to implement the principles of the various described embodiments. Thus, the description herein optionally supports any possible combination or separation or further definition of the functional blocks described herein.

As shown in fig. 10, the electronic device 1000 includes a display unit 1002 (e.g., corresponding to the display 112) configured to display a user interface, an intensity-sensitive input unit 1004 (e.g., corresponding to the main button 204) configured to detect an intensity of a user input with the input element, and a processing unit 1010 coupled to the display unit 1002 and the intensity-sensitive input unit 1004. In some embodiments, electronic device 1000 also includes a touch-sensitive surface unit 1006 for receiving touch input on a surface (such as a display surface of display unit 1002), and one or more haptic output units 1008 for generating haptic output, also coupled to processing unit 1010. In some embodiments, the processing unit 1010 includes one or more of the following sub-units: an intensity monitoring unit 1012, a single click determining unit 1014, a double click determining unit 1016, a feedback unit 1018, and a delay unit 1024. In some embodiments, the click determination unit 1014 includes an early confirmation unit 1015. In some embodiments, the feedback unit 1018 includes a response display unit 1020, and a haptic output unit 1022.

The processing unit 1010 is configured to: detecting a change in intensity of the input on the intensity-sensitive input unit (e.g., with intensity monitoring unit 912), which includes detecting an increase in intensity of the input on the intensity-sensitive input unit followed by a decrease in intensity of the input on the intensity-sensitive input unit; identifying (e.g., with the click determination unit 1014) at least a portion of the change in intensity of the input as a first input event associated with a first operation; and delaying (e.g., with delay unit 1024) execution of the first operation after identifying the first input event, while monitoring for a subsequent change in intensity of the input for a second input event, wherein the delaying is limited by a default delay time period.

The processing unit 1010 is further configured to, after delaying execution of the first operation: in accordance with a determination that a second input event has been identified (e.g., with the double-click determination unit 1016) before the default delay period has elapsed, performing a second operation and aborting the performance of the first operation; in accordance with a determination that the early-confirmation criteria for the first input event has not been met (e.g., with the early-confirmation unit 1014) before the default delay time period has elapsed without the second input event being recognized, performing the first operation before the default delay time period has elapsed; and in accordance with a determination that the default delay time period has elapsed without the early validation criteria for the first input event being satisfied (e.g., with early validation unit 1014) and without the second input event being recognized (e.g., with a double-tap determination unit), perform the first operation once the default delay time period has elapsed.

In some embodiments, the processing unit is further configured to perform a third operation after delaying the performance of the first operation in accordance with a determination that the second input event has been recognized (e.g., with the double-click determining unit 1016) after the default delay period has elapsed.

In some embodiments, the input on the intensity-sensitive input unit is or comprises an input on a touch-sensitive surface. In some embodiments, detecting a change in intensity of an input on the intensity-sensitive input element comprises continuously detecting an input on the intensity-sensitive input element.

In some embodiments, the processing unit is further configured to detect a series of different inputs on the intensity-sensitive input element separated by a time period during which no input is detected on the intensity-sensitive input element, and repeat the identifying (e.g., with the click determination unit) and delaying (e.g., with the delay unit 1024) for multiple inputs in the series of different inputs.

In some embodiments, the early-validation criterion for the first input event is or includes a criterion that an intensity of the input remains below a validation intensity threshold for longer than an early-validation time threshold. In some embodiments, the early acknowledgement time threshold is less than half of the default delay period. In some embodiments, the confirmation intensity threshold is below the up-click intensity threshold used to identify the second input event.

In some embodiments, the up-click intensity threshold is determined (e.g., with the click determination unit 1014) from a characteristic intensity of the input during a detected increase in intensity of the input in which the input reaches a peak intensity before a decrease in intensity of the input on the intensity-sensitive input element is detected.

In some embodiments, the processing unit is further configured to monitor (e.g., with the early validation unit 1015) the duration of the fast timeout period from when the intensity of the input decreases below the validation intensity threshold. In some embodiments, the duration of the fast timeout period is a cumulative amount of time after the start of the fast timeout period that the intensity of the input is below the confirmation intensity threshold.

In some embodiments, the processing unit is further configured to determine a confirmation intensity threshold from a peak characteristic intensity of the input detected during the detected increase in intensity of the input (e.g., with the single click determination unit 1014 or the early confirmation unit 1015) prior to detecting the decrease in intensity of the input on the intensity-sensitive input element. In some other embodiments, the confirmation intensity threshold is independent of a peak characteristic intensity of the input detected during the detected increase in intensity of the input.

In some embodiments, the processing unit is further configured to monitor (e.g., with the click determination unit 1014) the duration of the default timeout period from when the intensity of the input increases to the down-click intensity threshold. In some other embodiments, the processing unit is further configured to monitor a duration of the default timeout period from when the intensity of the input decreases to the up-click intensity threshold.

In some embodiments, the processing unit is further configured to generate (e.g., with the haptic output unit 1022) a first haptic output in conjunction with identifying the first input event. In some embodiments, the processing unit is further configured to generate a second tactile output (e.g., with the tactile output unit 1022) in conjunction with identifying the second input event.

In some embodiments, the processing unit is further configured to, after identifying a second input event (e.g., with the double-click determination unit), delay performance of the second operation (e.g., with the delay unit 1024) while monitoring for a subsequent change in intensity of the input for a third input event, wherein the delay is limited by a second default delay time period. The processing unit is further configured to, after delaying execution of the second operation, in accordance with a determination that a third input event has been identified before a second default delay period of time has elapsed, execute the third operation and forgo execution of the second operation. The processing unit is further configured to, in accordance with a determination that the early-confirmation criteria for the second input event have been met before the second default delay time period has elapsed without the third input event being recognized, perform a second operation before the second default delay time period has elapsed. In addition, the processing unit is configured to perform the second operation once the second default delay time period has elapsed in accordance with a determination that the second default delay time period has elapsed without the early-validation criterion for the second input event being satisfied and the third input event being unrecognized.

In some embodiments, the first operation includes ceasing to display the user interface of the application and the second operation includes a multitasking operation. In some other embodiments, the first operation includes scrolling from one icon screen in the application launch user interface to another icon screen in the application launch user interface, and the second operation includes a multitasking operation.

In some embodiments, the processing unit is further configured to, after delaying (e.g., with delay unit 1024) execution of the first operation, in accordance with a determination that the first input event satisfies the long press input criteria before the second input event has been identified, execute a third operation and abort execution of the first operation and the second operation.

In some embodiments, the identification of the first input event is based on detecting a change in a characteristic intensity of the input relative to a first intensity threshold; the identification of the second input event is based on detecting a change in the characteristic intensity of the input relative to a second intensity threshold that is different from the first intensity threshold.

The operations in the information processing methods described above with reference to fig. 7A-7E are optionally implemented by running one or more functional modules in an information processing apparatus, such as a general-purpose processor (e.g., as described above with respect to fig. 1A and 3) or an application-specific chip.

The operations described above with reference to fig. 7A-7E are optionally implemented by components depicted in fig. 1A-1B or fig. 3. For example, the detection operations 702, 706, etc. and the identification and determination operations 708, 714, 716, 718, etc. are optionally implemented by the contact/motion module 130, the feedback operation providing the haptic output is implemented by the tactile feedback module 133, and some other operations are optionally implemented by the event classifier 170, the event recognizer 180, and the event handler 190. Event monitor 171 in event sorter 170 detects a contact on touch-sensitive display 112 and event dispatcher module 174 communicates the event information to application 136-1. A respective event recognizer 180 of application 136-1 compares the event information to respective event definitions 186, and determines whether a first contact at a first location on the touch-sensitive surface (or whether rotation of the device) corresponds to a predefined event or sub-event, such as selection of an object on a user interface, or rotation of the device from one orientation to another. When a respective predefined event or sub-event is detected, the event recognizer 180 activates an event handler 190 associated with the detection of the event or sub-event. Event handler 190 optionally uses or calls data updater 176 or object updater 177 to update application internal state 192. In some embodiments, event handler 190 accesses a respective GUI updater 178 to update the content displayed by the application. Similarly, it will be clear to those skilled in the art how other processes may be implemented based on the components depicted in fig. 1A-1B.

Fig. 8A-8C are flow diagrams of a method 800 of monitoring input on an intensity sensitive input element, detecting a long press at an accelerated rate, and performing a corresponding operation if a long press is detected. Method 800 is performed at an electronic device (e.g., device 300, FIG. 3; or portable multifunction device 100, FIG. 1A) having a display, a touch-sensitive surface, and one or more sensors 165 for detecting intensity of contacts with the touch-sensitive surface. In some embodiments, the electronic device performing method 800 has a main button 204 that includes one of the sensors 165 in addition to the touch-sensitive surface. In some embodiments, the primary button 204 is separate from the display, and optionally includes a set of one or more intensity sensors separate from the intensity sensor used to detect the intensity of the input on the display. In some embodiments, the home button 204 is a virtual home button displayed on the display (e.g., having a set of one or more intensity sensors separate from the intensity sensor used to detect the intensity of the input on the display, or optionally using an intensity sensor integrated into the display to determine the intensity of the input utilizing the virtual home button). In some embodiments, the display is a touch screen display and the touch sensitive surface is on or integrated with the display. In some embodiments, the display is separate from the touch-sensitive surface. Some operations in method 800 are optionally combined, and/or the order of some operations is optionally changed.

As described below, the method 800 provides a way to quickly determine user intent as to whether the touch input is a long press by considering the strength of the user's input. The method 800 reduces latency by recognizing long press inputs faster than would otherwise be possible, while avoiding "false positives" (such as inputs incorrectly detected as long presses), thereby generating a more efficient human-machine interface. For battery-driven electronic devices, considering the strength of the user's touch input enables the user to input gestures, such as long presses, faster and more efficiently, which saves power and increases the time between battery charges.

The device detects (802) an input sequence comprising an increased intensity of an input corresponding to a first input event (e.g., a press-and-click event). In some embodiments, the input on the input element is or includes (804) an input on a touch-sensitive surface. In some embodiments, or in some cases, an increase in intensity of an input is followed by a decrease in intensity of the input, e.g., as shown in fig. 5 BB. The method 800 includes, in response to detecting the sequence of inputs (806), in accordance with a determination that a second input event (e.g., an up-click event) including a decrease in intensity of the input after the first input event is detected within a first time period after the first input event is detected, performing (808) a first operation (e.g., a one-click operation). For example, fig. 5X shows an input with an increase in intensity 532 followed by a decrease in intensity 534. If the second input event corresponding to the decrease occurs within a first time period after the first input event is detected, the long press has not been detected and a first operation, such as a single click operation, is performed.

The method 800 further includes, in accordance with a determination that a second input event (e.g., an event corresponding to a reversal of the first input event, such as an up-click event) is not detected for a second time period that is longer than the first time period and that the input has a characteristic intensity that is above a respective intensity threshold between when the first input event is detected and when the second time period elapses (e.g., as shown in FIG. 5BB, the characteristic intensity of the input is above the respective intensity threshold (e.g., I;) between when the first input event is detected and when the second input event elapses _D ) Performing (809) a second operation (e.g., a long press operation) once a second time period has elapsed (e.g., in response to the second time period having elapsed (e.g., earlier/faster than a normal long press operation), wherein the second time period is determined based at least in part on the intensity of the input after the first input event is detected。

The method 800 further includes, in accordance with a determination that the second input event was not detected for a third time period (e.g., the default time period described above with reference to FIG. 5 II) that is longer than the second time period and that the input did not have a characteristic intensity above the respective intensity threshold between when the first input event was detected and when the second time period elapsed, once the third time period has elapsed (e.g., in response to the third time period elapsing, corresponding to time T in FIG. 5II ₅ (ii) a See discussion of fig. 5A, 5B, and 5II above) a second operation (e.g., a long press operation) is performed 810.

Optionally, method 800 includes, in accordance with a determination that the second input event was not detected within the second time period and that the input did not have a characteristic intensity above the respective intensity threshold between when the first input event was detected and when the second time period elapsed (e.g., as shown in fig. 5 II), forgoing (512) performing the second operation (e.g., the long press operation) once the second time period has elapsed, at least until a third time period (e.g., invoking the default time period in the discussion of fig. 5 II) has elapsed. Therefore, in this case, since the intensity of the input is not higher than the corresponding intensity threshold, the long press operation is not accelerated.

In some embodiments, method 800 includes identifying (814) a first input event according to an increase in intensity of the input meeting a first intensity threshold and identifying a second input event according to a decrease in intensity of the input meeting a second intensity threshold different from the first intensity threshold. For example, FIG. 5X shows a user having satisfied the down click intensity threshold I _D Is increased 532, followed by meeting and pressing the click intensity threshold I _D Different up-click intensity thresholds I _U Is reduced 534. In some embodiments, the respective intensity threshold is greater (816) than the first intensity threshold (see, e.g., fig. 5Z and 5AA, the respective intensity threshold being an intensity threshold such as I _D+ Greater than the click intensity threshold I _D ) And a second intensity threshold (e.g., up-click intensity threshold I) _U ) Less than a first intensity threshold (e.g., a click intensity threshold I _D )。

In some embodiments, the method 800 includes monitoring (818) a duration of the timeout period from when an increase in intensity of the input satisfies a first intensity threshold, and comparing the duration of the timeout period to at least one of the first time period, the second time period, and the third time period. See the examples described above with reference to fig. 5BB, 5DD, and 5 FF.

In some embodiments, the method 800 includes stopping (820) monitoring of the duration of the timeout period when the decrease in intensity of the input meets a second intensity threshold. For example, referring to FIG. 5X, if the intensity of the input falls below the up-click intensity threshold I _U The monitoring of the duration of the timeout period is stopped.

In some implementations, the method 800 includes accelerating a rate of timeout period accumulation when the intensity of the input exceeds a first predefined intensity threshold (e.g., the respective intensity threshold), wherein the accelerated rate is higher than a default rate. As described above with reference to FIGS. 5BB, 5DD, and 5FF, when the intensity of the input exceeds a first predefined intensity threshold (in those examples the click intensity threshold I is pressed _D ) The rate at which the timeout period accumulates is an accelerated rate. In some embodiments, the method 800 further includes decelerating (824) the rate at which the timeout period accumulates as the intensity of the input decreases.

In some embodiments, the third time period is (826) a longest duration of a timeout period (e.g., a default timeout period, such as 500 ms) before performing the second operation. In some embodiments, monitoring the duration of the timeout period continues until it is determined that a second input event is detected or the duration of the timeout period equals a third time period (whichever occurs first), the second time period comprising a shortest duration of the timeout period before performing the second operation.

In some embodiments, the second time period is constrained (828) to at least a minimum duration (e.g., in some embodiments, the time for detecting a long press input is not reduced below 300ms regardless of how high the intensity of the input is reached). In some embodiments, the shortest duration (e.g., 300 ms) is longer than half of the longest duration (500 ms).

In some embodiments, monitoring the duration of the timeout period includes decaying (830) a time value from an initial time value at a rate that varies according to the intensity of the input. See, for example, fig. 5BB, 5DD, 5FF, and 5HH, and the discussion of those figures above.

In some embodiments, the first operation is or includes closing (832) the application and the second operation includes displaying a virtual assistant user interface.

In some embodiments, the first operation is or includes (834) scrolling from one icon screen in the application launch user interface (e.g., an application launch user interface that includes a first set of application launch icons) to another icon screen in the application launch user interface (e.g., that includes a second set of application launch icons that includes application launch icons not in the first set of application launch icons), as shown in the transition from fig. 5Q to 5R, and the second operation is or includes displaying a virtual assistant user interface (e.g., as shown in fig. 5BB, 5DD, 5FF, and 5 HH).

According to some embodiments, fig. 11 illustrates a functional block diagram of an electronic device 1100 configured in accordance with the principles of various described embodiments. The functional blocks of the device are optionally implemented by hardware, software, or a combination of hardware and software that perform the principles of the various described embodiments. Those skilled in the art will understand that the functional blocks described in fig. 11 are optionally combined or separated into sub-blocks in order to implement the principles of the various described embodiments. Thus, the description herein optionally supports any possible combination or separation or further definition of the functional blocks described herein.

As shown in fig. 11, the electronic device 1100 includes a display unit 1002 (e.g., corresponding to the display 112) configured to display a user interface, an intensity-sensitive input unit 1104 (e.g., corresponding to the main button 204) configured to detect an intensity of a user input with the input element, and a processing unit 1110 coupled to the display unit 1102 and the intensity-sensitive input unit 1104. In some embodiments, the electronic device 1000 also includes a touch-sensitive surface unit 1106 for receiving touch input on a surface (such as a display surface of the display unit 1102), and one or more haptic output units 1108 for generating haptic output, also coupled to the processing unit 1010. In some embodiments, processing unit 1110 includes one or more of the following sub-units: an intensity monitoring unit 1112, a long press determination unit 1114, a click determination unit 1116, and a feedback unit 1118. In some embodiments, the feedback unit 1118 includes a response display unit 1120, and a haptic output unit 1122.

The processing unit 1110 is configured to: detecting (e.g., with intensity monitoring unit 912) a sequence of inputs, including detecting an increase in intensity of an input corresponding to a first input event; and, in response to detecting the input sequence: in accordance with a determination (e.g., with the click determination unit 1116) that a second input event (including a decrease in intensity of the input after the first input event) is detected within a first time period after the first input event is detected, a first operation is performed; in accordance with a determination (e.g., with the click determination unit 1116) that a second input event has not been detected for a second period of time that is longer than the first period of time and a determination (e.g., with the long press determination unit 1114) that the input has a characteristic intensity that is above a respective intensity threshold between when the first input event was detected and when the second period of time has elapsed, performing a second operation once the second period of time has elapsed, wherein the second period of time is determined based at least in part on the intensity of the input after the first input event was detected; and in accordance with a determination (e.g., with the click determination unit 1116) that the second input event was not detected for a third period of time that is longer than the second period of time and a determination (e.g., with the long press determination unit 1114) that the input does not have a characteristic intensity that is above the respective intensity threshold between when the first input event was detected and when the second period of time has elapsed, perform a second operation once the third period of time has elapsed.

In some embodiments, the processing unit 1110 is further configured to, in accordance with a determination (e.g., with the click determination unit 1116) that a second input event has not been detected within a second time period and a determination (e.g., with the click determination unit 1116) that the input does not have a characteristic intensity above the respective intensity threshold between when the first input event was detected and when the second time period has elapsed, forgo performing the second operation once the second time period has elapsed at least until a third time period has elapsed.

In some embodiments, the input on the intensity-sensitive input unit comprises an input on a touch-sensitive surface.

In some embodiments, the processing unit 1110 is further configured to identify a first input event based on an increase in intensity of the input meeting a first intensity threshold and identify a second input event based on a decrease in intensity of the input meeting a second intensity threshold different from the first intensity threshold (e.g., with the click determination unit 1116).

In some embodiments, the respective intensity threshold is greater than the first intensity threshold, and the second intensity threshold is less than the first intensity threshold. See for the click intensity threshold I _D (corresponding to a first intensity threshold) and an up-click intensity threshold I _U (corresponding to the first intensity threshold) discussion of fig. 5X.

In some embodiments, the processing unit 1110 is further configured to monitor (e.g., with the intensity monitoring unit 1112 and/or the long press determination unit 1114) a duration of the timeout period from when the increase in intensity of the input satisfies the first intensity threshold, and compare the duration of the timeout period to at least one of the first time period, the second time period, and the third time period. In some embodiments, the processing unit 1110 is further configured to stop monitoring the duration of the timeout period when the decrease in intensity of the input meets a second intensity threshold. In some embodiments, the processing unit 1110 is further configured to accelerate a rate at which the timeout period accumulates when the intensity of the input exceeds a first predefined intensity threshold, wherein the accelerated rate is higher than the default rate. In some embodiments, the processing unit 1110 is further configured to slow down the rate at which the timeout period accumulates when the intensity of the input decreases.

In some embodiments, the third time period comprises a longest duration of a timeout period before performing the second operation. In some embodiments, the second time period is constrained to at least a minimum duration.

In some embodiments, monitoring the duration of the timeout period includes decaying a time value from an initial time value at a rate that varies according to the intensity of the input.

In some embodiments, the first operation is or includes closing the application and the second operation includes displaying a virtual assistant user interface. In some other embodiments, the first operation is or includes scrolling from one icon screen in an application launch user interface (e.g., an application launch user interface including a first set of application launch icons, an example of which is shown in fig. 5Q) to another icon screen in the application launch user interface (e.g., an application launch user interface including a second set of application launch icons, the second set of application launch icons including application launch icons that are not in the first set of application launch icons, an example of which is shown in fig. 5R), and the second operation includes displaying a virtual assistant user interface (e.g., an example of which is shown in fig. 5BB, 5DD, and 5 FF).

The operations in the information processing methods described above with reference to fig. 8A-8C are optionally implemented by running one or more functional modules in an information processing apparatus, such as a general-purpose processor (e.g., as described above with respect to fig. 1A and 3) or an application-specific chip.

The operations described above with reference to fig. 8A-8C are optionally implemented by components depicted in fig. 1A-1B or fig. 3. For example, the detection operations 802, 808, etc. and the identification and determination operations 808, 810, 812, etc. are optionally implemented by the contact/motion module 130, the feedback operation providing the haptic output is implemented by the tactile feedback module 133, and some other operations are optionally implemented by the event classifier 170, the event recognizer 180, and the event handler 190. Event monitor 171 in event sorter 170 detects a contact on touch-sensitive display 112 and event dispatcher module 174 communicates the event information to application 136-1. A respective event recognizer 180 of application 136-1 compares the event information to respective event definitions 186 and determines whether a first contact at a first location on the touch-sensitive surface (or whether rotation of the device) corresponds to a predefined event or sub-event, such as selection of an object on a user interface or rotation of the device from one orientation to another. When a respective predefined event or sub-event is detected, event recognizer 180 activates event handler 190 associated with the detection of the event or sub-event. Event handler 190 optionally uses or calls data updater 176 or object updater 177 to update the application internal state 192. In some embodiments, event handlers 190 access respective GUI updaters 178 to update the content displayed by the application. Similarly, it will be clear to those skilled in the art how other processes may be implemented based on the components depicted in fig. 1A-1B.

The foregoing description, for purpose of explanation, has been described with reference to specific embodiments. However, the exemplary discussion above is not intended to be exhaustive or to limit the invention to the precise form disclosed. Many modifications and variations are possible in light of the above teaching. The embodiments were chosen and described in order to best explain the principles of the invention and its practical applications, to thereby enable others skilled in the art to best utilize the invention and various embodiments described with various modifications as are suited to the particular use contemplated.

Claims

1. A method, comprising:

at an electronic device having a display and an intensity-sensitive input element, wherein the intensity-sensitive input element is to detect an intensity of a user input with the intensity-sensitive input element:

detecting an input sequence comprising an increase in intensity of an input corresponding to a first input event; and is

In response to detecting the input sequence:

in accordance with a determination that a second input event comprising a decrease in intensity of the input after the first input event is detected within a first time period after the first input event is detected, performing a first operation;

In accordance with a determination that the second input event was not detected for a second period of time longer than the first period of time and that the input has a characteristic intensity above a respective intensity threshold between when the first input event was detected and when the second period of time elapsed, performing a second operation once the second period of time has elapsed, wherein the second period of time is determined based at least in part on the intensity of the input after the first input event was detected; and

in accordance with a determination that the second input event was not detected for a third period of time longer than the second period of time and that the input did not have a characteristic intensity above the respective intensity threshold between when the first input event was detected and when the second period of time elapsed, performing the second operation once the third period of time has elapsed.

2. The method of claim 1, further comprising:

in accordance with a determination that the second input event was not detected within the second time period and the input did not have a characteristic intensity above the respective intensity threshold between when the first input event was detected and when the second time period elapsed, forgoing performing the second operation once the second time period has elapsed at least until the third time period has elapsed.

3. The method of claim 1, wherein the input on the intensity-sensitive input element comprises an input on a touch-sensitive surface.

4. The method of claim 1, wherein the second time period is constrained to at least a minimum duration.

5. The method of claim 1, wherein the first operation comprises closing an application and the second operation comprises displaying a virtual assistant user interface.

6. The method of claim 1, wherein the first operation comprises scrolling from one icon screen in an application launch user interface to another icon screen in the application launch user interface, and the second operation comprises displaying a virtual assistant user interface.

7. The method of claim 1, comprising identifying the first input event according to an increase in intensity of the input meeting a first intensity threshold, and identifying the second input event according to a decrease in intensity of the input meeting a second intensity threshold different from the first intensity threshold.

8. The method of claim 7, wherein the respective intensity threshold is greater than the first intensity threshold, and the second intensity threshold is less than the first intensity threshold.

9. The method of any of claims 7-8, comprising beginning to monitor a duration of a timeout period when the increase in intensity of the input satisfies the first intensity threshold, and comparing the duration of the timeout period to at least one of the first time period, the second time period, and the third time period.

10. The method of claim 9, comprising stopping the monitoring of the duration of the timeout period when the decrease in intensity of the input satisfies the second intensity threshold.

11. The method of claim 9, comprising accelerating a rate at which the timeout period accumulates when the intensity of the input exceeds a first predefined intensity threshold, wherein the accelerated rate is higher than a default rate.

12. The method of claim 11, comprising slowing down a rate at which the timeout period accumulates when the intensity of the input decreases.

13. The method of claim 9, wherein the third time period comprises a longest duration of the timeout period before performing the second operation.

14. The method of claim 9, wherein monitoring the duration of the timeout period comprises decaying a time value from an initial time value at a rate that varies according to an intensity of the input.

15. An electronic device, comprising:

a display;

an intensity sensitive input element for detecting an intensity of a user input with the intensity sensitive input element;

one or more processors;

a memory; and

one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs comprising instructions for performing the method of any of claims 1-14.

16. A computer readable storage medium storing one or more programs, the one or more programs comprising instructions, which when executed by an electronic device with a display and an intensity-sensitive input element for detecting intensity of user input with the intensity-sensitive input element, cause the electronic device to perform the method of any of claims 1-14.