CN108829325A

CN108829325A - For dynamically adjusting the equipment, method and graphic user interface of the presentation of audio output

Info

Publication number: CN108829325A
Application number: CN201810369048.8A
Authority: CN
Inventors: N·德夫雷斯; D·C·格拉哈姆; F·A·安祖雷斯; M·阿朗索鲁伊斯
Original assignee: Apple Computer Inc
Current assignee: Apple Inc
Priority date: 2016-06-12
Filing date: 2017-05-22
Publication date: 2018-11-16
Anticipated expiration: 2037-05-22
Also published as: CN108829325B; CN107491283B; CN107491283A

Abstract

The present invention is entitled for dynamically adjusting equipment, method and the graphic user interface of the presentation of audio output.A kind of electronic equipment shows the user interface of the first interaction area and the second interaction area including application program over the display.When showing the user interface, which is passing through input caused by the contact on touch sensitive surface with detection at the corresponding position of user interface element in the first interaction area over the display.In response to detecting through input caused by contacting, when the input meet based on intensity activation standard (such as, the contact has the characteristic strength higher than intensity threshold) when, in addition to user interface element, the equipment covers the first interaction area of application program, without covering the second interaction area of application program.When the input meets selection criteria (for example, the contact has the characteristic strength lower than intensity threshold), which executes selection operation corresponding with user interface element, without covering the first interaction area of application program.

Description

For dynamically adjusting the equipment, method and graphical user of the presentation of audio output Interface

Related application

It is on May 22nd, 2017 that the application, which is application No. is the 201710364610.3, applying date, entitled " is used for Dynamically adjust the equipment, method and graphic user interface of the presentation of audio output " application for a patent for invention divisional application.

Technical field

The present invention relates generally to the electronic equipments of offer audio output, and relate more specifically to dynamically adjust audio Equipment, method and the graphic user interface of the presentation of output.

Background technique

Some electronic equipments are by audiovisual interface in the way of the feedback for providing the interaction about user and equipment.

Summary of the invention

However, some electronic equipments provide audiovisual feedback in a manner of limited, inefficient and gloomy.For example, Certain methods interruption and stopping provide the audio (for example, lecture) that user is currently listening to and are suddenly switching to some other sounds Frequently (for example, audio associated with short video messaging).These transformations suddenly have dispersed the attention of user (so as to cause user Lose focus and the certain parts for the audio that they are listening to must be replayed), it forces user to execute additional input and comes It is restored back to the audio that they are listening to, user is forced to disable certain effects based on audio, and generates additional sense of frustration. By requiring user to execute additional input and/or user being forced to replay certain parts, conventional electronic device wastes energy.

Some electronic equipments are by audiovisual interface in the way of the feedback for providing the interaction about user and equipment.So And some electronic equipments provide audiovisual feedback in a manner of limited, inefficient and gloomy.For example, certain methods are rung Predetermined audible feedback should be provided (for example, in response to user small in the interaction of user and graphical user-interface element Number is keyed in provide audible tone for telephony application on keyboard).This predetermined audible feedback is not based on use Family is interactive and changes, so that user be forced to hear identical predetermined and constant audible feedback repeatedly.Therefore, many User disables certain effects based on audio and/or deletion has become too annoying certain application programs.

Therefore, electronic equipment needs to provide audiovisual feedback with more effective way and interface.Such method and interface It optionally supplements or is substituted for provide the conventional method of audiovisual feedback.Such method and interface reduce input from the user Quantity, range and/or property, and generate more effective man-machine interface (for example, by the way that dynamically to obscure the audio together defeated Out, embodiment disclosed herein allows the audio output of user's effectively preview new to listen to present video without stopping suddenly Output).In addition, such method, which reduces, handles electric power consumed by processing touch input, saves power and (filled to increase battery Time between electricity), unnecessary/unrelated/duplicate input is reduced, and potential reduction memory uses.

According to some embodiments, method is executed at the electronic equipment communicated with display and audio system.It should Method includes：The first acoustic information is provided so that the first audio output is presented to audio system, which includes：Volume With the audio attribute in addition to volume.This method further includes receiving and using when the first audio output is being presented in audio system In the corresponding input of request that the second audio output is presented.This method further includes in response to receiving and the second audio for rendering The corresponding input of the request of output：Information is provided to audio system, to dynamically adjust the first audio according to the magnitude of input The presentation of output, wherein the presentation for dynamically adjusting the first audio output includes dynamically adjusting as the magnitude of input changes Non- volume audio attribute；And second sound information is provided to audio system, the second sound to be presented simultaneously with the first audio output Frequency exports.

According to some embodiments, method is executed at the electronic equipment communicated with display and audio system.It should Method includes：When display includes that one group of one or more shows the user interface that can be indicated over the display：In first time point Detection is directed toward the group one or more and shows that first in capable of indicating shows the first input that can be indicated.This method further includes in response to inspection It measures direction first and shows the first input that can be indicated, start to provide the first acoustic information to audio system, be shown with presenting with first Corresponding first audio output can be indicated, wherein the first audio output has the first audio configuration file.This method further include The second time point detection after first time point is directed toward the group one or more and shows that second in capable of indicating shows can indicate the Two inputs.This method further includes in response to detecting that being directed toward second shows the can indicate second input：Meet audio more according to determination Change standard：So that audio system presentation is shown with first can indicate corresponding the first modified audio output, without being to continue with benefit The first audio output is presented with the first audio configuration file, wherein the first modified audio output, which has, is different from the first sound The modified audio configuration file of frequency configuration file；And second sound information is provided to audio system, to present and second Corresponding second audio output can be indicated by showing, wherein the second audio output has the second audio configuration file.This method further includes Audio change standard is unsatisfactory for according to determining：So that audio system continues to present and shows with first and can indicate corresponding and with first First audio output of audio configuration file；And third acoustic information is provided to audio system, show energy table with second to present Show corresponding third audio output, wherein the third audio output has third audio configuration file.

According to some embodiments, strong with the contact of touch sensitive surface with display, touch sensitive surface and for detecting Method is executed on the electronic equipment of the one or more sensors of degree.This method includes：Display includes media item over the display Expression user interface.This method further includes that the expression with media item when showing user interface on touch sensitive surface is corresponding Position at detection by contact caused by input.This method further includes in response to detecting through input caused by contacting：Root Meet media cue standard according to the determining input, wherein the media cue standard includes being higher than the first intensity threshold when the contact has The standard met when the characteristic strength of value：Start the corresponding portion of broadcasting media item；And when media item is playing, with The characteristic strength of contact changes and dynamically changes one group of one or more audio attribute of media item.This method further includes basis It determines that the input is unsatisfactory for media cue standard, abandon the corresponding portion for starting to play media item and abandons the spy with contact Sign intensity changes and dynamically changes the group one or more audio attribute of media item.

According to some embodiments, strong with the contact of touch sensitive surface with display, touch sensitive surface and for detecting Method is executed on the electronic equipment of the one or more sensors of degree.This method includes：Display includes applying journey over the display The user interface of second interaction area of the first interaction area and application program of sequence.This method further includes in display user interface When over the display at position corresponding with the first user interface element in the first interaction area detection pass through touch-sensitive table First input caused by contact on face.This method further includes in response to detecting through the first input caused by contacting：According to It determines that the first input meets the activation standard based on intensity, should wherein be had based on the activation standard requirements contact of intensity and be higher than the The characteristic strength of one intensity threshold is to meet the activation standard based on intensity, other than the first user interface element, hides The first interaction area for covering application program, without covering the second interaction area of application program.This method further includes according to really Fixed first input meets first choice standard, and wherein the first choice standard does not require contact to have higher than the first intensity threshold Characteristic strength executes first choice operation corresponding with the first user interface element, without masking to meet selection criteria First interaction area of application program.

According to some embodiments, the electronic equipment communicated with display and audio system includes at one or more Manage device, memory and one or more programs；The one or more program be stored in memory and be configured as by this one A or multiple processors execute, and the one or more program includes for executing or making to execute any side described herein The instruction of the operation of method.According to some embodiments, computer readable storage medium is stored with instruction wherein, the instruction when by The electronic equipment communicated with display and audio system makes the equipment execute or to execute described herein when executing The operation of any method.According to some embodiments, including memory and for execute be stored in memory one or Graphic user interface on the electronic equipment of the one or more processors of multiple programs is included in any method described herein One or more elements in the element of middle display, these elements are updated in response to input, any side as described herein Described in method.According to some embodiments, the electronic equipment communicated with display and audio system includes for executing Or make the device for executing the operation of any method described herein.According to some embodiments, for display and sound Information processing equipment used in the electronic equipment that display system is communicated includes described herein for executing or making to execute The device of the operation of any method.

Therefore, the electronic equipment communicated with display and audio system be provided with faster, more effective way and Interface is to be used to provide audible feedback and fuzzy audio, to improve the validity of such equipment, efficiency and user satisfaction. Such method and interface can supplement or be substituted for provide the conventional method of audible feedback and fuzzy audio.

According to some embodiments, electronic equipment includes display, touch sensitive surface, optionally for detection and touch sensitive surface Contact the one or more sensors of intensity, one or more processors, memory and one or more programs；This Or multiple programs are stored in memory and are configured as being executed by the one or more processors, and the one or more Program includes the instruction for executing or making to execute the operation of any method described herein.According to some embodiments, meter Calculation machine readable storage medium storing program for executing is stored with instruction wherein, and the instruction is when by with display, touch sensitive surface and optionally for inspection The equipment is made to execute or make when surveying the electronic equipment execution of the one or more sensors of the intensity of the contact with touch sensitive surface The operation of any method described herein must be executed.According to some embodiments, there is display, touch sensitive surface, optionally use Memory is stored in the one or more sensors for the intensity for detecting the contact with touch sensitive surface, memory and for executing In one or more programs one or more processors electronic equipment on graphic user interface be included in and be described herein Any method in one or more elements in shown element, these elements are updated in response to input, such as herein Described in any method of description.According to some embodiments, electronic equipment includes：Display, touch sensitive surface and optionally For detecting and the one or more sensors of the intensity of the contact of touch sensitive surface；And it is retouched herein for executing or making to execute The device of the operation for any method stated.According to some embodiments, for display and touch sensitive surface and optionally Ground is for detecting and information processing used in the electronic equipment of the one or more sensors of the intensity of the contact of touch sensitive surface Equipment includes the device for executing or making to execute the operation of any method described herein.

Therefore, one of the intensity with display, touch sensitive surface and the contact optionally for detection with touch sensitive surface Or the electronic equipment of multiple sensors be provided with faster, more effective way and interface is for providing audible feedback and mould Audio is pasted, to improve the validity of such equipment, efficiency and user satisfaction.Such method and interface can supplement or substitute use In offer audible feedback and the conventional method of fuzzy audio.

Detailed description of the invention

Various described embodiments in order to better understand, should refer to following specific implementation in conjunction with the following drawings Mode, drawing reference numeral similar in the accompanying drawings refer to corresponding part in all schemas.

Figure 1A is the block diagram for showing the portable multifunction device with touch-sensitive display according to some embodiments.

Figure 1B is the block diagram for showing the example components for event handling according to some embodiments.

Fig. 2 shows the portable multifunction devices with touch screen according to some embodiments.

Fig. 3 is the block diagram according to the exemplary multifunctional equipment with display and touch sensitive surface of some embodiments.

Fig. 4 A shows the exemplary of the application menu on the portable multifunction device according to some embodiments User interface.

Fig. 4 B is shown according to the multifunctional equipments with the touch sensitive surface separated with display of some embodiments Exemplary user interface.

Fig. 4 C to 4E shows the example of the resistance to vibration threshold value according to some embodiments.

Fig. 5 is shown to be set according to the exemplary electronic of some embodiments communicated with display and touch sensitive surface It is standby, wherein at least one subset of these electronic equipments, display and/or touch sensitive surface are integrated into electronic equipment.

Fig. 6 A to 6Y is shown according to some embodiments for dynamically adjusting the presentation and optionally of audio output Ground dynamically adjusts the exemplary user interface that the vision of user interface is presented.

Fig. 7 A to 7G is shown according to some embodiments for providing audio output based on audio configuration file Exemplary user interface.

Fig. 8 A to 8B is the process for showing the method for the presentation for dynamically adjusting audio output according to some embodiments Figure.

Fig. 8 C is the flow chart for showing the method for the presentation for dynamically adjusting audio output according to some embodiments.

Fig. 8 D to 8F is the process for showing the method for the presentation for dynamically adjusting audio output according to some embodiments Figure.

Fig. 8 G to 8H is to show the method presented according to the vision for dynamically adjusting user interface of some embodiments Flow chart.

Fig. 9 A to 9C is to show to provide the method for audio output based on audio configuration file according to some embodiments Flow chart.

Specific embodiment

Disclosed embodiment provides a kind of method and apparatus for dynamically adjusting the presentation of audio output.More Body, in some embodiments, equipment provided herein is by the magnitude that is inputted according to user (for example, on touch sensitive surface The intensity of contact and/or gently sweep the length of gesture) adjustment audio attribute (for example, volume or non-Volume attribute) comes in preview audio Hold.In some embodiments, when equipment is just providing the first audio output (for example, play First song), equipment according to The magnitude of user's input and make the second audio output (for example, second song) " obscure fade in ".In some embodiments, make Second audio is fuzzy fade in including the first audio content is dynamically adjusted based on the magnitude of input non-Volume attribute (for example, Stereo balance, applied to the first audio content low-pass filter cutoff frequency).

Consider the example that song is played via the music application on user equipment.Meanwhile user is openable just (imagine that video not yet plays and user must press the image of video to it in the message application of display video messaging It plays out).In some embodiments, user can pass through the pressing that carries out to video messaging and keep gesture to play video " prompt " of message.Prompt makes song is fuzzy to fade out (for example, by with pressing and keep the intensity of gesture to increase and reduces sound Measure and reduce the cutoff frequency of low-pass filter) and make video messaging is fuzzy to fade in (by with pressing and keep gesture Intensity increases and improves volume).In some embodiments, it is higher than specific contact strength, equipment plays video with full volume and disappears Breath, and song is played with amount of bass in the high fdrequency component for filtering out song, so that song is only in the background in soft mode Sound like " skirr ".

According to some embodiments, the method and apparatus for changing audible feedback is also provided.For example, when activation is accordingly shown When can indicate so that equipment generates audible feedback, in extremely rapid succession activating this accordingly to show can indicate to make equipment change audio twice Feedback.

In the following, Figure 1A to 1B, 2 and 3 provide the description to example devices.

Fig. 4 A to 4B and 6A to 6O shows the magnitude for inputting according to user to make audio output " fuzzy to fade in " (example Such as, when background audio play when) exemplary user interface.Fig. 8 A to 8B and Fig. 8 C is shown according to user's input Magnitude makes the two methods of audio output " fuzzy fade in " (for example, when background audio has been playing).Fig. 6 A is into 6O User interface for showing process of Fig. 8 A into 8C.

Fig. 4 A to 4B and 6P to 6Y shows the magnitude for inputting according to user to dynamically adjust the category of audio output Property (for example, at audio output prompt) exemplary user interface.Fig. 8 D to 8F show according to user input magnitude come The method for dynamically adjusting the attribute (for example, prompting at audio output) of audio output.User interface of Fig. 6 P into 6Y is used In showing process of Fig. 8 D into 8F.

In addition, Fig. 4 A to 4B and 6P to 6Y is shown for providing the exemplary use of visual feedback (for example, dimness of vision) Family interface.Fig. 9 A to 9C shows the stream of the method for the offer visual feedback (for example, dimness of vision) according to some embodiments Cheng Tu.User interface of Fig. 6 P into 6Y is for showing process of Fig. 9 A into 9C.

Example devices

Now with detailed reference to embodiment, its example is shown in the drawings.Many tools are shown in following detailed description Body details, in order to provide the thorough understanding to various described embodiments.But it will to those skilled in the art It is readily apparent that various described embodiments can be carried out without these specific details.In other feelings Under condition, it is not described in detail well known method, program, component, circuit and network, in order to avoid unnecessarily obscure each of the embodiment A aspect.

Although will be further understood that in some cases, term " first ", " second " etc. are various for describing herein Element, but these elements should not be limited by these terms.These terms are only intended to an element and another element area It separates.For example, the first contact can be named as the second contact, and similarly, the second contact can be named as the first contact, and The range of various described embodiments is not departed from.First contact and the second contact are contact, but they are not same connect Touching, unless the context clearly.

Term used in the description to the various embodiments is intended merely to description particular implementation side herein The purpose of case, and be not intended to be limiting.As to the various embodiments description and the appended claims in institute As use, " one (" a ", " an ") and "the" are intended to also include plural form singular, unless context is in addition clear Ground instruction.It will be further understood that term "and/or" used herein refers to and covers in the project listed in association One or more projects any and all possible combinations.It will be further understood that the terms "include", "comprise", " containing " And/or " having " be when used in this manual it is specified exist stated feature, integer, step, operation, element and/ Or component, but it is not excluded that in the presence of or add other one or more features, integer, step, operation, component, assembly unit and/or Its group.

As used herein, based on context, term " if " be optionally interpreted to mean " and when ... when " (" when " or " upon ") or " in response to determination " or " in response to detecting ".Similarly, based on context, phrase is " if really It is fixed ... " or " if detecting [condition or event stated] " be optionally interpreted to refer to " when in determination ... " or " in response to determination ... " or " when detecting [condition or event stated] " or " in response to detecting [the item stated Part or event] ".

This document describes electronic equipments, the reality of the user interface of such equipment and the associated process using such equipment Apply scheme.In some embodiments, the equipment be also comprising other function such as PDA and/or music player functionality just Take formula communication equipment, such as mobile phone.The exemplary implementation scheme of portable multifunction device includes but is not limited to come from Apple Inc.'s (Cupertino, California)Equipment, iPodEquipment andEquipment.Appoint Selection of land uses other portable electronic devices, such as knee with touch sensitive surface (for example, touch-screen display and/or touch tablet) Laptop or tablet computer.It is to be further understood that in some embodiments, which is not portable communication device, But the desktop computer with touch sensitive surface (for example, touch-screen display and/or touch tablet).

In the following discussion, a kind of electronic equipment including display and touch sensitive surface is described.However, should manage Solution, electronic equipment optionally include other one or more physical user-interface devices, such as physical keyboard, mouse and/or behaviour Vertical pole.

The equipment usually supports a variety of application programs, one or more application programs in such as following application program：Pen Remember application program, drawing application program, application program, word-processing application, website creation application program, disk editor is presented Application program, spreadsheet applications, game application, telephony application, videoconference application, Email Application program, instant message application program, body-building support application program, photo management application program, digital camera application program, Digital camera applications program, web page browsing application program, digital music player application and/or video frequency player Application program.

The various application programs executed in equipment optionally use at least one common physical user interface facilities, such as Touch sensitive surface.One or more functions of touch sensitive surface and the corresponding informance being displayed in equipment are optionally answered with different It is adjusted and/or is changed with program, and/or adjusted and/or change in corresponding application program.In this way, the shared object of equipment Reason framework (such as touch sensitive surface) supports various applications optionally with intuitive for a user and clear user interface Program.

Focusing on the embodiment of the portable device with touch-sensitive display.Figure 1A is shown according to some embodiment party The block diagram of the portable multifunction device 100 with touch-sensitive display system 112 of case.Touch-sensitive display system 112 is sometimes Convenience is called " touch screen ", and is called touch-sensitive display for short sometimes.Equipment 100 include memory 102 (its optionally Including one or more computer readable storage mediums), Memory Controller 122, one or more processing unit (CPU) 120, Peripheral device interface 118, RF circuit 108, voicefrequency circuit 110, loudspeaker 111, microphone 113, input/output (I/O) subsystem System 106, other inputs or control equipment 116 and outside port 124.Equipment 100 optionally includes one or more optical sensings Device 164.Equipment 100 optionally includes one or more intensity sensors 165 of the intensity of the contact on detection device 100 (for example, touch sensitive surface, touch-sensitive display system 112 of such as equipment 100).Equipment 100 is optionally included in equipment 100 Upper generation tactile output is (for example, in the touch-sensitive display system 112 of touch sensitive surface such as equipment 100 or the touch tablet of equipment 300 On 355 generate tactile output) one or more tactile output generators 167.These components are optionally via one or more Communication bus or signal wire 103 are communicated.

As used in the specification and claims, term " tactile output ", which refers to, to utilize user's by user The equipment that sense of touch detects is opposite relative to physical displacement, the component (for example, touch sensitive surface) of equipment of the previous position of equipment In the displacement relative to the mass center of equipment of physical displacement or component of another component (for example, shell) of equipment.For example, The surface (for example, other parts of finger, palm or user hand) sensitive to touch of the component and user of equipment or equipment In the case where contact, by physical displacement generate tactile output sense of touch will be read as by user, the sense of touch correspond to equipment or The change perceived of the physical features of the component of equipment.For example, the movement of touch sensitive surface (for example, touch-sensitive display or Trackpad) is appointed Selection of land is read as " pressing click " or " unclamp and click " to physical actuation device button by user.In some cases, user will Feel sense of touch, such as " click is pressed " or " unclamp and click ", even if being physically pressed (example by the movement of user Such as, be shifted) physical actuation button associated with touch sensitive surface when not moving.In another example, touch sensitive surface Movement optionally by user interpret or sense be touch sensitive surface " roughness ", even if the smoothness in touch sensitive surface is unchanged When.Although the individuation sensory perception by user is limited such interpretation of touch by user, there is being permitted for touch Intersensory perception is that most users are shared.Therefore, when tactile output is described as the specific sensory perception corresponding to user When (for example, " pressing click ", " unclamp and click ", " roughness "), unless otherwise stated, otherwise tactile output generated is corresponded to In equipment or the physical displacement of its component, which will generate the sensory perception of typical (or common) user.

It should be appreciated that equipment 100 is only an example of portable multifunction device, and equipment 100 optionally has There are components more more or fewer than shown component, optionally combines two or more components, or optionally there is this The different configurations of a little components or arrangement.Various parts shown in Figure 1A are in hardware, software, firmware or any combination of them Implement in (including one or more signal processing circuits and/or specific integrated circuit).

Memory 102 optionally includes high-speed random access memory, and also optionally includes nonvolatile memory, Such as one or more disk storage equipments, flash memory device or other non-volatile solid state memory equipment.Equipment 100 other component such as one or more CPU 120 and peripheral device interface 118 to the access of memory 102 optionally by Memory Controller 122 controls.

Peripheral device interface 118 can be used for the input peripheral of equipment and output peripheral equipment being couple to one or more A CPU 120 and memory 102.There are various in reservoir 102 by storage for the one or more processors 120 operation or implementation Software program and/or instruction set, to execute the various functions of equipment 100 and handle data.

In some embodiments, peripheral device interface 118, one or more CPU 120 and Memory Controller 122 are appointed Selection of land is realized on one single chip such as chip 104.In some other embodiments, they are optionally real on separate chips It is existing.

RF (radio frequency) circuit 108 sends and receivees the RF signal for being also designated as electromagnetic signal.RF circuit 108 turns electric signal Be changed to electromagnetic signal/by electromagnetic signal and be converted to electric signal, and via electromagnetic signal come with communication network and other communicate and set It is standby to be communicated.RF circuit 108 optionally includes the well known circuit for executing these functions, including but not limited to aerial system System, RF transceiver, one or more amplifiers, tuner, one or more oscillators, digital signal processor, encoding and decoding core Piece group, subscriber identity module (SIM) card, memory etc..RF circuit 108 comes and network and other optionally by wireless communication Equipment is communicated, and all for example internets of these networks, also referred to as WWW (WWW), Intranet and/or wireless network are such as Cellular phone network, WLAN (LAN) and/or Metropolitan Area Network (MAN) (MAN).Wireless communication optionally using a variety of communication standards, Any one of agreement and technology, including but not limited to global system for mobile communications (GSM), enhancing data GSM environment (EDGE), High-speed downlink packet access (HSDPA), High Speed Uplink Packet access (HSUPA), evolution, clear data (EV-DO), HSPA, HSPA+, double small area HSPA (DC-HSPDA), long term evolution (LTE), near-field communication (NFC), wideband code division multiple access (W- CDMA), CDMA (CDMA), time division multiple acess (TDMA), bluetooth, Wireless Fidelity (Wi-Fi) (for example, IEEE 802.11a, IEEE 802.11ac, IEEE 802.11ax, IEEE 802.11b, IEEE 802.11g and/or IEEE 802.11n), interconnection FidonetFido voice technology (VoIP), Wi-MAX, Email agreement (for example, internet message access protocol (IMAP) and/or Post office protocol (POP)), instant message is (for example, scalable message transmitting and there are agreement (XMPP), for instant message and deposit In Session initiation Protocol (SIMPLE), instant message and the presence service (IMPS) using extension) and/or short message service (SMS), or the submission date including ending this document also it is untapped go out communication protocol any other communication protocols appropriate View.

Voicefrequency circuit 110, loudspeaker 111 and microphone 113 provide the audio interface between user and equipment 100.Audio Circuit 110 receives audio data from peripheral device interface 118, audio data is converted to electric signal, and electric signal transmission is arrived Loudspeaker 111.Loudspeaker 111 converts electrical signals to the audible sound wave of the mankind.Voicefrequency circuit 110 is also received by microphone 113 The electric signal come from sound wave conversion.Voicefrequency circuit 110 converts electrical signals to audio data, and audio data is transferred to outside Peripheral equipment interface 118 is for handling.Audio data is optionally retrieved from and/or is transmitted to storage by peripheral device interface 118 Device 102 and/or RF circuit 108.In some embodiments, voicefrequency circuit 110 further includes earphone jack (for example, in Fig. 2 212).Earphone jack provides the interface between voicefrequency circuit 110 and removable audio input/output peripheral equipment, the periphery The earphone or there is output (for example, single head-receiver or bi-telephone) and input (for example, microphone) that equipment such as only exports The headset of the two.

I/O subsystem 106 is by such as touch-sensitive display system 112 of the input/output peripheral equipment in equipment 100 and other Input or control equipment 116 and peripheral device interface 118 couple.I/O subsystem 106 optionally includes display controller 156, light It learns sensor controller 158, intensity sensor controller 159, tactile feedback controller 161 and inputs or control for other One or more input controllers 160 of equipment.The one or more input controller 160 is from other inputs or controls equipment 116 reception electric signals/send other inputs for electric signal or control equipment 116.Other inputs or control equipment 116 are optionally Including physical button (for example, pushing button, rocker buttons etc.), dial, slider switch, control stick, click wheel etc..One A bit optionally in embodiments, one or more input controllers 160 are optionally coupled to any one of the following terms (or not coupling It is connected to any one of the following terms)：Keyboard, infrared port, USB port, stylus, and/or pointing device such as mouse.One A or multiple buttons (for example, 208 in Fig. 2) optionally include the volume control for loudspeaker 111 and/or microphone 113 Increase/reduction button.One or more buttons, which optionally include, pushes button (for example, 206 in Fig. 2).

Touch-sensitive display system 112 provides the input interface and output interface between equipment and user.Display controller 156 Electric signal is received from touch-sensitive display system 112 and/or electric signal is sent to touch-sensitive display system 112.Touch-sensitive display System 112 shows visual output to user.Visual output optionally includes figure, text, icon, video and their any group It closes (being referred to as " figure ").In some embodiments, the visual output of some visual outputs or whole corresponds to user interface Object.As used herein, term " showing can indicate " refers to user's interactive graphical user interface object (for example, being configured as The graphical user interface object that the input of order directional pattern user interface object is made a response).User interactive graphics user circle Example in face of elephant includes but is not limited to button, sliding block, icon, optional menu item, switch, hyperlink or other users interface Control.

Touch-sensitive display system 112, which has, receives the touch-sensitive of input from the user based on sense of touch and/or tactile Surface, sensor or sensor group.Touch-sensitive display system 112 and display controller 156 are (with any phase in memory 102 Associated module and/or instruction set are together) detection touch-sensitive display system 112 on contact (and the contact any movement or Interrupt), and detected contact is converted to and is displayed on the user interface object in touch-sensitive display system 112 The interaction of (for example, one or more soft-key buttons, icon, webpage or image).In an exemplary embodiment, touch-sensitive display system Contact point between system 112 and user corresponds to the finger or stylus of user.

Touch-sensitive display system 112 optionally uses LCD (liquid crystal display) technology, LPD (light emitting polymer displays) Technology or LED (light emitting diode) technology, but other display technologies are used in other embodiments.Touch-sensitive display system System 112 and display controller 156 are optionally using any in a variety of touch-sensing technologies of currently known or later exploitation Technology and other proximity sensor arrays or for its determining with one or more contact points of touch-sensitive display system 112 His element detects contact and its any movement or interruption, which includes but is not limited to capacitive character, resistance Property, infrared ray and surface acoustic wave technique.In an exemplary embodiment, using projection-type mutual capacitance detection technology, such as From Apple Inc.'s (Cupertino, California)iPodWithThe technology of middle discovery.

Touch-sensitive display system 112 is optionally with the video resolution for being more than 100dpi.In some embodiments, it touches Touching screen video resolution is more than 400dpi (for example, 500dpi, 800dpi or bigger).User optionally uses any suitable object Body or additives such as stylus, finger etc. are contacted with touch-sensitive display system 112.In some embodiments, the use Family interface is designed to work together with contact and gesture based on finger, due to the contact area of finger on the touchscreen compared with Greatly, therefore this may be accurate not as good as the input based on stylus.In some embodiments, equipment will be based on the rough of finger Input is translated as accurate pointer/cursor position or order, for executing the desired movement of user.

In some embodiments, in addition to a touch, equipment 100 optionally includes specific for activating or deactivating The touch tablet (not shown) of function.In some embodiments, touch tablet is the touch sensitive regions of equipment, the touch sensitive regions and touch Screen is different, does not show visual output.Touch tablet is optionally the touch sensitive surface separated with touch-sensitive display system 112, or It is the extension of the touch sensitive surface formed by touch screen.

Equipment 100 further includes the electric system 162 for powering for various parts.Electric system 162 optionally includes electricity Power management system, one or more power supply (for example, battery, alternating current (AC)), recharging system, power failure detection circuit, Power converter or inverter, power status indicator (for example, light emitting diode (LED)) and with the electric power in portable device Generate, manage, and distribute any other associated component.

Equipment 100 optionally further includes one or more optical sensors 164.Figure 1A show in I/O subsystem 106 The optical sensor that optical sensor controller 158 couples.One or more optical sensors 164 optionally include Charged Couple Device (CCD) or complementary metal oxide semiconductor (CMOS) phototransistor.One or more optical sensors 164 are from environment The light by one or more lens projects is received, and converts light to indicate the data of image.In conjunction with image-forming module 143 (also referred to as camera model), one or more optical sensors 164 optionally capture still image and/or video.In some realities It applies in scheme, optical sensor is located at the opposite facing rear portion of touch-sensitive display system 112 on the front with equipment of equipment 100 On, so that touch screen can be used as the view finder for still image and/or video image acquisition.In some embodiments, Another optical sensor is located on the front of equipment, thus obtain the user image (such as self-timer, for existing in user Video conference etc. is carried out when watching other video conference participants on touch screen).

Equipment 100 optionally further includes one or more contact strength sensors 165.Figure 1A is shown and I/O subsystem 106 In intensity sensor controller 159 couple contact strength sensor.One or more contact strength sensors 165 are optionally Including one or more piezoresistive strain gauges, capacitive force transducer, power sensor, piezoelectric force transducer, optics force snesor, Capacitive touch sensitive surfaces or other intensity sensors are (for example, the biography of the power (or pressure) for measuring the contact on touch sensitive surface Sensor).One or more contact strength sensors 165 receive contact strength information (for example, pressure information or pressure letter from environment The surrogate of breath).In some embodiments, at least one contact strength sensor and touch sensitive surface are (for example, touch-sensitive display System 112) Alignment or neighbouring.In some embodiments, at least one contact strength sensor be located at equipment 100 with On the opposite facing rear portion of touch-screen display system 112 on the front of equipment 100.

Equipment 100 optionally further includes one or more proximity sensors 166.Figure 1A is shown and peripheral device interface 118 The proximity sensor 166 of coupling.Alternatively, the input control in proximity sensor 166 and I/O subsystem 106 Device 160 couples.In some embodiments, when multifunctional equipment is placed near user's ear (for example, user is beating When phone), proximity sensor closes and disables touch-sensitive display system 112.

Equipment 100 optionally further includes one or more tactile output generators 167.Figure 1A is shown and I/O subsystem 106 In tactile feedback controller 161 couple tactile output generator.One or more tactile output generators 167 optionally wrap One or more electroacoustic equipments, such as loudspeaker or other acoustic components are included, and/or convert the energy into the electromechanics of linear movement Equipment, such as motor, solenoid, electroactive polymerizer, piezoelectric actuator, electrostatic actuator or other tactiles export generating unit (for example, the component for converting the electrical signal to the tactile output in equipment).The one or more tactile output generator 167 is from touching Feel that feedback module 133 receives touch feedback and generates instruction, and generates can be felt by the user of equipment 100 on the appliance 100 The tactile output arrived.In some embodiments, at least one tactile output generator and touch sensitive surface are (for example, touch-sensitive display Device system 112) Alignment or neighbouring, and optionally by vertically (for example, to the surface inside/outside of equipment 100) or side Tactile output is generated to ground (for example, in plane identical with the surface of equipment 100 rearwardly and a forwardly) mobile touch sensitive surface. In some embodiments, at least one tactile output generator sensor be located at equipment 100 be located at equipment 100 front On the opposite facing rear portion of touch-sensitive display system 112 on.

Equipment 100 optionally further includes one or more accelerometers 168.Figure 1A is shown and 118 coupling of peripheral device interface The accelerometer 168 connect.Alternatively, accelerometer 168 optionally with the input control in I/O subsystem 106 Device 160 couples.In some embodiments, based on to the analysis from the received data of one or more accelerometers in touch screen Information is shown with longitudinal view or transverse views on display.Equipment 100 is optionally in addition to one or more accelerometers 168 Except further include magnetometer (not shown) and GPS (or GLONASS or other Global Navigation Systems) receiver (not shown), with In information of the acquisition about the position of equipment 100 and orientation (for example, vertical or horizontal).

In some embodiments, the software component being stored in memory 102 includes operating system 126, communication mould Block (or instruction set) 128, contact/motion module (or instruction set) 130, figure module (or instruction set) 132, haptic feedback module (or instruction set) 133, global positioning system (GPS) module (or instruction set) 135, is answered at text input module (or instruction set) 134 With program (or instruction set) 136 and specific to module (including the audio preview module 163-1, audio modification module of audio 163-2 and audio change module 163-3).In addition, in some embodiments, memory 102 stores the internal shape of equipment/overall situation State 157, as shown in Figure 1A and Fig. 3.Equipment/overall situation internal state 157 includes one or more of the following terms：Activity is answered With program state, it is currently movable which, which is used to indicate which application program (if any),；It is aobvious Show state, which is used to indicate what application program, view or other information and occupies each of touch-sensitive display system 112 A region；Sensor states, the sensor states include obtaining from each sensor of equipment and other inputs or control equipment 116 The information taken；With about device location and/or posture position and/or location information.

Operating system 126 is (for example, iOS, Darwin, RTXC, LINUX, UNIX, OSX, WINDOWS or embedded operation System such as VxWorks) include for control and manage general system task (for example, memory management, storage equipment control, Electrical management etc.) various software components and/or driver, and be conducive to the communication between various hardware and software components.

Communication module 128 promotes through one or more outside ports 124 to be communicated with other equipment, and also wraps It includes for handling by the various software components of 124 received data of RF circuit 108 and/or outside port.Outside port 124 (for example, universal serial bus (USB), firewire etc.) is suitable for being directly coupled to other equipment or indirectly via network (for example, mutually Networking, Wireless LAN etc.) coupling.In some embodiments, outside port be with Apple Inc. (Cupertino, California) someEquipment, iPod30 needle connectors used in equipment and iPod equipment Same or like and/or compatible spininess (for example, 30 needles) connector.In some embodiments, outside port is and Apple Inc. (Cupertino, California's) is someEquipment, iPodMade in equipment and iPod equipment The same or like and/or compatible Lightning connector of Lightning connector.

Contact/motion module 130 optionally detects and touch-sensitive display system 112 (in conjunction with display controller 156) and its The contact of his touch-sensitive device (for example, touch tablet or physics click wheel).Contact/motion module 130 include various software components with The relevant various operations of detection for executing with contacting (for example, pass through finger or pass through stylus), such as to determine that whether having sent out Raw contact (for example, detection finger down event) determines the intensity of contact (for example, the power or pressure of contact, or the power of contact Or the substitute of pressure), determine whether there is contact movement and track across touch sensitive surface movement (for example, detection one or Multiple finger drag events) and determine contact (for example, detection digit up event or contact disconnect) whether it has stopped. Contact/motion module 130 receives contact data from touch sensitive surface.Determine that the movement of contact point optionally includes determining contact point Rate (magnitude), speed (magnitude and direction) and/or acceleration (variation in magnitude and/or direction), the movement of the contact point by A series of contact data indicate.These operations are optionally applied to single-contact (for example, single abutment or stylus contact) Or multiple spot contacts (for example, " multiple point touching "/more abutments) simultaneously.In some embodiments, contact/motion module 130 The contact on touch tablet is detected with display controller 156.

Contact/motion module 130 optionally detects the gesture input of user.Different gestures on touch sensitive surface have difference Contact patterns (for example, the different motion of detected contact, timing and/or intensity).Therefore, special optionally by detection Determine contact patterns and carrys out detection gesture.For example, detection singly refers to that Flick gesture includes detection finger down event, then pressed with finger Detection finger lifts and (is lifted away from) thing at (for example, at picture mark position) the identical position (or substantially the same position) of lower event Part.In another example, detecting that finger gently sweeps gesture on touch sensitive surface includes detecting finger down event, is then examined One or more finger drag events are measured, and subsequently detects finger and lifts and (be lifted away from) event.Similarly, optionally by The specific contact patterns of detection stylus detect tap for stylus, gently sweep, drag and other gestures.

In some embodiments, detection finger Flick gesture, which depends on, detects that finger down event and finger lift thing Time span between part, but be independently of and detect that the finger between finger down event and digit up event contacts strong Degree.In some embodiments, it is less than according to the time span determined between finger down event and digit up event preparatory Determining value (for example, less than 0.1,0.2,0.3,0.4 or 0.5 second) detects Flick gesture, but regardless of the finger during tap Whether the intensity of contact meets given intensity threshold (being greater than Nominal contact detection intensity threshold value), such as light Compressive Strength threshold value or depth Compressive Strength threshold value.Therefore, finger Flick gesture, which can meet, does not require the characteristic strength of contact to meet given intensity threshold so as to full The specific input standard of the specific input standard of foot.For the sake of clarity, the finger contact in Flick gesture usually requires to meet mark Claim contact detection intensity threshold value to detect finger down event, cannot be detected lower than the Nominal contact detection intensity threshold value Contact.Similar analysis is applied to the Flick gesture that detection is carried out by stylus or other contacts.It is able to detect that in equipment outstanding In the case where finger or the stylus contact being parked in above touch sensitive surface, Nominal contact detection intensity threshold value does not correspond to hand optionally Finger or the physical contact between stylus and touch sensitive surface.

Same concept is applied to other kinds of gesture in a similar manner.For example, being optionally based on satisfaction and being included in The intensity of contact in gesture is unrelated or the one or more contacts for executing gesture is not required to reach intensity threshold to be known Other standard gently sweeps gesture, kneading gesture, cancellation kneading gesture and/or long-pressing gesture to detect.For example, based on one or more The amount of movement of contact gently sweeps gesture to detect；Kneading gesture is detected based on two or more movements of contact toward each other； Cancellation kneading gesture is detected based on two or more movements of contact away from each other；And based on having on touch sensitive surface Long-pressing gesture is detected less than the duration of the contact of amount of threshold shift.Therefore, certain gestures criterion of identification does not require one Or the intensity of multiple contacts meets respective strengths threshold value and means certain gestures to meet the statement of certain gestures criterion of identification Criterion of identification can be met in the case where one or more contacts in gesture do not reach respective strengths threshold value, and It can also be met in the case where one or more contacts in the contact in gesture meet or exceed respective strengths threshold value. In some embodiments, finger down and digit up event are detected within the predefined period based on determination to detect Flick gesture without considering that contact is above or below respective strengths threshold value during the predefined period, and is based on It determines that contact is mobile and gently sweeps gesture to detect greater than predefined magnitude, even if contact is also above corresponding at the end of contacting mobile Intensity threshold.Even if being influenced by the intensity for the contact for executing gesture (for example, equipment is in the strong of contact in the detection to gesture Degree detects long-pressing when being higher than intensity threshold quickly, or postpones the detection to tap input when the intensity of contact is higher) Specific implementation in, to the detection of those gestures do not require contact reach certain strength threshold value, as long as can contact not reach To meeting standard for identifying a gesture in the case where certain strength threshold value (for example, even if amount changes the time spent in identification gesture Become).

In some cases, contact strength threshold value, duration threshold and mobile threshold value are combined into a variety of different groups It closes, to create the heuristics for distinguishing two or more the different gestures for being directed toward identical input element or region, so that Multiple and different interactions with identical input element are enabled to provide one group of richer user's interaction and response.One group of certain gestures Criterion of identification does not require one or more intensity contacted to meet respective strengths threshold value to meet certain gestures criterion of identification Statement, which is not precluded while other intensity rely on gesture identification standard, to be assessed to identify that having in gesture includes having higher than phase Answer other gestures of the standard met when the contact of the intensity of intensity threshold.For example, in some cases, for first gesture (it does not require one or more intensity contacted to meet respective strengths threshold value to meet first gesture to first gesture criterion of identification Criterion of identification) (it is contacted dependent on one or more reaches respective strengths with the second gesture criterion of identification for second gesture Threshold value) competition.In such competition, if meeting the second gesture criterion of identification for being used for second gesture first, gesture is optional Ground is not identified as meeting the first gesture criterion of identification for being used for first gesture.For example, if contact is mobile predefined in contact Amount of movement before reach respective strengths threshold value, then detect deep pressure gesture rather than gently sweep gesture.On the contrary, if contact is connecing Touching mobile predefined amount of movement before reaching respective strengths threshold value then detects and gently sweeps gesture rather than deep pressure gesture.Even if In such cases, one or more intensity contacted are not still required to meet for the first gesture criterion of identification of first gesture Respective strengths threshold value is to meet first gesture criterion of identification, because if contact keeps below respective strengths threshold value until gesture Terminate (for example, have the contact for the intensity for not increasing to above respective strengths threshold value gently sweep gesture), then gesture will be by the One gesture criterion of identification is identified as gently sweeping gesture.Therefore, it is not required to which the intensity of one or more contacts meets respective strengths threshold value (A) is ignored into contact relative to intensity in some cases to meet the certain gestures criterion of identification of certain gestures criterion of identification The intensity (for example, for Flick gesture) of threshold value and/or (B) still rely on contact relative to intensity threshold under certain conditions Intensity, in some sense if certain gestures criterion of identification recognize corresponding with input gesture (for example, for just with Deep pressure gesture is at war with the long-pressing gesture identified) the interdependent gesture identification standard of previous group competitiveness intensity (for example, Gesture is pressed for deep) input is identified as corresponding to the interdependent gesture of intensity, then the certain gestures criterion of identification is (for example, for length By gesture) it will failure.

Figure module 132 includes for figure to be presented and shown in touch-sensitive display system 112 or other displays Various known software components, including the visual impact for changing shown figure (for example, brightness, transparency, saturation Degree, contrast or other perceptual properties) component.As used herein, term " figure " includes can be displayed to user any Object, including but not limited to text, icon (such as including the user interface object of soft key), digital picture, video, move webpage Draw etc..

In some embodiments, figure module 132 stores ready for use for indicating the data of figure.Each figure is appointed Selection of land is assigned corresponding code.Figure module 132 specifies one or more of figure to be shown from receptions such as application programs A code also receives coordinate data and other graphic attribute data in the case of necessary, and then generates screen picture number It is output to display controller 156 accordingly.

Haptic feedback module 133 includes the various software components for generating instruction, and the instruction is by one or more tactiles Output generator 167 uses, one or more positions so as to the interaction in response to user and equipment 100 on the appliance 100 Place generates tactile output.

The text input module 134 for being optionally the component of figure module 132 is provided in various application program (examples Such as, contact person 137, Email 140, IM 141, browser 147 and any other application program for needing text input) in Input the soft key of text.

GPS module 135 determines the position of equipment, and provides the information to use in various application programs (for example, mentioning Supply phone 138 is to dial for location-based, be supplied to camera 143 as photo/video metadata and be supplied to For the application program of location based service, such as weather desktop small routine, local Yellow Page desktop small routine and map/navigation Desktop small routine).

It optionally includes using 136 with lower module (or instruction set) or its subset or superset：

Contact module 137 (sometimes referred to as address list or contacts list)；

Phone module 138；

Video conference module 139；

Email client module 140；

Instant message (IM) module 141；

News template 142；

For static image and/or the camera model 143 of video image；

Image management module 144；

Browser module 147；

Calendaring module 148；

Desktop small routine module 149, optionally includes one or more of the following terms：Weather desktop small routine 149-1, stock market desktop small routine 149-2, calculator desktop small routine 149-3, alarm clock desktop small routine 149-4, dictionary desktop The desktop small routine 149-6 of small routine 149-5 and other desktop small routines and user's creation for being obtained by user；

For making the desktop small routine builder module 150 of the desktop small routine 149-6 of user's creation；

Search module 151；

Video and musical player module 152, optionally by video player module and musical player module structure At；

Notepad module 153；

Mapping module 154；And/or

Online Video module 155.

The example for the other applications 136 being optionally stored in memory 102 includes other text processing applications Program, other picture editting's application programs, drawing application program, present application program, support the application program of JAVA, encryption, Digital copyright management, speech recognition and speech reproduction.

It is defeated in conjunction with touch-sensitive display system 112, display controller 156, contact module 130, figure module 132 and text Enter module 134, contact module 137 includes for managing address list or contacts list (for example, being stored in memory 102 Or in the application program internal state 192 of the contact module 137 in memory 370) executable instruction, including：By one Or multiple names are added to address list；One or more names are deleted from address list；Make one or more telephone numbers, one Or multiple e-mail addresses, one or more physical address or other information are associated with name；Keep image related to name Connection；Classified to name and is sorted；Telephone number and/or e-mail address are provided and pass through phone to initiate and/or promote 138, the communication that video conference 139, Email 140 or IM141 are carried out；Etc..

In conjunction with RF circuit 108, voicefrequency circuit 110, loudspeaker 111, microphone 113, touch-sensitive display system 112, display Controller 156, contact module 130, figure module 132 and text input module 134, phone module 138 include for input with The phone number that one or more telephone numbers, modification in the corresponding character string of telephone number, accessing address list 137 have inputted Code, the executable instruction for dialing corresponding telephone number, conversating and disconnecting or hang up when session is completed.Institute as above It states, wireless communication is optionally using any one of a variety of communication standards, agreement and technology.

In conjunction with RF circuit 108, voicefrequency circuit 110, loudspeaker 111, microphone 113, touch-sensitive display system 112, display Controller 156, one or more optical sensors 164, optical sensor controller 158, contact module 130, figure module 132, text input module 134, contacts list 137 and phone module 138, video conference module 139 include for according to The executable instruction of the video conference between user and other one or more participants is initiated, carries out and is terminated in family instruction.

In conjunction with RF circuit 108, touch-sensitive display system 112, display controller 156, contact module 130, figure module 132 With text input module 134, email client module 140 includes for creating, sending in response to user instruction, receive With the executable instruction of management Email.In conjunction with image management module 144, email client module 140 makes very It is easy creation and sends the Email with the still image or video image that are shot by camera model 143.

In conjunction with RF circuit 108, touch-sensitive display system 112, display controller 156, contact module 130, figure module 132 With text input module 134, instant message module 141 includes for inputting character string corresponding with instant message, modification first The character of preceding input sends corresponding instant message (for example, using the short message service for being directed to the instant message based on phone (SMS) or multimedia messaging service (MMS) agreement or using for instant message Internet-based XMPP, SIMPLE, Apple push notification service (APNs) or IMPS), receive instant message and check received instant message executable finger It enables.In some embodiments, the instant message transferred and/or received optionally includes figure, photo, audio file, view Frequency file and/or other attachmentes supported in MMS and/or enhancing messaging service (EMS).As used herein, " disappear immediately Breath " refers to message (for example, the message sent using SMS or MMS) based on phone and message Internet-based (for example, making Both the message sent with XMPP, SIMPLE, APNs or IMPS).

In conjunction with touch-sensitive display system 112, display controller 156, contact/motion module 130 and figure module 132, newly Hearing module 142 includes for showing the news article specific to user (for example, based on the preference specific to user from a variety of publication The article that object source is collected) and allow user and interact specific to the news article of user (or with specific to user's The partial content for being included in news article interacts) executable instruction.

In conjunction with touch-sensitive display system 112, display controller 156, one or more optical sensors 164, optical sensing Device controller 158, contact module 130, figure module 132 and image management module 144, camera model 143 include for capturing Still image or video (including video flowing) and the spy for storing them in memory 102, modifying still image or video Sign, and/or the executable instruction that still image or video are deleted from memory 102.

In conjunction with touch-sensitive display system 112, display controller 156, contact module 130, figure module 132, text input Module 134 and camera model 143, image management module 144 include for arranging, modifying (for example, editor) or with its other party Formula manipulates, tags, deleting, presenting (for example, in digital slide or photograph album) and storage still image and/or video The executable instruction of image.

In conjunction with RF circuit 108, touch-sensitive display system 112, display system controller 156, contact module 130, figure mould Block 132 and text input module 134, browser module 147 include according to user instructions come browse internet (including search, chain Be connected to, receive and show webpage or part thereof and be linked to the attachment and alternative document of webpage) executable instruction.

In conjunction with RF circuit 108, touch-sensitive display system 112, display system controller 156, contact module 130, figure mould Block 132, text input module 134, email client module 140 and browser module 147, calendaring module 148 include using According to user instructions come create, show, modify and store calendar and data associated with calendar (for example, calendar, Backlog etc.) executable instruction.

In conjunction with RF circuit 108, touch-sensitive display system 112, display system controller 156, contact module 130, figure mould Block 132, text input module 134 and browser module 147, desktop small routine module 149 are optionally to be downloaded and made by user Miniature applications program is (for example, weather desktop small routine 149-1, stock market desktop small routine 149-2, calculator desktop little Cheng Sequence 149-3, alarm clock desktop small routine 149-4 and dictionary desktop small routine 149-5) or by user create miniature applications program (for example, desktop small routine 149-6 of user's creation).In some embodiments, desktop small routine includes HTML (hypertext mark Remember language) file, CSS (cascading style sheets) file and JavaScript file.In some embodiments, desktop small routine packet XML (extensible markup language) file and JavaScript file are included (for example, Yahoo！Desktop small routine).

In conjunction with RF circuit 108, touch-sensitive display system 112, display system controller 156, contact module 130, figure mould Block 132, text input module 134 and browser module 147, desktop small routine builder module 150 include for creating desktop The executable instruction of small routine (for example, user's specified portions of webpage are gone in desktop small routine).

In conjunction with touch-sensitive display system 112, display system controller 156, contact module 130, figure module 132 and text Input module 134, search module 151 include for carrying out searching in searching storage 102 with one or more according to user instructions The matched text of rope condition (for example, search term that one or more user specifies), music, sound, image, video and/or its The executable instruction of his file.

In conjunction with touch-sensitive display system 112, display system controller 156, contact module 130, figure module 132, audio Circuit 110, loudspeaker 111, RF circuit 108 and browser module 147, video and musical player module 152 include allowing to use It downloads and resets with the record music and other audio files of such as MP3 or AAC file storage of one or more file formats in family Executable instruction, and for showing, presenting or otherwise reset video (for example, in touch-sensitive display system 112 Wirelessly or via outside port 124 connect external display on) executable instruction.In some embodiments, Equipment 100 optionally includes the function of MP3 player such as iPod (trade mark of Apple Inc.).

In conjunction with touch-sensitive display system 112, display controller 156, contact module 130, figure module 132 and text input Module 134, notepad module 153 include creating and managing holding for notepad, backlog etc. for according to user instructions Row instruction.

In conjunction with RF circuit 108, touch-sensitive display system 112, display system controller 156, contact module 130, figure mould Block 132, text input module 134, GPS module 135 and browser module 147, mapping module 154 include for being referred to according to user It enables to receive, show, modify and store map and data associated with map (for example, steering direction；About specific position At or near shop and other points of interest data；And other location-based data) executable instruction.

In conjunction with touch-sensitive display system 112, display system controller 156, contact module 130, figure module 132, audio Circuit 110, loudspeaker 111, RF circuit 108, text input module 134, email client module 140 and browser module 147, Online Video module 155 include executable instruction, these executable instructions allow user access, browsing, receive (for example, Pass through Streaming Media and/or downloading), reset (for example, on touch screen 112 or wirelessly or via outside port 124 connect On external display), send the Email with the link of specific Online Video, and otherwise management in a kind of or The Online Video of a variety of file formats such as H.264.In some embodiments, using instant message module 141 rather than it is electric Sub- mail client end module 140 is sent to the link of specific Online Video.

As depicted in fig. 1A, portable multifunction device 100 further includes for providing the sound of acoustic information to audio system Frequency output provides module 163, so that audio output can be presented (for example, as shown in figure 5, audio system can be included in audio system It is separated in portable multifunction device 100 or with portable multifunction device 100).Audio output provides module 163 optionally Comprise the following modules (or instruction set) or its subset or superset：

For storing about corresponding to or generate in response to for example being interacted with the user of portable multifunction device 100 The information of acoustic characteristic audio configuration file 402 (for example, audio envelope characteristic, pitch characteristic, left and right (L-R) equilibrium response, Reverberation curve, frequency filtering property)；

Audio preview module 163-1 comprising for providing information to play the pre- of audio content (for example, particular songs) Look at and be optionally in response to the request that the first audio output is presented and adjust the presentation of different content (for example, different song) The executable instruction of (for example, making this two song fuzzy together)；

Audio modification module 163-2 comprising in response to detecting that (example can be indicated by being directed toward showing in user interface Such as, it is identical show can indicate or difference is shown to indicate) second user input and change the first audio output (for example, being used by being directed toward The first user that showing in the interface of family can indicate inputs the audio output input generated) executable instruction.In some embodiment party In case, audio modification module 163-1 modifies corresponding audio configuration file 402；And

Audio changes standard module 163-3 comprising determines audio modification module for changing standard based on audio Whether 163-2 should modify the executable instruction of the first audio output.

Above-mentioned identified each module and application program correspond to for execute above-mentioned one or more functions and Method (for example, computer implemented method described herein and other information processing method) described herein One group of executable instruction.These modules (that is, instruction set) need not be implemented as separated software program, process, or module, and Therefore each seed group of these modules is optionally combined in various embodiments or otherwise rearranges.Some In embodiment, memory 102 optionally stores the subset of above-mentioned module and data structure.In addition, memory 102 is optionally deposited Store up add-on module and data structure not described above.

In some embodiments, equipment 100 is such a equipment, i.e., one group of predefined function in the equipment Operation is uniquely executed by touch screen and/or touch tablet.It is used as by using touch screen and/or touch tablet and is used for equipment The main input control apparatus of 100 operation, the control equipment that is physically entered optionally in reduction equipment 100 (such as press down Button, dial etc.) quantity.

User is uniquely optionally included in by one group of predefined function that touch screen and/or touch tablet execute Navigation between interface.In some embodiments, touch tablet shows equipment 100 when being touched by user from equipment 100 Any user interface navigation shown is to main menu, home menu or root menu.It is real using touch tablet in such embodiment Existing " menu button ".In some other embodiments, menu button is that physics pushes button or other are physically entered control Equipment rather than touch tablet.

Figure 1B is the block diagram for showing the example components for event handling according to some embodiments.In some implementations In scheme, memory 102 (in Figure 1A) or 370 (Fig. 3) include event classifier 170 (for example, in operating system 126) and Corresponding application program 136-1 is (for example, any of aforementioned applications program 136,137 to 155,380 to 390 applies journey Sequence).

Event classifier 170 receives event information and determination for application program 136-1 that event information is delivered to and answers With the application view 191 of program 136-1.Event classifier 170 includes event monitor 171 and event dispatcher module 174.In some embodiments, application program 136-1 includes application program internal state 192, the application program internal state The one or more that indicating is movable when application program or while being carrying out shows in touch-sensitive display system 112 is currently answered Use Views.In some embodiments, equipment/overall situation internal state 157 is by event classifier 170 for which to be determined (which) application program is currently movable, and application program internal state 192 will for determination by event classifier 170 The application view 191 that event information is delivered to.

In some embodiments, application program internal state 192 includes additional information, such as one of the following terms Or more persons：When application program 136-1 restores to execute recoverys information to be used, indicate just shown by application program 136-1 Information or be ready for by the user interface state information of the application program 136-1 information shown, for enabling a user to The repetition for the prior actions enough taken back to the state queue and user of the previous state of application program 136-1 or view/ Cancel queue.

Event monitor 171 receives event information from peripheral device interface 118.Event information includes about subevent (example Such as, as in the touch-sensitive display system 112 of a part of multi-touch gesture user touch) information.Peripheral equipment connects Mouthfuls 118 transmission its from I/O subsystem 106 or sensor such as proximity sensor 166, one or more accelerometers 168 and/or Microphone 113 (passing through voicefrequency circuit 110) received information.Peripheral device interface 118 is from the received letters of the institute of I/O subsystem 106 Breath includes the information from touch-sensitive display system 112 or touch sensitive surface.

In some embodiments, event monitor 171 sends the request to peripheral equipment at predetermined intervals and connects Mouth 118.In response, 118 transmitting event information of peripheral device interface.In other embodiments, peripheral device interface 118 is only When there are significant events (for example, receiving higher than the input of predetermined noise threshold and/or more than predetermined lasting Time) when transmitting event information.

In some embodiments, event classifier 170 further includes hit view determination module 172 and/or life event Identifier determining module 173.

When touch-sensitive display system 112 shows more than one view, hit view determination module 172 is provided for determining The subevent software process where occurred in one or more views.View can be seen over the display by user Control and other elements constitute.

The another aspect of user interface associated with application program is one group of view, is otherwise referred to as applied herein Views or user interface windows are wherein showing information and the gesture based on touch occur.Touch is detected wherein (corresponding application programs) application view optionally correspond in the sequencing or view hierarchies structure of application program Sequencing is horizontal.For example, being optionally referred to as hit view in the floor level view for wherein detecting touch, and it is considered It is that the event set correctly entered is based in part on the hit view of initial touch optionally at least to determine, which starts Gesture based on touch.

It hits view determination module 172 and receives information relevant to the subevent of the gesture based on contact.Work as application program When with the multiple views organized in hierarchical structure, hit view determination module 172 will hit view, and be identified as should be to sub- thing Minimum view in the hierarchical structure that part is handled.In most cases, hit view is to initiate subevent (to form thing The first subevent in the subevent sequence of part or potential event) in the floor level view wherein occurred.Once hitting view It is hit view determination module to be identified, hit view, which is usually received, to be identified as hitting the targeted same touch of view with it Or the relevant all subevents of input source.

It is specific that life event identifier determining module 173 determines which or which view in view hierarchies structure should receive Subevent sequence.In some embodiments, life event identifier determining module 173 determines that only hit view should receive specific Subevent sequence.In other embodiments, life event identifier determining module 173 determines the physical location including subevent All views be the active view participated in, and it is thus determined that all views actively participated in should receive specific subevent sequence Column.In other embodiments, it even if touch subevent is confined to region associated with a particular figure completely, is classified Higher view in structure will still maintain view for active participation.

Event information is assigned to event recognizer (for example, event recognizer 180) by event dispatcher module 174.It is wrapping In the embodiment for including life event identifier determining module 173, event information is delivered to by living by event dispatcher module 174 Dynamic 173 definite event identifier of event recognizer determining module.In some embodiments, event dispatcher module 174 exists Event information is stored in event queue, which is retrieved by corresponding event receiver module 182.

In some embodiments, operating system 126 includes event classifier 170.Alternatively, using journey Sequence 136-1 includes event classifier 170.In yet another embodiment, event classifier 170 is standalone module or quilt A part of another module such as contact/motion module 130 of storage in the memory 102.

In some embodiments, application program 136-1 includes multiple button.onreleases 190 and one or more application Views 191, wherein each application view includes for handling the corresponding view occurred in the user interface of application program The instruction of touch event in figure.Each application view 191 using 136-1 includes one or more event recognizers 180.It is logical Often, respective application view 191 includes multiple event recognizers 180.In other embodiments, one in event recognizer 180 A or multiple event recognizers are a part of standalone module, the standalone module such as user interface tool packet (not shown) or are answered With the higher levels of object of program 136-1 therefrom inheritance method and other attributes.In some embodiments, at corresponding event Manage program 190 include it is following one or more：It data renovator 176, object renovator 177, GUI renovator 178 and/or is engaged in The received event data 179 of part classifier 170.Button.onrelease 190 optionally with or call data renovator 176, right As renovator 177 or GUI renovator 178 carry out more new application internal state 192.Alternatively, application program One or more application views in view 191 include one or more corresponding event processing routines 190.In addition, one In a little embodiments, one or more of data renovator 176, object renovator 177 and GUI renovator 178 are included in In corresponding application programs view 191.

Corresponding event recognizer 180 receives event information (for example, event data 179) from event classifier 170, and From event information identification events.Event recognizer 180 includes Event receiver 182 and event comparator 184.In some embodiment party In case, event recognizer 180 further includes at least one of the following subset：Metadata 183 and event transmitting instruction 188 (its optionally It transmits and instructs including subevent).

Event receiver 182 receives event information from event classifier 170.Event information includes for example touching about subevent Touch or touch mobile information.According to subevent, event information further includes additional information, the position of such as subevent.Group thing When part is related to the movement touched, event information optionally further includes rate and the direction of subevent.In some embodiments, thing Part, which includes equipment, rotates to another orientation (for example, rotate to horizontal orientation from machine-direction oriented, or vice versa) from an orientation, And event information includes the corresponding informance of the current orientation (also referred to as equipment posture) about equipment.

Event information and predefined event or subevent definition are compared by event comparator 184, and are based on This compares, and determines an event or subevent, or the state of determining or update event or subevent.In some embodiments In, event comparator 184 includes that event defines 186.Event defines 186 and includes event (such as event 1 (187-1), event 2 (187-2) and other events) definition (for example, predefined subevent sequence).In some embodiments, event Subevent in 187 includes for example touching start, touch and terminate, touch mobile, touch cancellation and multiple point touching.In an example In, the definition of event 1 (187-1) is the double-click on shown object.For example, it includes preparatory on shown object for double-clicking Determine that the first time of duration touches (touch starts), the first time of predetermined duration is lifted away from (touch terminates), shown object On touch (touch starts) second of predetermined duration and being lifted away from for the second time for predetermined duration (touches knot Beam).In another example, the definition of event 2 (187-2) is the dragging shown on object.For example, dragging includes shown pair Touch (or contact), the lift that touches the movement in touch-sensitive display system 112 and touch as upper predetermined duration It rises (touch terminates).In some embodiments, event further includes for one or more associated button.onreleases 190 Information.

In some embodiments, it includes the definition for the event of respective user interfaces object that event, which defines 187,.One In a little embodiments, event comparator 184 executes hit test to determine which user interface object is associated with subevent.Example Such as, in the application view that three user interface objects are shown in touch-sensitive display system 112, when in touch-sensitive display system When detecting touch on system 112, event comparator 184 executes hit test to determine which of these three user interface objects User interface object is associated with touch (subevent).If each shown object and corresponding button.onrelease 190 is associated, then event comparator determines which button.onrelease 190 should be swashed using the result that the hit is tested It is living.For example, the selection of event comparator 184 button.onrelease associated with the object of subevent and triggering hit test.

In some embodiments, the definition of corresponding event 187 further includes delay voltage, which postpones event letter The delivering of breath, until having determined that whether subevent sequence exactly corresponds to or do not correspond to the event type of event recognizer.

It, should when the determining subevent sequence of corresponding event identifier 180 does not define any event in 186 with event to be matched 180 entry event of corresponding event identifier is impossible, event fails or event terminates state, ignores after this based on touch The subsequent subevent of gesture.In this case, for hit view keep other movable event recognizers (if there is Words) continue to track and handle the subevent of the lasting gesture based on touch.

In some embodiments, corresponding event identifier 180 includes having how instruction event delivery system should be held Configurable attribute, label and/or the metadata of list 183 of the row to the subevent delivering of the event recognizer of active participation.? In some embodiments, metadata 183 includes indicating how event recognizer interacts or how to interact each other and is configurable Attribute, mark and/or list.In some embodiments, metadata 183 includes whether instruction subevent is delivered to view or journey Configurable attribute, mark and/or the list of different levels in sequence hierarchical structure.

In some embodiments, when one or more specific subevents of identification events, corresponding event identifier 180 Activate button.onrelease 190 associated with event.In some embodiments, corresponding event identifier 180 will be with event Associated event information is delivered to button.onrelease 190.Activation button.onrelease 190 is different from sending subevent (and delaying to send) hits view to corresponding.In some embodiments, event recognizer 180 is dished out and the event phase that is identified Associated label, and button.onrelease 190 associated with the label obtains the label and executes predefined process.

In some embodiments, event delivery instruction 188 includes delivering the event information about subevent without swashing The subevent delivery instructions of button.onrelease living.On the contrary, event information is delivered to and subevent sequence by subevent delivery instructions It arranges associated button.onrelease or is delivered to the view of active participation.View with subevent sequence or with active participation Associated button.onrelease receives event information and executes predetermined process.

In some embodiments, data renovator 176 creates and updates the data used in application program 136-1. For example, data renovator 176 is updated telephone number used in contact module 137, or to video player Video file used in module 145 is stored.In some embodiments, object renovator 177 is created and is updated Object used in application program 136-1.For example, object renovator 177 creates new user interface object or updates user circle In face of the position of elephant.GUI renovator 178 updates GUI.For example, GUI renovator 178 prepares display information and sends it to figure Shape module 132 for showing on the touch sensitive display.

In some embodiments, one or more button.onreleases 190 include data renovator 176, object update Device 177 and GUI renovator 178 or with the access authority to them.In some embodiments, data renovator 176, right As renovator 177 and GUI renovator 178 are included in the individual module of corresponding application programs 136-1 or application view 191 In.In other embodiments, they are included in two or more software modules.

It should be appreciated that being also applied to about the discussed above of event handling that the user on touch-sensitive display touches using defeated Enter user's input that equipment carrys out the other forms of operating multifunction equipment 100, not all user's input is all in touch screen Upper initiation.For example, mobile optionally with mouse and mouse button presses, optionally in combination with single or multiple keyboard pressings or It keeps；Contact on touch tablet is mobile, tap, dragging, rolling etc.；Stylus input；The movement of equipment；Spoken command；Inspection The eye motion measured；Bio-identification input；And/or any combination thereof is as corresponding with the subevent for the event to be identified is defined Input.

Fig. 2 shows according to some embodiments with touch screen (for example, touch-sensitive display system 112 of Figure 1A) Portable multifunction device 100.Touch screen optionally shows one or more figures in user interface (UI) 200.In this reality In other embodiments applied in scheme and be described below, user can be by, for example, one or more fingers 202 (being not necessarily to scale in the accompanying drawings) utilize one or more stylus 203 (being not necessarily to scale in the accompanying drawings) Gesture is made on figure to select one or more figures in these figures.In some embodiments, when user is interrupted When with the contact of one or more figures, the selection to one or more figures will occur.In some embodiments, gesture is appointed Selection of land includes one or many taps, one or many gently sweeps (from left to right, from right to left, up and/or down) and/or The rolling (from right to left, from left to right, up and/or down) for the finger being in contact with equipment 100.In some specific implementations In or in some cases, inadvertently will not select figure with pattern contact.For example, when gesture corresponding with selection is tap When, what is swept above application icon gently sweeps gesture optionally and will not select corresponding application program.

Equipment 100 optionally further includes one or more physical buttons, such as " home " button or menu button 204.Such as Preceding described, menu button 204 is optionally for times navigate in one group of application program being optionally performed on the appliance 100 What application program 136.Alternatively, in some embodiments, menu button is implemented as being displayed on touch The soft-key button in GUI on panel type display 112.

In some embodiments, equipment 100 include touch-screen display, menu button 204, for make equipment be powered/ It powers off and pushing button 206, one or more volume knobs 208, subscriber identity module (SIM) for locking device blocks Slot 210, earphone jack 212 and docking/charging external port 124.Button 206 is pushed optionally for by pressing lower button and making Button keeps predefined time interval in pressed status to keep equipment power on/off；By by lower button and by pre- Release button carrys out locking device before the time interval of definition；And/or unlocking process is unlocked or initiated to equipment.Some In embodiment, equipment 100 also receives the input of the voice for activating or deactivating certain functions by microphone 113.If Standby 100 also optionally include one or more contact strengths of the intensity for detecting the contact in touch-sensitive display system 112 Sensor 165, and/or one or more tactile output generators 167 that tactile output is generated for the user for equipment 100.

Fig. 3 is the exemplary block diagram according to the multifunctional equipment with display and touch sensitive surface of some embodiments. Equipment 300 needs not be portable.In some embodiments, equipment 300 is laptop computer, desktop computer, plate electricity Brain, multimedia player device, navigation equipment, educational facilities (such as children for learning toy), game system or control equipment (example Such as, household controller or industrial controller).Equipment 300 generally include one or more processing units (CPU) 310, one or Multiple networks or other communication interfaces 360, memory 370 and one or more communication bus for making these component connections 320.Communication bus 320 optionally include make system unit interconnect and control system component between communication circuit (sometimes It is called chipset).Equipment 300 includes input/output (I/O) interface 330 with display 340, which is usually to touch Touch panel type display.I/O interface 330 also optionally include keyboard and/or mouse (or other sensing equipments) 350 and touch tablet 355, For generating the tactile output generator 357 of tactile output in equipment 300 (for example, being similar to above with reference to described in Figure 1A One or more tactile output generators 167), sensor 359 (for example, optical sensor, acceleration transducer, close to sensing Device, touch-sensitive sensors and/or strong similar to the contact above with reference to one or more contact strengths sensor 165 described in Figure 1A Spend sensor).Memory 370 includes high-speed random access memory such as DRAM, SRAM, DDR RAM or other arbitrary accesses Solid-state memory device, and optionally include nonvolatile memory such as one or more disk storage equipment, CD and deposit Store up equipment, flash memory device or other non-volatile solid-state memory devices.Memory 370 optionally include far from one or One or more storage equipment that multiple CPU 310 are positioned.In some embodiments, memory 370 is stored and is stored just Program, module in the memory 102 for taking formula multifunctional equipment 100 (Figure 1A) program similar with data structure, module and Data structure or their subgroup.In addition, memory 370 is optionally stored in the memory of portable multifunction device 100 Appendage, module and the data structure being not present in 102.For example, the memory 370 of equipment 300 optionally stores drawing mould Module 382, word processing module 384, website creation module 386, disk editor module 388, and/or electrical form is presented in block 380 Module 390, and the memory 102 of portable multifunction device 100 (Figure 1A) does not store these modules optionally.

Each element in above-mentioned identified element in Fig. 3 is optionally stored in one or more previously mentioned In memory devices.Each module in above-mentioned identified module corresponds to the instruction set for executing above-mentioned function.It is above-mentioned The module or program (that is, instruction set) identified need not be implemented as individual software program, process, or module, and therefore this Each seed group of a little modules is optionally combined in various embodiments or otherwise rearranges.In some embodiment party In case, memory 370 optionally stores the subset of above-mentioned module and data structure.In addition, memory 370 optionally stores above The add-on module and data structure not described.

It attention is drawn to the reality for the user interface (" UI ") optionally realized on portable multifunction device 100 Apply scheme.

Fig. 4 A shows the example of the application menu on the portable multifunction device 100 according to some embodiments Property user interface.Similar user interface is optionally realized in equipment 300.In some embodiments, user interface 400 is wrapped Include following element or its subset or superset：

One or more signal strengths for one or more wireless communication such as cellular signals and Wi-Fi signal refer to Show symbol 402；

Time 404；

Bluetooth indicator 405；

Battery status indicator 406；

Pallet 408 with the icon for common application program, these icons are such as：

The icon 416 for being marked as " phone " of zero phone module 138, the icon 416 optionally include missed call or The indicator 414 of the quantity of tone information；

The icon 418 for being marked as " mail " of zero email client module 140, the icon 418 optionally include The indicator 410 of the quantity of unread email；

The icon 420 for being marked as " browser " of zero browser module 147；And

Zero video and musical player module 152 (also referred to as iPod (trade mark of Apple Inc.) module 152) are marked It is denoted as the icon 422 of " iPod "；And

The icon of other applications, such as：

The icon 424 for being marked as " message " of zero IM module 141；

The icon 426 for being marked as " calendar " of zero calendaring module 148；

The icon 428 for being marked as " photo " of zero image management module 144；

The icon 430 for being marked as " camera " of zero camera model 143；

The icon 432 for being marked as " Online Video " of zero Online Video module 155；

The icon 434 for being marked as " stock market " of zero stock market desktop small routine 149-2；

The icon 436 for being marked as " map " of zero mapping module 154；

The icon 438 for being marked as " weather " of zero weather desktop small routine 149-1；

The icon 440 for being marked as " clock " of zero alarm clock desktop small routine 149-4；

The icon 442 for being marked as " news " of zero news template 142；

The icon 444 for being marked as " notepad " of zero notepad module 153；With

The icon 446 of zero setting application program or module, the icon 446 are provided to equipment 100 and its various application programs The access of 136 setting.

It should be pointed out that icon label shown in Fig. 4 A is only example.For example, in some embodiments, depending on The icon 422 of frequency and musical player module 152 is marked as " music " or " music player ".Other labels optionally for Various application icons.In some embodiments, the label of corresponding application programs icon includes and the corresponding application programs The title of the corresponding application program of icon.In some embodiments, the label of application-specific icon is different from and the spy Determine the title of the corresponding application program of application icon.

Fig. 4 B is shown with the touch sensitive surface 451 separated with display 450 (for example, plate or touch tablet in Fig. 3 355) the exemplary user interface in equipment (for example, equipment 300 in Fig. 3).Equipment 300 is also optionally included for detecting The intensity sensor of one or more contacts of the intensity of contact on touch sensitive surface 451 is (for example, one in sensor 359 Or multiple sensors), and/or one or more tactile output generators that tactile exports are generated for the user for equipment 300 359.Although by subsequent many is provided with reference to the input on touch-screen display 112 (being wherein combined with touch sensitive surface and display) Embodiment, but in some embodiments, the input on touch sensitive surface that equipment detection is separated with display, in Fig. 4 B It is shown.In some embodiments, touch sensitive surface (for example, 451 in Fig. 4 B) has and the master on display (for example, 450) The corresponding main shaft (for example, 452 in Fig. 4 B) of axis (for example, 453 in Fig. 4 B).According to these embodiments, equipment detection exists Position corresponding to the corresponding position on display is (for example, in figure 4b, 460, which correspond to 468 and 462, corresponds to 470) place The contact with touch sensitive surface 451 (for example, 460 in Fig. 4 B and 462).In this way, in touch sensitive surface (for example, 451 in Fig. 4 B) When separating with the display (for example, 450 in Fig. 4 B) of multifunctional equipment, the user detected on touch sensitive surface by equipment is defeated Enter (for example, contact 460 and 462 and its movement) to be used to manipulate the user interface on display by the equipment.It should be appreciated that similar Method is optionally for other users interface as described herein.

In addition, though inputting mostly in reference to finger (for example, gesture is gently swept in finger contact, finger Flick gesture, finger Deng) provide following embodiment, but it is to be understood that in some embodiments, one in the input of these fingers or Multiple finger inputs are by input (for example, input or stylus based on mouse input) substitution from another input equipment.For example, Gently sweep gesture and (for example, replace contact) optionally clicked by mouse, be cursor later along the moving of the path gently swept (for example, Instead of the movement of contact) it substitutes.As another example, position of the Flick gesture optionally by being located at Flick gesture in cursor Mouse when top is clicked and (for example, instead of the detection to contact, is off detection later and contacts) to substitute.Similarly, when same When detecting multiple users input, it should be understood that multiple computer mouses are optionally used simultaneously or mouse and finger connect Touching is optionally used simultaneously.

As used herein, term " focus selector " is the current of the user interface that instruction user is just interacting Partial input element.In some specific implementations for including cursor or other positions label, cursor serves as " focus selector ", So that being touched when cursor is above particular user interface element (for example, button, window, sliding block or other users interface element) Input (for example, pressing input) is detected on sensitive surfaces (for example, touch sensitive surface 451 in touch tablet 355 or Fig. 4 B in Fig. 3) In the case where, the particular user interface element is adjusted according to the input detected.It is including on realization and touch-screen display User interface element directly interactive touch-screen display (for example, in touch-sensitive display system 112 or Fig. 4 A in Figure 1A Touch screen) some specific implementations in, " focus selector " is served as in detected contact on the touchscreen, so that when touch It touches on panel type display at the position of particular user interface element (for example, button, window, sliding block or other users interface element) When detecting input (for example, the pressing input for passing through contact), the particular user interface is adjusted according to detected input Element.In some specific implementations, focus is moved to another region of user interface from a region of user interface, and nothing Light requirement target corresponds to the movement of the contact on movement or touch-screen display (for example, will be burnt by using Tab key or arrow key O'clock another button is moved to from a button)；In these specific implementations, focus selector is according to the not same district of user interface Focus between domain is mobile and moves.Do not consider the concrete form that focus selector is taken, focus selector be usually by with Family controls to transmit the interaction intended with the user of user interface (for example, by beating to the user of equipment indicative user interface The element interacted) user interface element (or contact on touch-screen display).For example, in touch sensitive surface When detecting pressing input on (for example, touch tablet or touch screen), focus selector (for example, cursor, contact or choice box) exists Position above the corresponding button will indicate that user it is expected to activate the corresponding button (rather than other users shown on device display Interface element).

As used in the present specification and claims, " intensity " of the contact on term touch sensitive surface refers to touching The power or pressure (power of per unit area) of contact (for example, finger contact or stylus contact) on sensitive surfaces, or refer to touch-sensitive The power of contact on surface or the substitute (surrogate) of pressure.The intensity of contact has value range, which includes at least Four different values and more typically include a different values up to a hundred (for example, at least 256).The intensity of contact optionally makes (or measurement) is determined with the combination of various methods and various sensors or sensor.For example, below touch sensitive surface or adjacent In touch sensitive surface one or more force snesors optionally for measurement touch sensitive surface on difference at power.In some tools During body is implemented, the power measured value from multiple force sensors is merged (for example, weighted average or adduction), to determine estimation The power of contact.Similarly, pressure of the pressure-sensitive top of stylus optionally for determining stylus on touch sensitive surface.As other one The capacitor of touch sensitive surface near kind selection, the size of the contact area detected on touch sensitive surface and/or its variation, contact And/or its resistance for changing and/or contacting neighbouring touch sensitive surface and/or its variation are optionally used as on touch sensitive surface Contact power or pressure substitute.In some specific implementations, the substitution measured value of contact force or pressure is directly used in really Whether fixed be more than intensity threshold (for example, intensity threshold is described with corresponding with measured value is substituted unit).Some In specific implementation, the substitution measured value of contact force or pressure is converted to and estimates power or pressure, and power or pressure are estimated in use To determine whether being more than intensity threshold (for example, intensity threshold is the pressure threshold measured with pressure unit).Use contact The attribute that intensity is inputted as user, so that user be allowed to access user for showing (for example, on the touch sensitive display) User's input can be indicated and/or received (for example, such as revolving via touch-sensitive display, touch sensitive surface or physical control/machinery control Button or button) limited area on the spot smaller equipment on the optional equipment function that cannot easily access originally.

In some embodiments, contact/motion module 130 determines user using one group of one or more intensity threshold Whether operation (for example, to determine whether user on icon carried out " click ") has been executed.In some embodiments, root Determine at least one subset of intensity threshold (for example, intensity threshold is not swashing by specific physical actuation device according to software parameter Living threshold value determines, and can be conditioned in the case where not changing the physical hardware of equipment 100).For example, can be by Trackpad Or to be set as any one of large-scale predefined threshold value predefined for mouse " click " threshold value of touch-screen display Threshold value, without changing Trackpad or touch screen viewing hardware.In addition, in some embodiments, the user of equipment is provided There is software setting, for adjusting one or more of this group of intensity threshold (for example, by adjusting individual intensity threshold And/or by adjusting multiple intensity thresholds simultaneously using system-level click " intensity " parameter).

As used in the specification and in the claims, the term " characteristic strength " of contact refers to based on one or more The contact characteristic of the intensity of a contact.In some embodiments, characteristic strength is based on multiple intensity samples.Property strengths are optional Ground is based on (for example, after detecting contact, before detecting that contact is lifted, detecting relative to predefined event Before or after contact start is mobile, before detecting that contact terminates, before or after detecting that the intensity of contact increases And/or detect contact intensity reduce before or after) in the predetermined period (for example, 0.05 second, 0.1 Second, 0.2 second, 0.5 second, 1 second, 2 seconds, 5 seconds, 10 seconds) during acquire predefined quantity intensity sample or one group of intensity sample. The property strengths of contact are optionally based on one or more of the following terms：The maximum value of contact strength, contact strength it is equal 90% of value, half maximum value of contact strength, contact strength at preceding the 10% of the average strength, contact strength that be worth, contact Maximum value etc..In some embodiments, when determining property strengths using the duration of contact (for example, in property strengths When being the average value of the intensity of contact in time).In some embodiments, property strengths and one group are one or more strong Degree threshold value is compared, to determine whether user has executed operation.For example, the group one or more intensity threshold may include first Intensity threshold and the second intensity threshold.In this example, the contact that characteristic strength is less than first threshold leads to the first operation, special Sign intensity is more than the first intensity threshold but is less than the contact of the second intensity threshold and leads to the second operation, and characteristic strength is more than The contact of second threshold causes third to operate.In some embodiments, using characteristic strength and one or more intensity thresholds Between comparison come determine whether to execute one or more operations (such as, if execute respective selection or abandon executing corresponding Operation), rather than the first operation or the second operation are executed for determining.

In some embodiments, identify a part of gesture for determining characteristic strength.For example, touch sensitive surface can connect Receipts continuously gently sweep contact, this is continuously gently swept contact from initial position transition and reaches end position (such as drag gesture), At the end position, the intensity of contact increases.In this example, characteristic strength of the contact at end position can the company of being based only upon Continuous a part for gently sweeping contact, rather than entirely gently sweep contact (for example, only gently sweeping contact portion at end position).Some In embodiment, algorithm can be smoothed in the intensity application that the forward direction for the characteristic strength for determining contact gently sweeps gesture.For example, this is flat Cunningization algorithm optionally includes one or more of the following terms：It does not weight and gently sweeps average smoothing algorithm, triangle smoothing Algorithm, median filter smoothing algorithm and/or exponential smoothing algorithm.In some cases, in order to determine characteristic strength Purpose, these smoothing algorithms eliminate narrow spike or the decline gently swept in the intensity of contact.

User interface map (for example, Fig. 6 A-6Y and Fig. 7 A-7G) described herein, which optionally includes, to be shown on touch sensitive surface Contact relative to one or more intensity thresholds (for example, contact detection intensity threshold value IT₀, light press intensity threshold IT_L, deep press Compressive Strength threshold value IT_D(for example, being at least initially higher than I_L)) and/or other one or more intensity thresholds (for example, being lower than I_HIt is strong Spend threshold value I_L) current intensity various intensity maps.The intensity map is not usually a part of shown user interface, still It is provided to help explain the figure.In some embodiments, light press intensity threshold corresponds to equipment and will execute usually and point Hit the button of physics mouse or the intensity of the associated operation of Trackpad.In some embodiments, deep pressing intensity threshold pair Intensity as Ying Yu：Under the intensity equipment will execute with usually with click physics mouse or Trackpad button it is associated Operate different operations.In some embodiments, when detect characteristic strength lower than light press intensity threshold (for example, and The Nominal contact detection intensity threshold value IT of contact is no longer detected higher than lower than it₀) contact when, which will be according to touch-sensitive table Contact movement on face carrys out moving focal point selector, related to light press intensity threshold or deep pressing intensity threshold without executing The operation of connection.In general, unless otherwise stated, these intensity thresholds are consistent between different groups of user interface map.

In some embodiments, equipment depends on based on connecing during input the response of input detected by equipment Touch the standard of intensity.For example, being inputted for some " light press ", more than the intensity of the contact of the first intensity threshold during input The first response of triggering.In some embodiments, equipment depends on the response of the input detected by equipment to include during inputting Contact strength and time-based both criteria standard.For example, for some " deep to press " inputs, only when satisfaction first Intensity threshold and meet between the second intensity threshold when passing by delay time, more than the second intensity threshold and big during input The second response is just triggered in the intensity of the contact of the first intensity threshold of light press.The duration of the delay time is usually less than (for example, 40ms, 100ms or 120ms, this depends on the magnitude of the second intensity threshold to 200ms, and wherein delay time is with second Intensity threshold increases and increases).This delay time helps to avoid unexpected deep pressing input.As another example, for Some " deep pressing " inputs, will appear the period of susceptibility reduction after meeting the first intensity threshold.Subtract in the susceptibility During the small period, the second intensity threshold increases.This temporary increase of second intensity threshold also helps to avoid unexpected Deep pressing input.For other pressing inputs deeply, time-based standard is not dependent on to the deep response for pressing input detection.

In some embodiments, one or more of input intensity threshold value and/or corresponding output are based on one or more A factor and change, such as user setting, contact movement, incoming timing, application program operation, the rate for applying intensity, concurrent Quantity, user's history, environmental factor (for example, ambient noise), focus selector position of input etc..In U.S. Patent application sequence Example factors are described in row number 14/399,606 and 14/624,296, the full content of the patent application is by reference simultaneously Enter herein.

For example, Fig. 4 C shows the intensity for being based in part on touch input at any time 476 and the dynamic changed over time Intensity threshold 480.Resistance to vibration threshold value 480 be following two component and：First component 474, is arrived since initial detecting Decay at any time after the predefined delay time p1 of touch input 476；And second component 478, tracking touching at any time Touch the intensity of input 476.The initial high-intensitive threshold value of first component 474 reduces the unexpected triggering of " deep pressing " response, while such as Fruit touch input 476 provides enough intensity, then still allows for " deep pressing " response immediately.Second component 478 passes through touch Gradually intensity fluctuation in input reduces the inadvertent free of " deep pressing " response.In some embodiments, work as touch input 476 when meeting resistance to vibration threshold value 480 (for example, point 481 at) in figure 4 c, triggering " deep pressing " response.

Fig. 4 D shows another resistance to vibration threshold value 486 (for example, intensity threshold I_D).Fig. 4 D also show two other Intensity threshold：First intensity threshold I_HWith the second intensity threshold I_H.In fig. 4d, although touch input 484 is full before time p2 The first intensity threshold I of foot_HWith the second intensity threshold I_L, but response is just provided until having pass by delay time p2 in the time 482.Together In fig. 4d, resistance to vibration threshold value 486 decays sample at any time, wherein from the time 482 through past predefined delay time p1 Later, which starts in the time 488 (when triggering and the second intensity threshold I_LWhen associated response).Such dynamic Intensity threshold reduces：And then response associated with lower intensity threshold is triggered or in triggering and lower intensity threshold phase While associated response, unexpected triggering and resistance to vibration threshold value I_DAssociated response, the lower intensity threshold such as first Intensity threshold I_HOr the second intensity threshold I_H。

Fig. 4 E shows another resistance to vibration threshold value 492 (for example, intensity threshold I_D).In Fig. 4 E, since initial After detecting that touch input 490 passes by delay time p2, triggering and intensity threshold I_LAssociated response.Meanwhile certainly From initial detecting to touch input 490 pass by predefined delay time p1 after, resistance to vibration threshold value 492 decay.Therefore, Even if when the intensity of touch input 490 is lower than another intensity threshold (for example, intensity threshold I_L) when, in triggering and intensity threshold I_LThe intensity of touch input 490 reduces after associated response, and the intensity of subsequent touch input 490 increases, without release Touch input 490 can trigger and intensity threshold I_DAssociated response (for example, in time 494).

The characteristic strength of contact is from lower than light press intensity threshold IT_LIntensity increase between light press intensity threshold IT_L Intensity threshold IT is pressed with deep_DBetween intensity be sometimes referred to as " light press " input.The characteristic strength of contact is pressed from lower than deep Compressive Strength threshold value IT_DIntensity to being higher than deep pressing intensity threshold IT_DThe increase of intensity be sometimes referred to as " deep pressing " input. The characteristic strength of contact is from lower than contact detection intensity threshold value IT₀Intensity increase between contact detection intensity threshold value IT₀With it is light Press intensity threshold IT_LBetween intensity be sometimes referred to as and detect contact on touch-surface.The characteristic strength of contact is from height In contact detection intensity threshold value IT₀Intensity be decreased below contact detection intensity threshold value IT₀Intensity sometimes referred to as detect Contact is lifted from touch-surface.In some embodiments, IT₀It is zero.In some embodiments, IT₀Greater than zero.Some In diagram, shade circle or oval for indicating the intensity of the contact on touch sensitive surface.In some diagrams, shadeless circle or Ellipse is for indicating the corresponding contact on touch sensitive surface without the specified intensity accordingly contacted.

Herein in some embodiments, in response to detecting gesture or response including corresponding pressing input One or more operations are executed in detecting using the corresponding pressing input of corresponding contact (or multiple contacts) execution, wherein extremely It is at least partly based on and detects that the intensity of the contact (or multiple contacts) increases to above pressing input intensity threshold value and detects Corresponding pressing input.In some embodiments, in response to detecting that it is strong that the intensity accordingly contacted increases to above pressing input Threshold value is spent to execute corresponding operating (for example, executing corresponding operating on " downward stroke " of corresponding pressing input).In some realities It applies in scheme, pressing input includes that the intensity accordingly contacted increases to above pressing input intensity threshold value and the intensity of the contact Be decreased subsequently to lower than pressing input intensity threshold value, and in response to detect the intensity accordingly contacted be decreased subsequently to lower than by Pressure inputs threshold value to execute corresponding operating (for example, executing corresponding operating on " up stroke " of corresponding pressing input).

In some embodiments, equipment is lagged using intensity to avoid the accident input sometimes referred to as " shaken ", wherein Equipment limits or selection has the lag intensity threshold of predefined relationship (for example, lag intensity with pressing input intensity threshold value Threshold value than the low X volume unit of pressing input intensity threshold value, or lag intensity threshold be pressing input intensity threshold value 75%, 90% or some rational proportion).Therefore, in some embodiments, pressing input includes that the intensity accordingly contacted increases to height It is decreased subsequently in the intensity of pressing input intensity threshold value and the contact lower than the lag for corresponding to pressing input intensity threshold value Intensity threshold, and in response to detecting that the intensity accordingly contacted is decreased subsequently to execute corresponding behaviour lower than lag intensity threshold Make (for example, executing corresponding operating on " up stroke " of corresponding pressing input).Similarly, in some embodiments, only It detects that the intensity of contact is increased to from the intensity for being equal to or less than lag intensity threshold in equipment and is equal to or higher than pressing input The intensity of intensity threshold and the intensity optionally contacted are decreased subsequently to be equal to or less than just to detect when the intensity of lag intensity It is inputted to pressing, and (for example, according to environment, the intensity of contact increases or the intensity of contact in response to detecting pressing input Reduce) execute corresponding operating.

In order to be easy to explain, optionally, in response to detect following situations and trigger in response to pressing input intensity The description of the associated pressing input of threshold value or the operation executed in response to including the gesture of pressing input：The intensity of contact increases It is big to increase to above pressing to higher than pressing input intensity threshold value, the intensity of contact from the intensity lower than lag intensity threshold and input The intensity of intensity threshold, the intensity of contact be decreased below pressing input intensity threshold value or contact intensity be decreased below with Press the corresponding lag intensity threshold of input intensity threshold value.In addition, describing the operations as in response to the intensity for detecting contact Pressing input intensity threshold value is decreased below in the example that executes, is optionally in response to detect that the intensity of contact is reduced to low Operation is executed in the lag intensity threshold for corresponding to and be less than pressing input intensity threshold value.As described above, in some implementations In scheme, the triggering of these responses is also depended on and meets time-based standard (for example, in the first intensity threshold of satisfaction and completely Pass by delay time between the second intensity threshold of foot).

Fig. 5 shows the example electronic device communicated with display 450 and touch sensitive surface 451.Electronics is set At least one standby subset, according to some embodiments, display 450 and/or touch sensitive surface 451 are integrated into electronic equipment In.Although with reference to electronic equipment (for example, the portable multifunction device 100 in Figure 1A -1B or equipment 300 in Fig. 3) into The touch sensitive surface 451 and display 450 of row communication describes the example of following more detail, it is to be understood that, according to some Embodiment, touch sensitive surface and/or display and electronic equipment are integrated, and in other embodiments, touch sensitive surface and display One or more of device is separated with electronic equipment.In addition, in some embodiments, electronic equipment has integrated display And/or integrated touch sensitive surface, and carried out with the one or more additional displays and/or touch sensitive surface separated with electronic equipment Communication.

In some embodiments, to user interface navigation logical block 483 Single Electron equipment (for example, with The lower calculating equipment A described with reference to Fig. 5) execute all operations described below with reference to Fig. 6 A-6Y and 7A-7G.However, should manage Solution, usually links together multiple and different electronic equipments, to execute the behaviour below with reference to Fig. 6 A-6Y and 7A-7G description Make (for example, electronic equipment with user interface navigation logical block 483 and the independent electronics with display 450 and/ Or the independent electronics with touch sensitive surface 451 are communicated).Any one embodiment in these embodiments In, the electronic equipment below with reference to Fig. 6 A-6Y and 7A-7G description is that the electronics comprising user interface navigation logical block 483 is set Standby (or multiple equipment).In addition, it will be appreciated that in various embodiments, user interface navigation logical block 483 can be multiple It is divided between different modules or electronic equipment；However, for the purpose of description herein, user interface navigation logic section Part 483 will refer mainly to reside in Single Electron equipment, to avoid other aspects for unnecessarily obscuring embodiment.

In some embodiments, user interface navigation logical block 483 includes one or more modules (for example, one Or multiple event handlers 190 comprising one or more object renovators 177 and one or more GUI renovator 178, such as Be more fully described above with reference to Figure 1B), the one or more module receive explain input and it is defeated in response to these explanations Enter to generate for updating the instruction of graphic user interface according to input is explained, which is used subsequently to update on display Graphic user interface.In some embodiments, explain that input is detected (for example, by Figure 1A -1B contact in 3 Movement 130), identification (for example, passing through event recognizer 180 in Figure 1B), and/or be prioritized (for example, by the thing in Figure 1B Part classifier 170) input.In some embodiments, explain that input is generated by the module of electronic equipment (for example, electronics Equipment receives original contact input data to identify gesture from original contact input data).In some embodiments, some Or all explanation inputs by electronic equipment as explain input and receive (e.g., including at the electronic equipment of touch sensitive surface 451 Original contact input data is managed to identify gesture from original contact input data, and sends packet for the information for indicating gesture Include the electronic equipment of user interface navigation logical block 483).

In some embodiments, both display 450 and touch sensitive surface 451 with include user interface navigation logical block 483 electronic equipment (for example, calculating equipment A in Fig. 5) is integrated.For example, electronic equipment can be with integrated display (example Such as, 340 in Fig. 3) and touch tablet (for example, 355 in Fig. 3) desktop computer or laptop computer.As another Example, electronic equipment can be the portable multifunction device 100 with touch screen (for example, 112 in Fig. 2) (for example, intelligence Phone, PDA, tablet computer etc.).

In some embodiments, touch sensitive surface 451 and electronic equipment are integrated, and display 450 not with include user circle The electronic equipment (for example, calculating equipment B in Fig. 5) of face navigational logic component 483 is integrated.For example, electronic equipment can be tool Have and (pass through wired connection or wireless connection) and is connected to the integrated of independent display (for example, computer, monitor, television set etc.) The equipment 300 (for example, desktop computer or laptop computer) of touch tablet (for example, 355 in Fig. 3).As another example, Electronic equipment can be have (passing through wired connection or wireless connection) be connected to independent display (for example, computer monitor, Television set etc.) touch screen (for example, 112 in Fig. 2) portable multifunction device 100 (for example, smart phone, PDA, flat Plate computer etc.).

In some embodiments, display 450 and electronic equipment are integrated, and touch sensitive surface 451 not with include user circle The electronic equipment (for example, calculating equipment C in Fig. 5) of face navigational logic component 483 is integrated.For example, electronic equipment can be tool Have and (pass through wired connection or wireless connection) and is connected to independent touch sensitive surface (for example, remote touch plate, portable multifunction device Deng) integrated display (for example, 340 in Fig. 3) equipment 300 (for example, desktop computer, laptop computer, have collection At the television set of set-top box).As another example, electronic equipment, which can be, has (passing through wired connection or wireless connection) even Be connected to independent touch sensitive surface (for example, remote touch plate, with the touch screen for being used as remote touch plate another is portable Multifunctional equipment etc.) touch screen (for example, 112 in Fig. 2) portable multifunction device 100 (for example, smart phone, PDA, tablet computer etc.).

In some embodiments, both display 450 and touch sensitive surface 451 not with include user interface navigation logic section The electronic equipment (for example, calculating equipment D in Fig. 5) of part 483 is integrated.For example, electronic equipment can be (by wired connection or It is wirelessly connected) it is connected to independent touch sensitive surface (for example, remote touch plate, portable multifunction device etc.) and independent display The independent electronics 300 of (for example, computer monitor, television set etc.) are (for example, desktop computer, laptop computer, control Platform processed, set-top box etc.).As another example, electronic equipment, which can be, has (passing through wired connection or wireless connection) connection To independent touch sensitive surface (for example, remote touch plate, with the touch screen for being used as remote touch plate another is portable more Function device etc.) touch screen (for example, 112 in Fig. 2) portable multifunction device 100 (for example, smart phone, PDA, Tablet computer etc.).

In some embodiments, calculating equipment has integrated audio system.In some embodiments, calculate equipment with Separated audio system is communicated with the calculating equipment.In some embodiments, audio system is (for example, be integrated in TV Audio system in machine unit) it is integrated with independent display 450.In some embodiments, audio system is (for example, stereo System) it is the autonomous system separated with calculating equipment and display 450.

User interface and associated process

Pay attention to can be used now and is communicated with display and touch sensitive surface and/or realized including its electronic equipment The embodiment of user interface (" UI ") and associated process, the electronic equipment such as calculate equipment (for example, the meter in Fig. 5 Calculate a calculating equipment in equipment A-D).In some embodiments, which includes for detection and touch sensitive surface Contact intensity one or more sensors.In some embodiments, which includes display.In some realities It applies in scheme, which includes audio system.In some embodiments, the calculating equipment neither include display nor Including audio system.In some embodiments, display includes audio system (for example, display and audio system are TVs The component of machine).In some embodiments, certain components of audio system and display are separated (for example, display is electricity Depending on the component of machine, and audio system includes the bar speaker separated with television set).In some embodiments, equipment is calculated It is communicated with independent remote controler, which receives user's input (for example, remote controler includes that user passes through it by the remote controler The touch sensitive surface or touch screen interacted with calculating equipment).In some embodiments, remote controler includes for detecting remote control Device moves the motion sensor (for example, accelerometer and/or gyroscope) of (for example, user picks up remote controler).

Although by some examples below are provided with reference to the input on the touch sensitive surface 451 separated with display 450, It is that in some embodiments, as shown in Figure 4 A, equipment detects input (wherein touch sensitive surface and display on touch-screen display Device combination).For ease of description, reference pair there is the operation of the equipment execution of touch-sensitive display system 112 some realities to be discussed Apply scheme.In such embodiment, focus selector is optionally：Corresponding finger or stylus contact and finger or stylus Contact is corresponding to represent point (for example, the centroid of corresponding contacts or point associated with corresponding contacts), or in touch-sensitive display system The centroid of two or more contacts detected on 112.However, alternatively, shown in figure on display display 450 When user interface in response to detect contact on touch sensitive surface 451 and to display 450 and independent touch sensitive surface 451 with And the equipment of focus selector executes similar operation.

Fig. 6 A-6Y shows the exemplary use of the presentation for dynamically adjusting audio output according to some embodiments Family interface.More specifically, Fig. 6 A-6Y shows based on the magnitude of user's input the volume for adjusting audio content and/or non-sound The user interface of flow characteristic.For example, user can be performed for requesting to provide other audio contents when playing song in equipment Gesture.For example, song can be played in music application, and user can be in same music application program or different application Request another audio content (for example, another audio content can be in instant message application by another in program The audio-frequency unit for the video that a user sends).As explained in further detail below, the fuzzy of audio content includes dynamically adjusting The non-volume characteristic of first audio content (for example, the song played in equipment before user's input) is (for example, stereo flat Weighing apparatus).

Fig. 6 A-6H shows following example：Equipment (for example, for simplicity and referred to as equipment 100) is in music application When First song or album being presented in program, the second song in music application according to the length for gently sweeping gesture Or it is obscured in album.

Fig. 6 A shows the music application user interface 602 shown on the touch screen 112 of equipment 100 (in order to regard For the sake of feeling clear, the remainder other than touch screen 112 of equipment 100 is not shown).Music application user interface 602 include the title 604 that the Moonlight Sonata of Beethovan is currently played in indicating equipment 100.That is, equipment 100 is to sound Display system provides acoustic information 606-1, so that Moonlight Sonata (or the moon from Beethovan of Beethovan can be presented in audio system Track/song of light sonata).As writing a Chinese character in simplified form, phrase " audio content is presented in equipment 100 " etc. is for indicating equipment 100 to sound Display system provides acoustic information, so that audio output corresponding with audio content can be presented in audio system.As explained with reference to figure 5 , audio system can be integrated into equipment 100 or separate with equipment 100.

As shown schematically in sonagram 608, the Moonlight Sonata of Beethovan is indicated by acoustic information 606-1.Sound letter Breath 606-1 includes volume (being indicated by position of the acoustic information 606-1 on the vertical axis of audio chart 608) and removes volume Except acoustic characteristic, in this example the acoustic characteristic be left-right stereo balance (by acoustic information 606-1 in sonagram " the L-R balance " that position on 608 horizontal axis indicates).In some embodiments, non-volume acoustic characteristic is that filtering is special Property (for example, cutoff frequency and/or decay factor of low-pass filter).

Music application user interface 602 further includes audio content expression 610 (for example, indicating that the moonlight of Beethovan is played The bent audio content of ring indicates 610-1；Indicate that the audio content of the requiem of Mozart indicates 610-2；And indicate Bo Lamu The audio content of this alto rhapsody indicates 610-3).Audio content indicates that 610 be the use respectively occupied on touch screen 112 The Drawing Object (also referred to as " graphical user interface object ") of the corresponding region at family interface 602.In all cases, audio content Indicating 610 indicates：Song, album, the tinkle of bells, video content object are (for example, video content occurs in text instant message window Position), the audio file object that occurs in text instant message window or any other type including audio frequency component Media content.In this example, audio content indicate 610 be album expression and including shown album cover.

The exemplary beginning shown in Fig. 6 A-6H, L-R balance is balanced, and other than acoustic information 606-1, Equipment 100 does not provide any other acoustic information.

As shown in Figure 6B, when the Moonlight Sonata of Beethovan is being presented in audio system, equipment 100, which receives to correspond to, is in The input 612 (for example, gently sweeping gesture) of the request of the requiem of existing Mozart.In this example, because by with Mozart's The corresponding audio content of requiem indicates 610-2 to receive input 612, so input 612 is the peace soul of Mozart for rendering Bent request.

In some embodiments, in response to the initial part of input 612 (for example, the slight movement for gently sweeping gesture), if For 100 hints (for example, acoustically and/or visually), it will start fuzzy audio.In some embodiments, as made herein , term " fuzzy audio ", which refers to, changes acoustic characteristic to change high-lighting of the fuzzy audio in its broadcasting, so that User can better discriminate between other audios played simultaneously with fuzzy audio.When audio " fuzzy ", " obscuring gradually in audio Enter " correspond to the high-lighting for increasing fuzzy audio (for example, passing through the cutoff frequency for increasing low-pass filter and/or making audio court Deviated to center channel), and " fuzzy gradually to go out " corresponds to the high-lighting for reducing fuzzy audio (for example, by reducing low-pass filtering The cutoff frequency of device and/or audio is removed from center channel).In some embodiments, the acoustic characteristic packet being fade-in is obscured Include volume.In some embodiments, obscure the acoustic characteristic that is fade-in include one or more non-volume acoustic characteristics (for example, The cutoff frequency or left/right of low-pass filter balance).For example, equipment 100 provides auditory cues, i.e., by using the first of input 612 The magnitude (for example, the slight movement for gently sweeping gesture) of initial portion dynamically adjusts the L-R balance of acoustic information 606-1, and audio will Start to obscure.In this example, the L-R of acoustic information 606-1 balances slight shift to the left.Another as auditory cues shows Example, the volume down of information volume 606-1.In some embodiments, equipment 100 also indicates 610- by increasing audio content The vision high-lighting of 2 (it is in fig. 6b by indicating that the runic frame of 610-2 is schematically shown around audio content) provides Visual cues.In some embodiments, increasing vision high-lighting includes visually obscuring in addition to audio content indicates 610-2 Other audio contents indicate (and optionally equally visually obscure user interface rest part).In some embodiments In, as described below, these prompts acoustically " fuzzy to be fade-in " are provided before in the requiem of Mozart.

In some embodiments, visual effect (for example, blur radius) and the non-volume acoustic characteristic dynamically adjusted Change in combination (for example, with extension is gently swept, user interface 602 (in addition to audio content indicates 610-2) passes through offset sound The L-R balance of information 606-1 is fuzzy with lockstep).

As shown in Fig. 6 C-6D, equipment 100 provides acoustic information 606-2 to audio system, to play with the moonlight of Beethovan The bent requiem that Mozart is presented simultaneously of ring.It in some embodiments, is more than predetermined in response to the magnitude of input 612 Threshold value (for example, it is more than small predetermined quantities that this, which is gently swept mobile), completes " fuzzy to be fade-in " of the second audio content.In some realities It applies in scheme, the requiem of Mozart is statically presented in equipment 100；That is, once the requiem of Mozart starts to play, just with Fixed volume and fixed L-R balance (for example, balanced L-R balance) to carry out.Alternatively, as shown, also basis 612 magnitude is inputted to dynamically adjust the presentation of the requiem of Mozart.For example, when user further moves to left inputting 612 When dynamic, the L-R balance of the requiem of Mozart deviates (the acoustic information 606- in sonagram 608 in such as Fig. 6 C-6D more to the left Shown in 2 offset to the left).Continue the presentation that the Moonlight Sonata of Beethovan is dynamically adjusted according to gesture is gently swept.For example, When user, which will input 612, further to be moved left, the L-R balance of the Moonlight Sonata of Beethovan deviates (such as Fig. 6 C-6D more to the left In sonagram 608 in acoustic information 606-1 offset to the left shown in).This gives user or less and feels：The moon of Beethovan Light sonata is deviated from arena theatre to the left, and the requiem of Mozart is displaced on arena theatre from right side, To occupy the position of Moonlight Sonata.

In some embodiments, equipment 100 dynamically adjusts the moonlight of Beethovan also according to the magnitude for gently sweeping gesture (for example, when input 612 is moved farther leftward as, equipment 100 reduces volume to the volume of sonata, such as the audio in Fig. 6 C-6D In Figure 60 8 acoustic information 606-1 offset downward it is represented).In some embodiments, equipment 100 sweeps hand also according to light The size of gesture come dynamically adjust the requiem of Mozart volume (for example, when input 612 is moved farther leftward as, equipment 100 increase the volume of the requiem of Mozarts, as acoustic information 606-2 offsets up institute in the sonagram 608 in Fig. 6 C-6D It indicates).In some embodiments, the volume of the requiem of the Moonlight Sonata and Mozart of Beethovan generally with display The expression 610-1 that is shown respectively in the predefined region (for example, central area of whole display or display) of device and The score of 610-2 is proportional.Therefore, when audio content is indicated that 610-2 is drawn onto the center of display by user, Mozart's Requiem become it is acoustically more prominent and acoustically it is more placed in the middle.

The equipment 100 that Fig. 6 E shows detects the end (for example, user's input 612 has been released) of user's input 612.Cause This, no longer shows user's input 612 in Fig. 6 E.

In some embodiments, in response to detecting that user inputs 612 end, equipment 100 is based on determining input 612 Magnitude be more than (or being more than) predetermined threshold value come determine whether to continue present the first audio content (for example, and stopping The second audio content is only presented), or vice versa.

Fig. 6 F is shown in this example, because input 612 has dragged audio content and indicated 610-2, so that it compares audio Content representation 610-1 is more shown, so the release of input 612 is so that audio content indicates that 610-2 is moved in screen Between position, wherein equipment 100 only shows the requiem of Mozart.Accordingly, in response to the end for detecting input 612, equipment 100 stop that the first audio content (for example, Moonlight Sonata) is presented and continue that the second audio content (peace soul of Mozart is presented It is bent).In some embodiments, equipment 100 completes the dynamic adjustment to the requiem of Mozart (for example, in release input 612 Later, L-R balance and volume were gradually changed within 0.5 second period, so that balancedly presenting not with the volume that pre-sets The requiem of Zha Te).This is deviated by the upward of the acoustic information 606-2 in the sonagram 608 in Fig. 6 E-6F and to the left to show Show to meaning property.In some embodiments, in response to detecting the end of input 612, visual effect (for example, fuzzy) also by Reversion.

If Fig. 6 G-6H, which shows input 612, which does not drag audio content, indicates 610-2 so that it is indicated than audio content 610-1 is more shown what will then occur in this example.Therefore, Fig. 6 G is shown immediately in release ratio input The state of user interface 602 after 612 inputs that are shorter but being similar to input 612.In response, as shown in figure 6h, equipment 100 stop presenting the requiem of (for example, audio system is caused to stop presenting) Mozart, and restart that Beethovan is only presented Moonlight Sonata.Then the Moonlight Sonata that Beethovan is presented, without being dynamically adjusted to it (for example, at 0.5 second The change of interior reversion L-R balance and volume, so that balancedly and with the volume pre-seted being played the moonlight of Beethovan is presented Ring is bent).This is deviated by the upward of the acoustic information 606-1 in the sonagram 608 in Fig. 6 G-6H and to the left schematically to show Out.In some embodiments, in response to detecting the end of input, visual effect (for example, fuzzy) is also inverted.

In some embodiments, when input 612 magnitude meet be greater than the first predetermined threshold value second in advance When determining threshold value, equipment 100 visually and acoustically by the second audio output " pop-up " to position appropriate (for example, depositing In first threshold, wherein if release input, the second audio output are moved to appropriate location, and there are second threshold, In even release input before, the second audio output is ejected into position appropriate).

In some embodiments, as exemplary alternative solution shown in Fig. 6 E-6H, in response to detecting input 612 End, equipment 100 continue present adjustment the first audio output and continue present the second audio output.

Fig. 6 I-6O shows following example：Equipment (for example, for simplicity and referred to as equipment 100) is in music application When song or album being presented in program, according to the intensity of pressing input in instant message application (for example, and music application The different application program of program) in the video for being sent to user audio-frequency unit in obscure.It is exemplary each shown in Fig. 6 I-6O A aspect is similar to example shown in Fig. 6 A-6H.These details are not repeated herein.Example shown in Fig. 6 I-6O and Fig. 6 A-6H institute The difference between example shown is：In Fig. 6 I-6O, input is pressing input, and magnitude is the intensity of pressing input；And In addition, equipment 100 is fuzzy to come from two different applications (for example, music application and instant message application in Fig. 6 I-6O Program) audio.

It is obscured in addition, Fig. 6 I-6O is shown by dynamically adjusting low pass filter cutoff frequency rather than L-R balancing Audio.For this purpose, Fig. 6 I-6O includes sonagram 613.The vertical axis of sonagram 613 indicates volume, and horizontal axis indicates low Pass filter cut off frequency.

Fig. 6 I shows the user interface 614 of the instant message application program for showing on touch screen 112.User circle Face 614 shows message 616, some message in the message 616 by equipment 100 user (for example, message 616-1 and 616-2) It receives, and some message in the message 616 send (for example, message 616-3) by the user of equipment 100.User interface 614 Head portrait 618 including the participant in instruction session is (for example, " Alex " and its head portrait that its head portrait is 618-1 are 618-2 " Tina ").For example, Tina sends the message 616-1 of the expression for video to Alex.

In Fig. 6 I, sonagram 613 includes the expression of acoustic information 606-3, to mean that audio is being presented in equipment Content.In this illustration, acoustic information 606-3 corresponds to the alto rhapsody of Bo Lamusi, therefore equipment 100 is logical first It crosses and is separated from instant message application program or different music applications is presented the alto rhapsody of Bo Lamusi.With spy Accordatura amount and the low pass filter cutoff frequency by being set as high level (hereinafter referred to " cutoff frequency ") provide vigorous drawing This alto rhapsody of nurse.In some embodiments, cutoff frequency when beginning is higher than the audible maximum frequency (example of people Such as, about 20kHz) so that when do not execute audio it is fuzzy when, low-pass filter does not have audible influence to audio output.One In a little embodiments, when starting (that is, when do not execute audio it is fuzzy when) do not apply low-pass filter.On the contrary, when audio is fuzzy When beginning, low-pass filter is opened.For simplicity, Fig. 6 I show by start when be set as the cutoff frequency of high level come The acoustic information 606-3 of presentation.

As shown in Fig. 6 J-6K, when alto rhapsody is presented in equipment 100, the reception of equipment 100 disappears with video for rendering The request of breath 616-1 (audio including video messaging 616-1) is corresponding to input 620 (for example, input is in touch screen 112 Input on region corresponding with video messaging 616-1).

In this example, input 620 is the pressing input (for example, press and keep inputting) on touch screen 112.Touch screen 112 have the one or more sensors of the intensity for detecting contact.The intensity of input 620 is indicated on intensity map 622. In Fig. 6 J, the intensity for inputting 620 is higher than prompting threshold value (for example, IT_H).Therefore, equipment 100 starts video messaging 616-1 Audio-frequency unit in the alto rhapsody for acoustically obscuring Bo Lamusi.That is, when the intensity of input 620 is higher than prompt threshold Value is (for example, IT_H) but be not above and pry through threshold value (for example, IT_L) when, equipment 100 reduces female when the intensity for inputting 620 increases The volume of bass rhapsody and reduce cutoff frequency.These effects are downward by the acoustic information 606-3's in sonagram 613 Offset is schematically shown to the left.The effect for reducing the cutoff frequency of low-pass filter is filter out alto rhapsody more next More high fdrequency components is to only be left its bass.

Meanwhile (acoustic information 606-4's such as corresponding with video messaging 616-1 is upward as the intensity of contact 620 increases Represented by arrow), the volume of video messaging 616-1 increases (for example, proportionally).Acoustic information 606-4 actually not by Low-pass filtering is (for example, low-pass filter is filtered by the cutoff frequency higher than human audible range to it or it is basic It is not low pass filtering).

In some embodiments, as long as the intensity of input 620 has been higher than prompting threshold value (for example, IT_H) but have not exceeded Threshold value is pried through (for example, IT_L), then the vision high-lighting of video messaging 616-1 dynamically changes (example with the intensity of input 620 Such as, the sense of hearing high-lighting of the vision high-lighting of video and corresponding audio increases with the increase of intensity, and with intensity Reduce and reduce).For example, the intensity for increasing input 620 increases the size of video messaging 616-1 as shown in Fig. 6 J-6K, Keep video messaging 616-1 placed in the middle, and is made in user interface 614 by the blur radius proportional to the intensity of input 620 Any message in addition to video messaging 616-1 is fuzzy, and (blur radius of increase in these figures is by removing video messaging 616-1 Except cover all user interfaces 614 the reduced transparency of pattern schematically show).

It can be seen that IT by comparing Fig. 6 J and Fig. 6 K_HAnd IT_LBetween increase intensity influence.In some embodiments In, user can repeatedly purify alto rhapsody and video messaging 616-1 by increasing and reducing the intensity of input 620 Volume (and cutoff frequency of alto rhapsody).For example, if Fig. 6 J and 6K are inverted, the sound of acoustic information 606-3 Amount and cutoff frequency will increase, and the volume of acoustic information 606-4 will reduce.

In this example, when the audio output of video messaging 616-1 is presented in equipment 100, equipment 100 is also presented video and disappears Cease the video output of 616-1.Therefore, Fig. 6 J-6O shows video progress (for example, video messaging 616-1 is that bird flies around frame The simple video of Xiang).

As shown in Fig. 6 L-6M, when the intensity for inputting 620 reaches " prying through " threshold value, equipment 100 passes through the cut-off pre-seted Bass rhapsody is presented in frequency and the first volume (for example, amount of bass) for pre-seting, and passes through the second volume pre-seted (for example, the higher volume of the volume pre-seted than first) is presented video messaging 616-1, and with the intensity of input 620 unrelated As long as (for example, the intensity of input 620 keeps below " pop-up " threshold value, being somebody's turn to do " pop-up " threshold value in this example is IT_D).For example, Equipment 100 detects that the intensity of input 620 increases above IT_L(Fig. 6 L), the intensity for subsequently inputting 620 decreases below IT_L(figure 6M), but acoustic information 606-3 and acoustic information 606-4 is maintained it in sonagram 613 and " pry through " (example at position Such as, even if the intensity of contact is decreased below IT_L, the vision high-lighting of video and the sense of hearing high-lighting for corresponding to audio are also maintained The rank shown at least 6K).Effect is that the audio from video messaging 616-1 is mainly presented, wherein only alto chimae Bent quiet bass background keeps audible.

In addition, vision changes locked after the intensity of input 620 reaches " prying through " threshold value.In Fig. 6 L, video disappears Breath 616-1 is centered about, and wherein the remainder of user interface 614 seriously obscures.When the intensity of input 620 is decreased below IT_LWhen, this fuzzy and video messaging 616-1 center is kept in Fig. 6 M.

However, in some embodiments, release of the input 620 at any point in Fig. 6 J-6M will lead to equipment 100 stop that video messaging 616-1 is presented and restore to present alto rhapsody completely (for example, the volume pre-seted with second And without filtering), and alternatively return to the state of the user interface shown in Fig. 6 I.

Fig. 6 N-6O is shown in some embodiments, when the intensity of input 620 is higher than pop-up threshold value (for example, IT_D) When, video messaging 616-1 " pop-up " is arrived position appropriate by equipment 100.Even if the intensity of input 620 is subsequently fallen below IT_DOr input terminal 620 is fully completed, once the intensity of input 620 is higher than deep pressing threshold value IT_D, the sound of video messaging 616-1 Amount is just maintained at the volume that second pre-sets and keeps not filtered (Fig. 6 O).In some embodiments, equipment 100 will regard Frequency message 616-1 be visually ejected into position appropriate (for example, video messaging 616-1 extension so as to occupy entire screen or Most of screen).In addition, equipment 100 stops that the first audio output (for example, alto rhapsody) is presented.

Fig. 6 P-6Y shows following example：According to some embodiments, since the characteristic strength of contact changes, equipment (example Such as, for simplicity and referred to as equipment 100) by dynamically change media item one or more acoustic characteristics set come Audio of the preview from media item.For example, at least for a certain range of contact strength, when equipment 100 receives the table of song When the contact shown, equipment 100 will play song with the volume proportional to the intensity of contact.In some embodiments, When contact reaches the first intensity threshold, even if the intensity of contact then declines, equipment 100 also will be pre- by the way that volume is maintained The rank of setting carrys out " prying through " previewing media item.In some embodiments, when contact reaches the second intensity threshold, equipment 100 will execute another operation (for example, other than previewing media item), such as execution selection operation.

Fig. 6 P-6Y also shows following example：In response to input, equipment 100 hides (for example, fuzzy) entire user interface (in addition to selected user interface object) or a part for only hiding user interface.In some embodiments, work as contact When meeting the activation standard based on intensity, masking user interface is executed (for example, entire user interface or only one of user interface Point).For example, intensity rises above prompting threshold value IT_HPressing and keep input to cause to cover, and Flick gesture causes to execute Different operations (for example, to selection operation of user interface object).

Fig. 6 P shows the user interface 640 for instant message application program.User interface 640 includes interaction area 642-1.Interaction area 642-1 be include that multiple message 644 between sessions participant (for example, Alex and Tina) are (therein Message 644-1, message 644-2 and message 644-3 are representative) conversation region.User can be by rolling through message 644 (for example, the position in conversation region is shown by scroll bar 646) or by interacting with each message 644 (for example, in message Tap presses (such as and keeping) to play video, or on message 644-3 " to pry through " video on 644-3, such as refers to Described in Fig. 6 I-6O) it interacts with interaction area 642-1.Interaction area 642-1 further include it is multiple show can indicate (for example, Icon 648-1, icon 648-2, icon 648-3 and icon 648-4, respectively can be realized specific functions of the equipments) and frame 650 (for example, user inputted but be not yet sent to the content of other sessions participants show region).

Fig. 6 P shows the input 652 (for example, Flick gesture) on icon 648-3.As shown in Fig. 6 Q, Flick gesture 652 So that user interface 640 shows interaction area 642-2 (for example, independent interaction area different from interaction area 642-1).One In a little embodiments, interaction area 642-1 is the user interface for primary application program (for example, instant message application program), And interaction area 642-2 is display configured to the content from the different miniature applications programs operated in primary application program. User can gently sweep between miniature applications program, to change the miniature applications program shown in interaction area 642-2.Interaction Region 642-2 further includes proposing showing for list of the miniature applications program that will be shown in interaction area 642-2 can indicate 659.Indicator 654 indicates currently show which miniature applications program (for example, gently sweeping will lead to the left in interaction area 642-2 Cause shows different miniature applications programs in the 642-2 of region, and the solid dot in indicator 654 is moved right).

In this example, miniature applications program is for selecting shared media in the session between sessions participant Media select miniature applications program 656.(for example, nearest 30 song that display plays on the appliance 100 rolls table It is single).Media selection miniature applications program includes the expression (for example, expression of media item 658-1 to 658-4) of media item 658.? In some embodiments, media item is song, album, video etc..Indicate 658-3 include related media item (for example, artist, Song title) and album cover picture display information.In some embodiments, the corresponding Flick gesture indicated on 658 is not ( Show) select accordingly to indicate 658 by placing it in frame 650 (for example, as audio message so that its be segmented with For being sent to session).In some embodiments, the corresponding Flick gesture (not shown) indicated on 658 is by playing media (for example, on the appliance 100 local play) selects to indicate 658 accordingly.

Fig. 6 R is shown including indicating the defeated of the contact on 658-2 (for example, indicate the pressing on 658-2 and keep gesture) Enter 660.As shown in intensity map 668, it is higher than prompting threshold value IT by having_HContact strength, press and hold gesture satisfaction Media cue standard (for example, activation standard based on intensity).Therefore, equipment 100 start play media item with indicate 658-2 It is corresponding a part (for example, beginning or be selected as comprising media item recognizable part representative part).In audio The broadcasting of media item is indicated in Figure 66 2 by audio output 664.Sonagram 662 shows the volume of media item and the time closes System.

As shown in Fig. 6 R-6S, when media item is playing, equipment 100 changes and dynamic with the characteristic strength of contact Ground changes the volume of media item (for example, the volume of media item increases as the characteristic strength of contact increases, and with contact Characteristic strength reduce and reduce).For example, increasing from the intensity of the contact of Fig. 6 R to Fig. 6 S (for example, from IT is just above_HIt arrives Just it is lower than IT_L) along with the increase of corresponding volume, (arrow in intensity map 668 in Fig. 6 R-6V is shown from the strong of preceding figure The change of degree；Similarly, the solid line in the sonagram 662 in Fig. 6 R-6U show the figure with the arrow pair in intensity map 668 The change answered, and dotted line indicates the change in preceding figure).If intensity is inverted (for example, if Fig. 6 S occurs before Fig. 6 R), Then the volume of media item will reduce.

In some embodiments, it when having generated different audio output before media item starts to play, such as refers to Described in Fig. 6 A-6O, method 800, method 840 and Fig. 8 A-8C, equipment 100 is fuzzy together by two audio output.

In addition, Fig. 6 R-6S is shown when the change of the volume of media item, equipment 100 dynamically changes interaction area 642-2 Appearance.For example, when the characteristic strength of contact changes, 100 dynamic hidden of equipment (for example, fuzzy) interaction area 642-2 (example Such as, with the lockstep of the change with audio, so that the characteristic strength with contact increases, blur radius also increases).Cause This, increases from the intensity of the contact of Fig. 6 R to Fig. 6 S and increases along with the corresponding of fuzziness (when equipment is moved to Fig. 6 S from Fig. 6 R When, the fuzziness of the increase in these figures by the other content in covering interaction area 642-2 pattern reduced transparency It schematically shows).In this example, 100 dynamic hidden interaction area 642-2 of equipment is without hiding interaction area 642-1 (for example, to indicate that the prompt is related to the miniature applications program part of user interface and is not related to entire user interface).

In addition, the expression 658-2 not being hidden dimensionally increases with the increase of contact strength and is moved to friendship The center of mutual region 642-2.In this example, as long as contact strength not up to " pries through " intensity threshold (for example, IT_L), user Can by the intensity of " prompt " range (for example, in IT_HWith IT_LBetween) change the intensity of contact and repeatedly purify view Feel to change and change with the sense of hearing.

Fig. 6 T-6U shows contact and reaches the result for prying through threshold value (for example, equipment 100 is detected higher than IT_LContact The result that characteristic strength increases).In this example, once the intensity of contact, which reaches, pries through threshold value, the volume of audio output 664 is just It is kept fixed (for example, being locked to the rank pre-seted, any volume that equipment 100 is arranged to by such as user).Therefore, IT_L With IT_DBetween the intensity increase of contact do not lead to the increase of volume.Similarly, it as shown in Fig. 6 U, subsequently detects lower than IT_L The reduction of characteristic strength of contact volume is not influenced (as long as example, continuous maintain contact；In certain embodiments, It is promoted and gradually or immediately stops the presentation of media item and visible change is inverted to interaction area 642-2).

In some embodiments, once contact reaches IT_L, equipment 100, which is just shown, indicates 661, that is, the feature contacted is strong Degree increases will be so that equipment executes selection operation (for example, language " 3D TOUCH TO SHARE ") relative to media item.It is lower than IT_LContact intensity it is subsequent reduce the masking for not modifying the rest part of interaction area 642-2, the size for indicating 658-2, Or " 3D TOUCH TO SHARE " instruction.

Fig. 6 V, which shows equipment 100 and detects, is higher than " pop-up " threshold value (for example, IT_D) contact characteristic strength increase. Therefore, equipment 100 stops playing media item (therefore, not having sonagram 662 in Fig. 6 V-6X) and executes choosing for media item Select operation (for example, media item is placed in frame 650 by equipment 100).

In some embodiments, as shown, detecting more than IT_DContact intensity after be immediately performed selection Operation.It is invalid that input 660 subsequently becomes.In other words, although input 660 is illustrated in original representation 658-2 in Fig. 6 V On, but input 660 does not influence original representation 658-2.User, which can pass through release input 660 and input new user, to be inputted To reselect original representation 658-2.

In Fig. 6 V, media item is indicated by audio message 644-4 now, and corresponding to indicates as shown in Fig. 6 Q-6U 658-2.In some embodiments, audio message 644-4 seems identical as expression 658-2 shown in Fig. 6 R.In addition, equipment 100 stop masking interaction area 642-2 (for example, reversion is fuzzy).

Place the content at also causes equipment 100 to show " transmission " button 670 in frame 650.Fig. 6 W shows selection and sends The user of button 670 inputs 672 (for example, Flick gestures), sends sessions participant for audio message 644-4 and causes Audio message 644-4 is added to conversation region, as shown in Fig. 6 X.

Fig. 6 Y is shown including the contact on audio message 644-4 (for example, pressing and guarantor on audio message 644-4 Hold gesture) input 674.Because the contact in 674 contact of input is higher than IT_H, so other than audio message 644-4, if Standby 100 hiding interaction area 642-1 and interaction area 642-2.In some embodiments, in response to being higher than IT_HContact Under pressing and keep inputting, equipment 100 also in a manner similar to that previously described mode (e.g., including dynamically change Audio) carry out preview audio message 644-4.In contrast, in response to Flick gesture (for example, to be lower than IT_HIntensity), equipment 100 execute selection operation (for example, playing entire audio message 644-4) relative to audio message 644-4, answer without masking The second interaction area with program and the first interaction area without covering application program.

Additional detail in relation to Fig. 6 A-6Y (and user interface shown in it) is provided below with reference to Fig. 8 A-8H.

Fig. 7 A to 7G is shown according to some embodiments for providing audio output based on audio configuration file Exemplary user interface.More specifically, Fig. 7 A-7G shows following example：Showing can indicate (for example, pressing on touch screen 112 Button) activation so that equipment output has audio configuration file (for example, management tone, reverberation, and/or audio output are at any time Decaying audio configuration file) sound.When activation second, which is shown, to be indicated, equipment determines whether to meet audio change mark Standard, and if it is satisfied, then modification is shown with first can indicate corresponding audio configuration file.For example, when energy table is shown in activation first Activation second shows that the equipment makes first to show that the sound that can be indicated quickly is decayed when can indicate shortly after showing.

User interface in these figures is for illustrating procedures described below, including the process in Fig. 9 A-9C.

For convenience's sake, Fig. 7 A-7G is described referring to equipment 100.As writing a Chinese character in simplified form, " equipment 100 presents and shows energy phrase Indicate corresponding audio " etc. for meaning that equipment 100 provides (or initiate provide) acoustic information to audio system so that sound Display system, which can be presented, can indicate corresponding audio output with showing.As explained with reference to figure 5, audio system can be integrated into equipment It is separated in 100 or with equipment 100.

More specifically, Fig. 7 A-7G shows following example：As the threshold time T for showing the activation that can be indicated first_thresIt is interior When activating second to show can indicate, meet audio change standard.

Fig. 7 A shows the user interface 702 for telephony application (for example, phone module 138).User interface is aobvious Show keypad 704, character can be for example input in dialed number column 706 by user by the keypad 704.For this purpose, small key Disk 704 include and number it is corresponding one group show and can indicate 708 (for example, it is corresponding with " 5 " show can indicate 708-1 and corresponding with " 2 " Show can indicate 708-2).

As shown in Figure 7 B, equipment 100 is in time T₁It detects and shows the can indicate 708-1 first input 710 for activating, and And " 5 " are placed in dialed number column 706.Fig. 7 B also shows the audio output provided at any time and indicates (for example, audio configuration File) sonagram 712.Detect time T when the first input 710₁It is labeled on the horizontal axis of audio chart 712. The volume for the audio output that the vertical axis instruction of audio figure 712 is shown on audio figure 712.First audio configuration file 714-1 be the first audio output with show that indicate 708-1 corresponding and in response to showing the table that can be indicated 708-1 and be presented Show.Show the first audio configuration file 714-1 of the first audio output completely in the sonagram 712 of Fig. 7 D.First audio is defeated Out in time T₁It steeply rises later, then finally decays to zero volume.Come in addition, sonagram 712 is indicated with c sharp tone Generate the first audio output.

Fig. 7 C shows equipment 100 in the second time T₂(for example, time after T1) detects that direction is shown and can indicate " 2 " (for example, detecting the activation of " 2 " button of dui, are placed into dialed number column 706 by the second input 716 of 708-2 In).In this example, when equipment 100 is detected to the activation that can indicate 708-2 is shown, the generation of equipment 100 is matched by the second audio Set the second sound output of file 714-2 expression.In some embodiments, the second audio configuration file 714-2 and the first sound Frequency configuration file 714-1 identical (for example, default audio configuration file of keypad sound).

Because detecting the second time T of the second input 716₂In threshold time T_thresLater, so equipment 100 continues root The first audio output is presented according to the first audio configuration file 714-1.In contrast, Fig. 7 E-7G is similar to Fig. 7 B-7D, in addition to Detect the second time T of the second input 716₂In threshold time T_thresExcept before.In this case, meet audio change mark Standard, therefore equipment 100 does not continue that first audio output with the first audio configuration file 714-1 is presented, but presenting has (the first modified audio output follows Fig. 7 F- to the first modified audio output of modified audio configuration file 714-3 The solid line for being is marked in 7G；Be marked as the dotted line of 714-1 for show if less than footsteps frequency change standard without What the audio configuration file of change should be).

In some embodiments, when the first audio output is presented in equipment 100, equipment 100 presents defeated with the first audio Corresponding visual effect out.For example, as shown in figs. 7 c-7d, visual effect includes that can indicate 708-1 outward (for example, remote from showing From) extend one or more figures 718 (for example, ring, ripple).Similarly, when the second audio output is presented in equipment 100, Equipment 100, which is presented, can indicate the outwardly extending figure 720 (for example, ring, ripple) of 708-2 far from showing.

Additional detail in relation to Fig. 7 A-7G (and user interface shown in it) is provided below with reference to Fig. 9 A-9C.

Fig. 8 A-8B is the stream for describing the method 800 of the presentation for dynamically adjusting audio output according to some embodiments Cheng Tu.Fig. 6 A-6O is used for the method and/or process of explanatory diagram 8A-8B.Although touch-sensitive display (wherein touch sensitive surface will be referred to Be combined with display) on input provide some examples below, but in some embodiments, equipment detect with Input on the separated touch sensitive surface 451 of display 450, as shown in Figure 4 B.

In some embodiments, method 800 by electronic equipment (for example, portable multifunction device 100 of Figure 1A) and/ Or one or more components of electronic equipment (for example, I/O subsystem 106, operating system 126 etc.) execute.In some embodiment party In case, method 800 is managed by instruction, which is stored in non-transient computer readable storage medium and by the one of equipment A or multiple processors (such as one or more processors 122 of equipment 100 (Figure 1A)) execute.For ease of description, below The method 800 executed by equipment 100 is described.In some embodiments, with reference to Figure 1A, the operation of method 800 is at least partly Executed by audio preview module 163-1, audio modification module 163-2 and touch-sensitive display (for example, touch screen 112) or to its into It exercises and uses.Some operations in method 800 are optionally combined and/or the sequence of some operations is optionally changed.

As described below, method 800 (and associated interface) reduce the quantity of input from the user, range and/or Property, and more effective man-machine interface is generated, come and user interface to provide easy to use and intuitive way for user It interacts.For battery powered electronic equipment, method 800 by provide should be readily appreciated that with informative audio output come It realizes effective, seamless and quick interaction, save electric power and increases the time between battery charging (for example, passing through reduction Draw the demand of extensive and inefficient user's interaction of battery capacity).

In some embodiments, when equipment provides (802) first sound to the audio system communicated with the equipment When information is to be presented acoustic characteristic (" the non-volume acoustic characteristic ") including volume and in addition to volume, method 800 starts.Example Such as, in Fig. 6 A-6H, the first audio output is the Moonlight Sonata of Beethovan.In all cases, the first audio output be The song that occurs in text instant message window, album, the tinkle of bells, the audio-frequency unit of video content or from audio file object Audio.

In some embodiments, non-volume acoustic characteristic is the reverberation time of (804) first audio output, the first audio The LPF Cutoff (for example, cutoff frequency) of output or the stereo balance of the first audio output (also referred to as control flat Weighing apparatus, " L-R " balance).

When the first audio output is presented in audio system, equipment reception (806) is asked with the second audio output for rendering Ask corresponding input.In some embodiments, it when focus selector is located on Drawing Object, is carried out with electronic equipment The input of (808) (or being otherwise associated) corresponding with the second audio output is received on the display of communication.

For example, input be touch sensitive surface (for example, with electronic equipment integrate touch-sensitive display or with electronic equipment carry out Touch sensitive surface on the remote controler of signal communication) on pressing input.In graphical user circle associated with the second audio output In face of as receiving pressing input on (for example, the video content object in text message window, Fig. 6 I-6O).

As another example, input corresponds in graphical user interface object (example associated with the second audio output Such as, the original image of album cover, Fig. 6 A-6H) on gently sweep gesture on the touch sensitive surface that starts.

In some embodiments, when graphical user interface object indicates song, album or video, the second audio output It is the preview of song, album or video (for example, a part of song, album or video, has been preselected as representative or can recognize Song, album or video).In some embodiments, a part of song, album or video has the length (example pre-seted Such as, 30 seconds).

For example, input, which corresponds on touch sensitive surface, gently sweeps gesture, this gently sweeps gesture associated with the second audio output Graphical user interface object (for example, graphical user interface object includes the original image for album cover, Fig. 6 A-6H) above (or thereon) start.Another above-mentioned example is：Wherein it is located at figure associated with the second audio output in focus selector In user interface object (for example, graphical user interface object includes the video content object in text message window, Fig. 6 I-6O) Fang Shi receives pressing input.Other examples of graphical user interface object include the text representation of the tinkle of bells, in text instant message Audio file object occurred in window etc..

In response to receiving input corresponding with the request of the second audio output for rendering, equipment is provided to audio system (810) information, to dynamically adjust the presentation of the first audio output according to the magnitude of input.In some embodiments, The magnitude of input is the contact in the characteristic strength of contact, the time span of contact in input or input in (812) input The distance (for example, the length for gently sweeping gesture) of traveling.

As the magnitude of input changes, equipment dynamically adjusts (814) non-volume acoustic characteristic.As the magnitude of input changes Become and adjusting non-volume acoustic characteristic allows user effectively preview or to listen to the second audio output without interrupting the first audio Output, and user is given to the additional control of audio output by single user's input.Which enhance operating for equipment Property, and make user device interface more effective (for example, must be into when operating equipment/interact with equipment by reducing user The quantity of capable user's input), this makes to reduce electric power additionally by allowing users to faster and more efficiently use equipment With and improve the battery life of equipment.

In some embodiments, the stereo balance degree of first audio output of device shifting, with the second sound of hour offset The stereo balance degree (816) (for example, with reference to as described in Fig. 6 A-6O) of frequency output.For example, the input corresponds on touch sensitive surface Gently sweep gesture, this is gently swept gesture and moves (for example, gently sweeping from left to right) in a first direction far from graphical user interface object Initial position associated with the second audio output (for example, this gently sweeps gesture for the original image of album cover towards display Center dragging, Fig. 6 A-6H).In this example, offset stereo balance degree includes that the first audio output of offset and the second audio are defeated Presentation out, so that they, which are tracked, gently sweeps input moving (for example, the stereo balance degree of the first audio output along first direction It deviates to the right, and the second audio output fades in since the left side of audio system).In other embodiments, based on input The variation of the intensity of contact or time of contact based on input and deviate the stereo of the first audio output and the second audio output The degree of balance.

As another embodiment, in some embodiments, non-volume audio attribute is the low of the first audio output Pass filter cutoff.The equipment offsets downward LPF Cutoff according to the increase of the magnitude of input.Therefore, with defeated The magnitude that enters increases, and the first audio content is reduced to bass background (for example, " in a grand fashion Hong "), enable the second audio content with Generate the second audio content clear audible mode and the first audio content simultaneously (for example, because the high-lighting of background audio Reduce due to application low-pass filter).

In some embodiments, which adjusts the volume of the first audio output with the variation of the magnitude of input (818) (for example, both volume and non-audio attribute change with the variation of the magnitude of input).In some embodiments, The volume of first audio output reduces with the increase of the magnitude of input.The volume of the first audio output is adjusted in this way The sense of hearing for also increasing the second audio output is prominent, without interrupting the first audio output.This makes the mould of two audio output Paste is more effective, thus enhances the operability of the equipment and make user equipment interface more effective.

In some embodiments, the equipment is before the presentation of the second audio output, with the variation of the magnitude of input And non-volume audio attribute is dynamically adjusted, until the magnitude of input meets the first predetermined threshold value (820) (for example, defeated The characteristic strength of contact in entering keep below Preview Threshold but be higher than prompting threshold value, thus the equipment equipment is provided will be pre- Look at the auditory cues and/or visual cues of the second audio output, as described with reference to FIG. 6B).Once the magnitude of input meets first Predetermined threshold value (for example, being more than slightly gently to sweep movement, Fig. 6 C), the second audio output starts to play, and some In embodiment, the first audio output is further adjusted with the variation of the magnitude of input.In the presentation of the second audio output It is provided before audio obscures imminent prompt for user, non-volume sound is dynamically adjusted with the variation of the magnitude of input Frequency attribute, the prompt can be used for for example alerting user when user chances on equipment input function described in method 800.

In response to receiving input corresponding with the request of the second audio output for rendering, which mentions for audio system For second sound information, the second audio output (822) to be presented simultaneously with the first audio output.In some embodiments, should Equipment provides the presentation (824) that information dynamically adjusts the second audio output with the magnitude according to input.In some embodiment party In case, execute while the magnitude according to input dynamically adjusts the presentation of the first audio output to the second audio output It presents.In this way, the first audio output and the second audio output dynamically obscure simultaneously (for example, the first audio output is gradually faded out, And the second audio output gradually fades in).Therefore, the presentation that the second audio output is dynamically adjusted according to the magnitude of input provides The seamless transitions of high-lighting from the high-lighting of the first audio output to the second audio output, this makes the fuzzy for idol of audio It so finds more pleasant for the user of equipment input function described in method 800 and less causes panic.This Then it increases user and is kept for a possibility that realizing this function on their device, and therefore by increasing user and equipment Interactive efficiency and save the power of battery.

In some embodiments, in response to receiving input corresponding with the request of the second audio output for rendering, The equipment provides data to display to show in conjunction with the visual effect for dynamically adjusting non-volume audio attribute and changing (826).For example, the equipment to display provide data with visually obscure user interface in addition to the second audio output Anything except corresponding graphical user interface object, wherein blur radius is proportional to the magnitude of input.Display combines The visual effect for dynamically adjusting non-volume audio attribute and changing provides intuitive visual cues corresponding with the audio of variation, This facilitates user and their equipment interacts.

In some embodiments, which detects that the magnitude of input meets and is greater than the of the first predetermined threshold value Two predetermined threshold values and in response, so that audio system stops presenting the first audio output and (and continues presentation the Two audio output) (828).In some embodiments, the second predetermined threshold value (" pop-up " threshold value) with first in advance really Fixed threshold value compares the intensity of the higher contact corresponding to the input (for example, pop-up threshold value is IT in Fig. 6 I-6O_D), should Input the longer time quantum that the bigger distance advanced or the input are kept in contact with touch sensitive surface.Therefore, audio system is stopped The first audio output is only presented and provides a kind of mode for user to use user to pass through the identical defeated of its first audio output of preview Enter and be switched to " the presenting completely " of the second audio output.This reduces the numbers of user's interaction needed for realizing specific function Amount.

In some embodiments, which detects the end (830) (for example, as shown in Fig. 6 E-6H) of input.One A bit in alternative embodiment, substitution detects the end of input, which detects that the variation of the magnitude of input has stopped (example Such as, the intensity of contact returns to its green strength or input returns to its home position).In some alternative embodiments, substitution Detect the end of input, which detects that input is more than the magnitude of predetermined threshold value (for example, pop-up threshold value) Increase.In some alternative embodiments, substitution detects the end of input, which detects the preparatory of the magnitude of input Determining variation.For example, the magnitude of input drops below predetermined threshold value (for example, the magnitude of input drops below Threshold value is popped up, and inputs and is kept in contact with touch sensitive surface).In some embodiments, replace the end for detecting input, this sets It is standby to detect that the variation of input has stopped the predetermined period.

Under any circumstance, in response to condition appropriate, the equipment make audio system perform the following operation in one Operation：

Stop that the second audio output is presented and the first audio output is presented, without being moved to the first audio output State adjusts (832) (for example, revocation adjusts the dynamic of the first audio output).In some embodiments, the second sound is being played Frequency export when, adjustment rather than suspend the first audio output broadcasting, and the first audio output delete adjustment in the case where, Continue to play after the second audio output stops playing.This provides a kind of intuitive way back to only listening to for user One audio output, to reduce the quantity of user's input.For example, the equipment stops that the second audio output is presented in Fig. 6 H And the first audio output is presented in the case where adjusting without dynamic.

Stop that the first audio output is presented and continues that the second audio output (834) are presented.This provides one for user Kind intuitive way, which switches to, only listens to the second audio output, to reduce the quantity of user's input.For example, in Fig. 6 H, The equipment stops that the first audio output is presented and the second audio output is presented in the case where adjusting without dynamic.

Stop that adjusted first audio output is presented and continues that the second audio output (836) are presented.This is user A kind of intuitive way is provided to continue fuzzy audio；Or

Meet predetermined threshold value (for example, being more than pop-up threshold value during input) according to determining magnitude, stopping is in Show the first audio output and continues that the second audio output is presented；And the threshold value is unsatisfactory for (for example, this is defeated according to determining magnitude Enter to keep below pop-up threshold value during input), stop that the second audio output is presented and is in the case where being adjusted without dynamic Existing first audio output (for example, restoring to adjust the dynamic of the first audio output) (838).In this way, the system automatically determines User, which actually wants to continue listening to the first audio output, still wants to switch to the second audio output, to provide one for user Intuitive way is planted to realize any result and reduce the quantity of user's input.For example, in Fig. 6 E-6H, the equipment root Meet according to the magnitude for determining user's input and is unsatisfactory for threshold value still to determine whether to stop the second audio output to be presented and in nothing The first audio output is presented in the case where adjusting in dynamic.

It should be appreciated that being only an example to the concrete order for the operation being described in Fig. 8 A-8H, it is not intended that table The bright order is the but one sequence that these operations can be performed.Those skilled in the art will recognize that various ways come to this Operation described in text is resequenced.In some embodiments, by one or more operations of method 800 with it is described herein Other methods (for example, method 840,854,875 and/or 900) one or more operative combinations or by they supplement or replace Generation.

Fig. 8 C is the process for showing the method 840 of the presentation for dynamically adjusting audio output according to some embodiments Figure.In some embodiments, one or more operations by method 840 and other methods as described herein are (for example, method 800) one or more operative combinations are supplemented or substitution by them.In addition, many operations with reference to described in method 840 with On the operation with reference to described in method 800 share identical benefit (for example, the use that the intuitive of audio is fuzzy, user necessarily is formed The reduction of the quantity of family input).Therefore, method 840 is also (for example, by helping user to carry out in operation equipment/and the equipment Correctly input is provided when interactive and reduces user's mistake) enhance the operability of equipment and user equipment interface is more had Effect, this in addition also by using family can more rapidly and more efficiently use the equipment and reduce power use and improve The battery life of the equipment.

In some embodiments, method 840 is held at the electronic equipment with memory and one or more processors Row, and the electronic equipment is communicated with display and audio system.

In some embodiments, data are provided so that the user interface including media object is presented to display in the equipment When, method 840 starts, which indicates at least one items of media content (842).

The equipment provides the first acoustic information to audio system the first audio output not corresponding with media object is presented (844)。

When providing the first acoustic information to audio system, which detects first of the direction media object of input Divide (846).

In response to detecting the first part of the direction media object of input, the equipment：(i) start to provide to audio system Second sound information is to be presented the second audio output corresponding with media object；And (ii) continues to provide first to audio system Acoustic information is to be presented the first audio output (848) not corresponding with media object.

When providing the first acoustic information and second sound information to audio system, which detects the direction matchmaker of input The second part of body object, wherein detecting that the second part of input includes variation (the 850) (example for the parameter for detecting input Such as, it is detected when focus selector is above media object and the variation of the intensity of the contact of touch sensitive surface or detects focus The position of selector or the variation of movement).

In response to detecting the variation of the parameter of input, the equipment：

(i) (852) data are provided (for example, visually obscuring and/or putting by using family interface background to display Big media object) presented user interface dynamically changed according to the variation of the parameter of input；

(ii) information is provided (for example, by reducing volume, increasing reverberation time, reduction low-pass filtering to audio system Device cutoff frequency, and/or the L-R degree of balance of audio to the right or is moved to the left) according to the variation of the parameter of input come dynamically Change the first audio output not corresponding with media object；And

(iii) information is provided (for example, by reducing volume, increasing reverberation time, reduction low-pass filtering to audio system Device cutoff frequency, and/or the L-R degree of balance of audio to the right or is moved to the left) according to the variation of the parameter of input come dynamically Change and media object are to the second audio output (852) of drink.

It should be appreciated that being only an example to the concrete order for the operation being described in Fig. 8 C, it is not intended that Show that the order is the but one sequence that these operations can be performed.Those skilled in the art will recognize that various ways come pair Operations described herein is resequenced.In some embodiments, by one or more operations of method 840 and this paper institute It states one or more operative combinations of other methods (for example, method 800,854,875 and/or 900) or is supplemented or replaced by them Generation.

Fig. 8 D-8F is the stream for showing the method 854 of the presentation for dynamically adjusting audio output according to some embodiments Cheng Tu.Fig. 6 P-6Y is used to show the method and/or process of Fig. 8 D-8F.Although (touching will be wherein combined with reference to touch-sensitive display Sensitive surfaces and display) on input provide some embodiments in subsequent embodiment, but in some embodiments, The input on touch sensitive surface 451 that equipment detection is separated with display 450, as shown in Figure 4 B.

In some embodiments, method 854 by electronic equipment (for example, portable multifunction device 100, Figure 1A) and/ Or one or more components (for example, I/O subsystem 106, operating system 126 etc.) of the electronic equipment execute.In some implementations In scheme, method 854 is by being stored in non-transient computer readable storage medium and by the one or more of equipment The instruction of one or more processors 122 (Figure 1A) execution of device such as equipment 100 is managed to dominate.In order to be easy to explain, hereafter Describe the method 854 executed by equipment 100.In some embodiments, with reference to Figure 1A, the operation of method 854 is at least partly Ground is by audio preview module 163-1, audio modification module 163-2 and touch-sensitive display (for example, touch screen 112) execution or at least Partly it is carried out using.Some operations in method 854 be optionally combined and/or the sequence of some operations optionally It is changed.

As described below, method 854 (and associated interface) reduce the quantity of input from the user, range and/or Property, and more effective man-machine interface is generated, come and user interface to provide easy to use and intuitive way for user It interacts.For battery-operated electronic equipment, method 854 allows to fill twice by providing to save power and increase battery Time interval between electricity be readily appreciated that and the audio output of informedness is (for example, by reducing to exhausting the close of the power of battery Collection and the demand of inefficient user interaction) realize effective, seamless and quick interaction.

In some embodiments, which shows the user interface (855) of the expression including media item over the display When, method 854 starts.For example, Fig. 6 Q shows the user interface 640 of the expression including media item 658.

When showing user interface, expression with media item of the equipment on touch sensitive surface detects at corresponding position By input (856) caused by contact (for example, finger contact or stylus contact) (for example, media item on the touch sensitive display Expression on detect contact).For example, Fig. 6 R shows the beginning of user's input 660, user input is to indicate 658-2 On the pressing that detects and keep gesture.

In response to detecting through input caused by contacting：Meet media cue standard according to the determining input, wherein should Media cue standard includes being higher than the first intensity threshold (for example, IT when the contact has_H, Fig. 6 R) characteristic strength when meet Standard：The equipment starts to play the corresponding portion of media item (for example, being selected to the beginning of the recognizable part comprising media item Part represents part) (for example, audio and/or video for playing them) (857).For example, equipment 100 starts in Fig. 6 R Play Brahms ' s Alto Rhapsody.Moreover, when playing media item, the equipment change with the characteristic strength of contact and One group of one or more audio attribute is dynamically changed (for example, the characteristic strength with contact passes through the change of multiple values, the media The audio attribute of item passes through multiple values and changes).In some embodiments, the volume of media item with contact characteristic strength Increase and increases and reduce with the reduction of the characteristic strength of contact.For example, as the intensity of input 660 is from Fig. 6 R to figure 6S increases, and the volume of Brahms ' s Alto Rhapsody increases.

In some embodiments, the media item played based on the contact in input position and determine (for example, according to Determining detect at position corresponding with the first media item contacts, and above step is executed for the first media item, and root It detects and contacts at position corresponding with the second media item according to determining, above step is for the second media item rather than for the One media item and execute).For example, if indicating to detect the input 660 in Fig. 6 R on 658-2 indicating 658-1, Equipment 100 will start to play media item corresponding with 658-1 is indicated.

It is unsatisfactory for media cue standard according to the determining input, which abandons the corresponding portion for starting to play media item, And abandon the one or more audio attributes of this group for changing and dynamically changing media item with the characteristic strength of contact.One In a little embodiments, which executes alternative operation relative to media item.For example, in response to the tap in the expression of media item Gesture, the equipment execute selection operation relative to media item.

In some embodiments, the audio attribute of media item include volume, low-pass filter cutoff frequency and/or Weighing apparatus setting (858) is (for example, Fig. 6 P-6Y is shown in which that equipment dynamically changes matchmaker as the characteristic strength of contact changes One example of the volume of body item).

In some embodiments, when this of media item, which organizes one or more audio attributes, to be dynamically changed, the equipment Dynamically change the visual appearance (859) of user interface.In some embodiments, outside the vision for dynamically changing user interface Sight includes increasing the size (860) of the expression of media item (for example, with the spy of contact with the increase of the characteristic strength of contact Levy intensity increase and increase the expression of media item size and with the characteristic strength of contact reduction and reduce media item Expression size).For example, the intensity with input 660 increases from Fig. 6 R to Fig. 6 S, the volume and expression 658-2 of media item Size increase.In some embodiments, this is the same expression that initially shows in the user interface (such as Fig. 6 R-6T institute Show, indicates 658-2 by its original size and position " growth ", the central part towards the second area of user interface is mobile, together When also increase size).In other embodiments, when the second area of the expression and user interface that initially show is maintained at same When one position, the expression is different from initial representation, and the second different increasings for indicating the characteristic strength with contact of media item Big and increased in size and mobile (for example, at the beginning of difference expression is shown in towards the central part of the second area of user interface The top that beginning indicates).

In some embodiments, the visual appearance for dynamically changing user interface includes as the characteristic strength of contact changes Become and dynamically change the fuzzy quantity (861) of a part of the expression of the neighbouring media item of user interface.For example, with input 660 intensity increases from Fig. 6 R to Fig. 6 S, and interaction area 642-2 is fuzzy, and blur radius and the intensity for contacting 660 are proportional (in these attached drawings, the blur radius of increase by cover interaction area 642-2 pattern reduced transparency schematically It indicates).

In some embodiments, after the corresponding portion for starting to play media item：The equipment detects the spy of contact The increase for levying intensity detects the reduction (862) of the characteristic strength of contact (for example, Fig. 6 T shows the intensity of contact 660 later Increase；Fig. 6 U shows the reduction of the intensity of contact 660).In some embodiments, multiple condition behaviour are described below Make, whether these conditional operations meet media preview standard according to the strength characteristic for determining contact and be adjusted.These operations In response to detecting the increase of the characteristic strength of contact, the reduction of the characteristic strength of contact is detected later and is executed.Some In embodiment, in the increase for the characteristic strength for detecting contact, when detecting the reduction of the characteristic strength of contact later, execute Conditional operation described below.

For this purpose, before the reduction for the characteristic strength for detecting contact, being contacted according to determining when media item is playing Characteristic strength meet media preview standard, the media preview standard include when contact characteristic strength increase to above than first Intensity threshold is (for example, IT_H) higher second intensity threshold is (for example, IT_L, Fig. 6 T-6U) when the standard that meets, the equipment with The increase of the characteristic strength of contact and in the first way come change media item audio attribute (for example, increase media item audio Volume) and so that audio attribute is maintained at first state (for example, continuing to set sound with the reduction of the characteristic strength of contact Frequency volume playback media item).In some embodiments, as the audio attribute of media item changes, the audio of background audio Attribute also changes (for example, the increasing for making background audio fade in the reduction of intensity and make background audio with intensity It fades out greatly).Audio attribute is set to be maintained at first state with the reduction of the characteristic strength of contact (for example, continuing to set Audio volume playback media item) allow user enter " prying through " mode, wherein the audio attribute of media item is fixed, this for The user for wanting to listen to a part of media item is easily, without lasting modification audio attribute.Since audio attribute only exists Contact strength, which increases to above, to be only after the second intensity threshold fixed, therefore user neatly can select him or she to want Operation mode.Which reduce realize it is expected functions of the equipments needed for user input quantity, thus (for example, by help user When operating the equipment/interact with the equipment, offer is correct inputs and reduces user's mistake) and enhance grasping for equipment The property made simultaneously makes user equipment interface more effective, in addition this also more rapidly and can more efficiently use the equipment by using family And reduce the battery life that power uses and improves the equipment.

For example, the intensity due to contact 660 is higher than IT in Fig. 6 T_L, therefore the volume of media item is " locked " to preview Volume (for example, full volume), so that the intensity of contact 660 then drops below IT as shown in Fig. 6 U_LIt not will lead to volume Variation.According to determining that the characteristic strength of contact is unsatisfactory for media preview standard (that is, the characteristic strength of contact does not increase to above the Two intensity thresholds), when playing media item (and when contact is maintained on touch sensitive surface), the equipment with contact feature The increase of intensity and in the first way come change media item the audio attribute audio volume of media item (for example, increase) and with The reduction of the characteristic strength of contact and change the audio attribute of media item (for example, increasing audio in turn in a second manner Volume).For example, the volume of media item reduces with the reduction of intensity if sequence shown in Fig. 6 S-6R is reversed.Therefore, In Fig. 6 S-6R, user can scroll up and down the volume of media item and increasing and reducing the intensity of contact 660, as long as contact 660 intensity is no more than IT_L.In some embodiments, the corresponding portion of media item with contact characteristic strength change and It continuously plays, wherein this organizes one or more audio frequency parameters and changes with the broadcasting of media item.

In some embodiments, in response to detecting that the input meets media preview standard, the equipment is over the display Display (863) contact characteristic strength increase by make equipment relative to media item execute selection operation instruction (for example, " more forcibly pressing to be selected " or " carrying out 3D to touch with shared ").

In some embodiments, after the corresponding portion for starting to play media item, which is maintained at touch-sensitive in contact When on surface, the increase (864) of the characteristic strength of contact is detected.In response to detecting the increase of the characteristic strength of contact：Root According to determine contact characteristic strength meet media selection criteria, the media selection criteria include when contact characteristic strength be greater than than First intensity threshold it is higher (and also higher than the second intensity threshold, the standard that when operation selection intensity threshold value 862) meets, The equipment stops the corresponding portion of display media item and executes selection operation (for example, media item is placed for media item In message compositing area or with another user sharing media item in instant messaging session).For example, in Fig. 6 V-6W, due to The intensity of contact 660 is more than IT_D, therefore select to indicate 658-2, so that corresponding audio message 644-4 is placed on frame 650 In.It is unsatisfactory for media selection criteria according to the characteristic strength for determining contact, which continues to play the corresponding portion of media item, and Without executing selection operation.

In some embodiments, when playing media item, which detects the end (865) of input (for example, detection It is lifted away to the contact for causing media item to start to play).In response to detecting the end of input, which stops playing media ?.In some embodiments, stop playing media item including making neighbouring media item in response to detecting the end of input The dimness of vision of the content of expression inverts and changes the one or more attributes of this group of audio items so that the media item played It gradually fades out (866).For example, equipment 100 will stop broadcasting if input 660 has terminated in any one figure of Fig. 6 R-6U It puts media item and makes the fuzzy reversion of interaction area 642-2.

In some embodiments, in response to detecting the end of input after stopping playing media item, which exists Caused by detecting at the corresponding position of expression (for example, Flick gesture) of media item and being contacted by second on touch sensitive surface Selection input, wherein second contact does not have the characteristic strength (867) for reaching the first intensity threshold.In response to detecting selection Input, the equipment execute selection operation for media item (for example, media item is placed on message compositing area such as frame 650 In (Fig. 6 P) or with another user sharing media item in instant messaging session).

Operation 868-870 describes the fuzzy of audio (that is, the media item with background media that played in equipment). It is fuzzy that audio is described in detail in method 800 and 840.Therefore, operation 868-870 can share in feature described in these methods Any one feature.

In some embodiments, when playing background media in equipment, detect input (868) (for example, operating The input detected in 856).Meet media cue standard according to the determining input：When playing media item, the equipment is with connecing The characteristic strength of touching changes and dynamically changes one group of one or more audio attribute (869) of background media item.Therefore, one In a little embodiments, which makes the media item " fading in " above background media, this is provided with reference to described in method 800/840 The benefit being all appended.

It is unsatisfactory for media cue standard according to the determining input, which continues to play background media, carries on the back without changing This of scape media organizes one or more audio attributes.In some embodiments, when playing media item, which detects defeated The end (870) entered.In response to detecting the end of input, the equipment restore play background media item, wherein this group one or Multiple audio attributes return to their value before detecting input.

It should be appreciated that being only an example to the concrete order for the operation being described in Fig. 8 D-8F, it is not intended that table The bright order is the but one sequence that these operations can be performed.Those skilled in the art will recognize that various ways come to this Operation described in text is resequenced.In some embodiments, by one or more operations of method 854 with it is described herein One or more operative combinations of other methods (for example, method 800,840,875 and/or 900) are supplemented by them or are substituted.

Fig. 8 G-8H is the stream that the method 875 of multiple portions of masking graphic user interface is shown according to some embodiments Cheng Tu.Fig. 6 P-6Y is used to show the method and/or process of Fig. 8 G-8H.Although (touching will be wherein combined with reference to touch-sensitive display Sensitive surfaces and display) on input provide some examples in subsequent example, but in some embodiments, equipment detection The input on touch sensitive surface 451 separated with display 450, as shown in Figure 4 B.

In some embodiments, method 875 by electronic equipment (for example, portable multifunction device 100, Figure 1A) and/ Or one or more components (for example, I/O subsystem 106, operating system 126 etc.) of the electronic equipment execute.In some implementations In scheme, method 875 is by being stored in non-transient computer readable storage medium and by the one or more of equipment The instruction of one or more processors 122 (Figure 1A) execution of device such as equipment 100 is managed to dominate.In order to be easy to explain, hereafter Describe the method 875 executed by equipment 100.

As described below, method 875 (and associated interface) reduce the quantity of input from the user, range and/or Property, and more effective man-machine interface is generated, come and user interface to provide easy to use and intuitive way for user It interacts.For battery-operated electronic equipment, method 875 allows to fill twice by providing to save power and increase battery Time interval between electricity be readily appreciated that and the audio output of informedness is (for example, by reducing to exhausting the close of the power of battery Collection and the demand of inefficient user interaction) realize effective, seamless and quick interaction.

In some embodiments, the first interaction area (example including application program is shown over the display in the equipment Such as, interaction area 642-2, Fig. 6 Q) and application program the second interaction area (for example, interaction area 642-1, Fig. 6 Q) user When interface (876), method 875 starts.In some embodiments, interaction area is that user can be such as a variety of defeated by providing The interface region for entering any of gesture (gently sweep, tap) input gesture and interacting, these input hands Gesture is when being detected, so that electronic equipment modifies user interface according to these gestures.In other words, interaction area is not The user for being displayed on top section cannot execute the status bar of any friendship by it.

In some embodiments, the second interaction area is for primary application program (for example, instant message application program) User interface (877).First interaction area is configured as showing the difference from being configured as operating in primary application program The content of miniature applications program, and shown miniature applications program in the first interaction area is defeated based on the user at equipment Enter that (for example, user can execute in the first interaction area gently sweeps gesture, these are gently swept gesture and are being detected to be selected When, so that the equipment shows user interface for the different miniature applications programs in the first interaction area, such as reference designator Described in 654 (Fig. 6 Q)).For example, instant message application program can be shown and equipment media (for example, music, video), equipment phase Machine, the miniature applications program drawing character art etc. and interacting.

In some embodiments, the second interaction area be include multiple message in session between sessions participant Conversation region (878) (for example, the second interaction area be instant message application program (user such as with electronic equipment with extremely The conversation recording of message between few other users) in conversation region such as interaction area 642-1 (Fig. 6 Q).First hands over Mutual region is for selecting media so that media selection region shared in the session between sessions participant is (for example, immediately Music sharing region such as interaction area 642-2 (Fig. 6 Q) in message application, the music sharing region include that can roll List, this can roll the expression that list shows 30 songs shown recently in the equipment).

When showing user interface, the equipment over the display with the first user interface element in the first interaction area It is detected at corresponding position through the first input caused by the contact (for example, finger contact or stylus contact) on touch sensitive surface (879).For example, Fig. 6 R shows the beginning of user's input 660, user input is the pressing detected on indicating 658-2 And keep gesture.

In response to detecting through the first input caused by contacting：Meet the activation based on intensity according to determining first input Standard, wherein the contact of the activation standard requirements based on intensity, which has, is higher than the first intensity threshold (for example, IT_H, Fig. 6 R) feature Intensity is to meet the activation standard based on intensity, in addition to the first user interface element, the of the equipment hide application program One interaction area, the second interaction area without hide application program is (for example, keep interaction area 642-2 fuzzy, without making Indicate that 658-2 is fuzzy, Fig. 6 R-6V) (880).In addition to the first user interface element, the first interactive areas of application program is covered Domain indicates to the user that he or she opens in such as miniature applications program without covering the second interaction area of application program With the functions of the equipments based on intensity.Which reduce the chance that user will make the equipment execute unnecessary operation, this (for example, By helping user's offer when operating the equipment/interact with the equipment is correct to input and reduce user's mistake) it enhances The operability of equipment simultaneously makes user equipment interface more effective, this in addition can also more rapidly and more effectively by using family Reduce the battery life that power uses and improves the equipment using the equipment.

According to determining that the first input meets first choice standard, wherein first choice standard does not require contact to have to be higher than the To meet selection criteria (for example, first input be Flick gesture), which executes and the characteristic strength of one intensity threshold The corresponding first choice operation of one user interface element, without covering the first interaction area of application program.In some implementations In scheme, for cover above-mentioned interaction area first choice operate based on contact position (for example, according to determination with the first matchmaker Body item detects contact at corresponding position) and determine, above step is executed for the first media item, and is existed according to determination It detects and contacts at position corresponding with the second media item, above step is for the second media item rather than is directed to the first media item And it executes.For example, equipment 100 is hidden if indicating to detect the input 660 in Fig. 6 R on 658-2 indicating 658-1 The anything in addition to indicating 658-1 in interaction area 642-2 is hidden, rather than hides in interaction area 642-2 and removes table Show the anything except 658-2.

In some embodiments, meet first choice standard (881) when the first input is Flick gesture.

In some embodiments, the first interaction area for covering application program includes as the characteristic strength of contact changes And the first interaction area (882) of application program is dynamically covered (for example, the characteristic strength with contact is changed by multiple values Become, the fuzzy blur radius applied to the first interaction area is changed by multiple values).In some embodiments, first The fuzzy blur radius of interaction area increases with the increase of the characteristic strength of contact and with the characteristic strength of contact Reduction and reduce.For example, the intensity with input 660 increases from Fig. 6 R to Fig. 6 S, interaction area 642-2 is fuzzy, obscures Radius and the intensity for contacting 660 it is proportional (in these attached drawings, figure that the blur radius of increase passes through covering interaction area 642-2 The reduced transparency of case and schematically show).

In some embodiments, when showing user interface, the equipment over the display in the second interaction area The corresponding position of second user interface element at detection pass through on touch sensitive surface second contact caused by second input (883).In response to detecting through the second input caused by the second contact：Meet second according to determining second input and is based on intensity Activation standard, wherein second based on intensity activation standard requirements second contact have higher than the second intensity threshold feature it is strong Degree is to meet the second activation standard based on intensity, except second user interface element is in addition to, the equipment hide application program The first interaction area and application program the second interaction area (for example, the equipment makes the first interaction area and the second interactive areas The content in domain is faded out, while preview media item corresponding with not fuzzy second user interface element).In some embodiments In, the second intensity threshold is identical as the first intensity threshold.In some embodiments, the second intensity threshold and the first intensity threshold It is different.In addition to second user interface element, the first interaction area of application program and the second interactive areas of application program are covered Domain indicates to the user that he or she for example enables relative to second user interface element based on intensity in primary application program Functions of the equipments.Which reduce the chances that user will make the equipment execute unnecessary operation, this is (for example, by helping user When operating the equipment/interact with the equipment, offer is correct inputs and reduces user's mistake) enhance operating for equipment Property simultaneously makes user equipment interface more effective, this in addition also by using family can more rapidly and more efficiently use the equipment and Reduce the battery life that power uses and improves the equipment.

For example, basis 674 has more than IT as shown in Fig. 6 Y_HIntensity, and therefore interaction area 642-1 and 642-2 The two is blurred.Therefore, it is higher than IT in the expression of the media item in miniature applications program_HPressing input cause it is only micro- Type application program is fuzzy, and in interaction area corresponding with primary application program (for example, the conversation area of instant message application program Domain) in media item expression on similar pressing input cause include the two interaction areas 642 entire user interface It is fuzzy.

According to second input the second selection criteria of satisfaction is determined, wherein the second selection criteria does not require the second contact to have height In the characteristic strength of the second intensity threshold to meet the second selection criteria (for example, second input be Flick gesture), the equipment Execute corresponding with second user interface element the second selection operation, without masking application program the second interaction area and Without covering the first interaction area (for example, device playback media item corresponding with second element) of application program.For example, figure Flick gesture in any expression of expression 658 in 6Q media item is placed in frame 650 (or according to some embodiments, These media items are sent directly to session).

In some embodiments, it is added in the first user interface element based on the input in the first interaction area After second user interaction area, second user interface element corresponds to the first user interface element (884) (for example, second uses Family interface element corresponds to from selecting in the first interaction area and be added to the session shown in the second interaction area Media item).For example, in Fig. 6 Q-6Y, audio message 644-4 and both indicate 658-2 and correspond to same media item (for example, Brahms’s Alto Rhapsody)。

In some embodiments, when the first interaction area for covering application program in addition to the first user interface element When (without covering the second interaction area of application program), which detects the increase of the characteristic strength of contact.In response to Detect the increase of the characteristic strength of contact：Meet third selection criteria according to the characteristic strength for determining contact, third selection Standard includes when the characteristic strength of contact is greater than selection intensity threshold value more higher than the first intensity threshold (for example, IT_D, Fig. 6 V) when The standard of satisfaction, the equipment execute third selection operation to the first user interface element and stop covering the first of application program Interaction area.Therefore, because contact 660 is in Fig. 6 V more than IT_D, therefore indicate that 658-2 is ejected into frame 650 and (wherein should Expression is denoted as audio message 644-4 again).Meet third selection criteria according to the characteristic strength of determining contact and uses first Once family interface element execute selection operation and stop cover application program the first interaction area allow user user sent out It plays the above-mentioned operation based on intensity and just selects the first user interface element, to eliminate to the first user interface element of selection simultaneously And make the needs of the more effective further user input of man-machine interface.

It is unsatisfactory for third selection criteria according to the characteristic strength for determining input, in addition to first user's bound component, the equipment Continue the first interaction area of hide application program, the second interaction area without hide application program.

It should be appreciated that being only an example to the concrete order for the operation being described in Fig. 8 G-8H, it is not intended that table The bright order is the but one sequence that these operations can be performed.Those skilled in the art will recognize that various ways come to this Operation described in text is resequenced.In some embodiments, by one or more operations of method 875 with it is described herein One or more operative combinations of other methods (for example, method 840,840,854 and/or 900) are supplemented by them or are substituted.

Fig. 9 A-9C is the process for showing the method 900 for dynamically adjusting audio output presentation according to some embodiments Figure.Fig. 7 A-7G is used to show the method and/or process of Fig. 9 A-9C.Although (will be wherein combined with touch-sensitive with reference to touch-sensitive display Surface and display) on input provide some examples in subsequent example, but in some embodiments, equipment detection with Input on the separated touch sensitive surface 451 of display 450, as shown in Figure 4 B.

In some embodiments, method 900 by electronic equipment (for example, portable multifunction device 100, Figure 1A) and/ Or one or more components (for example, I/O subsystem 106, operating system 126 etc.) of the electronic equipment execute.In some implementations In scheme, method 900 is by being stored in non-transient computer readable storage medium and being handled by the one or more of equipment Instruction that the one or more processors 122 (Figure 1A) of device such as equipment 100 execute dominates.In order to be easy to explain, hereafter retouch The method 900 executed by equipment 100 is stated.In some embodiments, with reference to Figure 1A, the operation of method 900 is at least partly By audio configuration file 402, audio change module 163-3 and touch-sensitive display (for example, touch-sensitive display system 112) execution or extremely Partially it is carried out using.Some operations in method 900 are optionally combined and/or the sequence of some operations is optional Ground is changed.

As described below, method 900 (and associated interface) reduce the quantity of input from the user, range and/or Property, and more effective man-machine interface is generated, come and user interface to provide easy to use and intuitive way for user It interacts.For battery-operated electronic equipment, method 900 allows to fill twice by providing to save power and increase battery Time interval between electricity be readily appreciated that and the audio output of informedness is (for example, by reducing to exhausting the close of the power of battery Collection and the demand of inefficient user interaction) realize effective, seamless and quick interaction.

In some embodiments, method 900 is executed on the electronic equipment communicated with display and audio system (902), while on the display of electronic equipment display include one group of one or more show can indicate user interface (for example, Shown user interface includes virtual numeric keypad and this group one or more shows that can indicate includes being displayed on this virtually Number key on numeric keypad, Fig. 7 A-7G).When method 900 starts, equipment detects this group of direction in first time point Or it is multiple show first in capable of indicating show can indicate first input (for example, detecting light on the first key in dummy keyboard Hit, the tap on " 5 " key in such as Fig. 7 B) (904).

In response to detecting that being directed toward first shows the can indicate first input, which starts to provide the first sound to audio system Message breath shows with first to present and can indicate corresponding first audio output (906).There is first audio output the first audio to match Set file (for example, audio configuration file 714-1, Fig. 7 B-7G).

In some embodiments, corresponding audio configuration file includes changing over time to dominate corresponding audio output The information of one or more attributes.For example, corresponding audio configuration file changes over time to dominate the sound of corresponding audio output Height, voice, tone color, reverberation and/or volume (for example, decaying).In some embodiments, corresponding audio configuration file also wraps Include the information for dominating one or more static attributes of corresponding audio output.For example, corresponding audio configuration file includes being used for The information of audio output is generated, which has the fixed pitch of C sharp, wherein non-static volume increases over time Then decaying.

In some embodiments, when the first audio output is being presented in audio system, which show (908) Visual effect corresponding with the first audio output is presented (for example, showing one or more for capable of indicating to extend outward or away from first in device A figure such as ring, such as ripple effect, wherein ring is extended outward or away from the position corresponding with the first input on display) (for example, figure 718,720, Fig. 7 C-7D).In some embodiments, these figures include showing to indicate movement far from first One or more Drawing Objects animation.Visual effect corresponding with the first audio output is presented to provide and the audio for user Corresponding intuitive visual prompt is exported, this facilitates user and their equipment interacts.

The equipment is in the second time point after first time point detects that this group one or more of direction is shown and can be indicated Second show can indicate second input (for example, the tap on the second key in dummy keyboard is detected, in such as Fig. 7 C Tap on " 2 " key) (910).In some embodiments, second shows and can indicate and first shows can be expressed as same showing and capable of indicating (for example, same key on shown keyboard in shown user interface) (912).In some embodiments, first Showing can indicate to show from second and can indicate different (for example, different keys on shown keyboard in shown user interface) (914)。

In response to detect be directed toward second show can indicate second input and according to determine meet audio change standard, be somebody's turn to do Equipment (916)：

(i) the first audio output of corresponding change can be indicated by showing audio system presentation with first, without being to continue with benefit The first audio output is presented with the first audio configuration file, matches wherein the first modified audio output has with the first audio Set the different modified audio configuration file of file.In some embodiments, so that modified is presented in audio system One audio output includes providing information to audio system such as to instruct and/or voice data, so that warp can be presented in audio system First audio output of change；And

(ii) second sound information is provided to audio system, showing with presentation with second can indicate that corresponding second audio is defeated Out, wherein the second audio output has the second audio configuration file.In some embodiments, the second audio configuration file and One audio configuration file is identical (for example, each audio configuration file is defeated in response to the audio showing the activation that can be indicated and generating Default audio configuration file out).In some embodiments, the second audio configuration file and the first audio configuration file be not Together.Presentation is shown with first can indicate corresponding first audio output, rather than continue with the first audio configuration file to present First audio output, this makes audio output interfere less with user and is that user is more desirable.This then makes user more likely Utilize audio output function.

In some embodiments, in response to detecting that being directed toward second shows the can indicate second input, which determines warp Whether meet audio change standard (918) in the first audio output that the second time point was presented by audio system.

In some embodiments, audio change standard includes when the process between first time point and the second time point Time quantum is less than predetermined time quantum (for example, T_{Threshold value}, Fig. 7 B-7G) when the standard (920) that meets.

In some embodiments, the audio change standard include passed through since initiating the first audio output when Between the standard (922) that meets when being less than predetermined time quantum.

In some embodiments, audio change standard includes the magnitude when the first audio output under the second time point The standard (924) met when falling below predetermined magnitude.In some embodiments, whether the first audio output is determined Meet audio change standard and is included in predetermined time (for example, at second time point or close to the time at the second time point) Determine whether elapsed time is less than predetermined time threshold and/or determining first since initiating the first audio output Whether the magnitude of audio output drops below predetermined magnitude (for example, the initial magnitude or maximum of the first audio output The half of magnitude).

In response to detecting that being directed toward second shows that can indicate second inputs and according to the less than footsteps frequency change standard of determination, The equipment (926)：

(i) audio system is made to continue to present and show with first and can indicate corresponding and there is the first audio configuration file First audio output；And

(ii) second sound information is provided to audio system to show with second to present and can indicate corresponding third audio output, Wherein third audio output has third audio configuration file.In some embodiments, third audio configuration file and second Audio configuration file is identical.In some embodiments, the second audio configuration file is different from third audio configuration file.

In some embodiments, modified audio configuration file has and continues to use the first audio when audio system Configuration file generates the first audio output volume when the first audio output is presented compares reduced volume (for example, modified Audio configuration file includes the decaying for increasing the first audio output).In some embodiments, modified audio configuration file Pitch including the first modified audio output.In some embodiments, modified audio configuration file is in the equipment Detect that at least one period after the second input is interior with non-zero volume.

It should be appreciated that being only an example to the concrete order for the operation being described in Fig. 9 A-9C, it is not intended that table The bright order is the but one sequence that these operations can be performed.Those skilled in the art will recognize that various ways come to this Operation described in text is resequenced.In some embodiments, by one or more operations of method 900 with it is described herein One or more operative combinations of other methods (for example, method 800,840,854 and/or 875) are supplemented by them or are substituted.

For illustrative purposes, the description of front is described by reference to specific embodiment.However, example above The property shown discussion is not intended to exhausted or limits the invention to disclosed precise forms.According to above teaching content, very More modifications and variations are all possible.Selection and description embodiment are to most preferably illustrate the principle of the present invention And its practical application, so as to so that others skilled in the art most preferably can be suitable for being conceived using having The described embodiment of the invention and various of the various modifications of special-purpose.

Claims

1. a kind of method (854), including：

With display, touch sensitive surface and be configured as detection and the one or more of the intensity of the contact of the touch sensitive surface At the electronic equipment of sensor：

Display (855) includes the user interface of the expression of media item on the display；

It is characterized in that

While showing the user interface, in position corresponding with the expression of the media item on the touch sensitive surface It sets place's detection (856) and passes through input caused by contact；

In response to detecting through the input (857) caused by the contact：

Meet media cue standard according to the determination input, wherein the media cue standard includes when the contact has height The standard met when the characteristic strength of the first intensity threshold：

Start to play the corresponding portion of the media item；And

While media item is playing, as the characteristic strength of the contact changes and the media are dynamically changed One group of one or more audio attribute of item；And

It is unsatisfactory for the media cue standard according to the determination input, abandons the corresponding portion for starting to play the media item Divide and abandons described one group one changing and dynamically changing the media item with the characteristic strength of the contact Or multiple audio attributes.

2. according to the method described in claim 1, wherein the audio attribute of the media item includes volume, low-pass filter Cutoff frequency and/or equalizer setting (858).

3. method described in any one of -2 according to claim 1, including：

After starting to play the corresponding portion of the media item (862)：

The increase for detecting the characteristic strength of the contact detects the reduction of the characteristic strength of the contact later

Meet media preview standard according to the characteristic strength of the determination contact, the media preview standard includes working as examining Before the reduction for measuring the characteristic strength of the contact, the characteristic strength of the contact is increased to above than institute The standard met when stating higher second intensity threshold of the first intensity threshold, while the media item is playing, with The increase of the characteristic strength of the contact and in the first way come change the media item the audio attribute and with The reduction of the characteristic strength of the contact and so that the audio attribute is maintained at first state；And

It is unsatisfactory for the media preview standard according to the characteristic strength of the determination contact, is playing in the media item While, change the audio of the media item in the first way with the increase of the characteristic strength of the contact Attribute and change the sound of the media item with the reduction of the characteristic strength of the contact and in a second manner Frequency attribute.

4. according to the method described in claim 3, include, in response to detecting that the input meets the media preview standard, Show that the increase of the characteristic strength of the contact is incited somebody to action so that the equipment is relative to the media item on the display Execute the instruction (863) of selection operation.

5. the method according to any one of claim 3-4, including：

It is playing the media item simultaneously, is detecting the end of the input；

In response to detecting the end of the input, stop playing the media item (865).

6. according to the method described in claim 5, wherein stopping playing institute in response to detecting the end of the input Stating media item includes that the dimness of vision of the content of the expression of the neighbouring media item is made to invert and gradually change the sound One group of one or more attribute of frequency project is so that the media item played gradually fades out (866).

7. the method according to any one of claim 5-6, including (867)：

After in response to detecting the end of the input and stopping playing the media item, in the institute with the media item Stating indicates to detect at corresponding position through selection input caused by the second contact on the touch sensitive surface, wherein described the Two contacts do not have the characteristic strength for reaching first intensity threshold；And

In response to detecting the selection input, selection operation is executed for the media item.

8. method described in any one of -7 according to claim 1, including (864)：

After starting to play the corresponding portion of the media item, it is maintained at same on the touch sensitive surface in the contact When, detect the increase of the characteristic strength of the contact；And

In response to detecting the increase of the characteristic strength of the contact：

Meet media selection criteria according to the characteristic strength of the determination contact, the media selection criteria includes when described The characteristic strength of contact is greater than the standard met when selection intensity threshold value more higher than first intensity threshold, stops aobvious Show the corresponding portion of the media item and executes selection operation for the media item；And

It is unsatisfactory for the media selection criteria according to the characteristic strength of the determination contact, continues to play the media item The corresponding portion, without executing the selection operation.

9. method described in any one of -8 according to claim 1, described one group one or more including working as the media item When a audio attribute dynamically changes, the visual appearance of (859) user interface is dynamically changed.

10. according to the method described in claim 9, the visual appearance for wherein dynamically changing the user interface include with The increase of the characteristic strength of the contact and the size of the expression that increases (860) described media item.

11. the method according to any one of claim 9-10, wherein dynamically changing the described of the user interface Visual appearance includes dynamically changing the neighbouring of (861) described user interface as the characteristic strength of the contact changes The fuzzy quantity of a part of the expression of the media item.

12. method described in any one of -11 according to claim 1, wherein playing the same of background media on said device When, the input is arrived in detection (868)；

In response to detecting the input (869)：

Meet the media cue standard according to the determination input：

While playing the media item, as the characteristic strength of the contact changes and the background is dynamically changed One group of one or more audio attribute of media item；And

It is unsatisfactory for the media cue standard according to the determination input, continuously plays the background media without changing the back One group of one or more audio attribute of scape media.

13. the method according to claim 11, including (870)：

While playing the media item, the end of the input is detected；

In response to detecting the end of the input, restore to play the background media item, wherein described one group one or Multiple audio attributes return to their value before detecting the input.

14. a kind of electronic equipment (100), including：

Display (112)；

Touch sensitive surface (112)；

One or more sensors (165), one or more of sensors are configured as detection and connect with the touch sensitive surface The intensity of touching；

One or more processors (122)；

Memory (102)；With

One or more programs, wherein one or more of programs are stored in the memory and are configured as by described One or more processors execute, and one or more of programs include the instruction for following operation：

In response to detecting through the input (857) caused by the contact：

Start to play the corresponding portion of the media item；And

15. electronic equipment (100) according to claim 14, wherein the audio attribute of the media item includes sound Amount, the cutoff frequency of low-pass filter and/or equalizer setting (858).

16. electronic equipment described in any one of 4-15 (100) according to claim 1, one or more of programs include Instruction for following operation：

After starting to play the corresponding portion of the media item (862)：

17. electronic equipment (100) according to claim 16, one or more of programs include for following operation Instruction：In response to detecting that the input meets the media preview standard, the institute of the contact is shown on the display The increase for stating characteristic strength will make the equipment relative to the instruction (863) of media item execution selection operation.

18. electronic equipment described in any one of 6-17 (100) according to claim 1, one or more of programs include Instruction for following operation：

While playing the media item, the end of the input is detected；

19. electronic equipment (100) according to claim 18, wherein in response to detecting the end of the input and Stop playing the dimness of vision reversion that the media item includes the content of the expression for making the neighbouring media item and gradually Change one group of one or more attribute of the audio items so that the media item played gradually fades out (866).

20. electronic equipment described in any one of 8-19 (100) according to claim 1, one or more of programs include Instruction (867) for following operation：

21. electronic equipment described in any one of 4-20 (100) according to claim 1, one or more of programs include Instruction (864) for following operation：

22. electronic equipment described in any one of 4-21 (100) according to claim 1, one or more of programs include Instruction for following operation：When one group of one or more audio attribute of the media item dynamically changes, dynamic The visual appearance of ground change (859) described user interface.

23. electronic equipment (100) according to claim 22, wherein dynamically changing the vision of the user interface Appearance includes the size for the expression for increasing (860) described media item with the increase of the characteristic strength of the contact.

24. the electronic equipment (100) according to any one of claim 22-23, wherein dynamically changing the user The visual appearance at interface includes dynamically changing (861) described user as the characteristic strength of the contact changes The fuzzy quantity of a part of the expression of the neighbouring media item at interface.

25. electronic equipment described in any one of 4-24 (100) according to claim 1, wherein playing back on said device While scape media, the input is arrived in detection (868), and one or more of programs include the finger for following operation It enables：；

In response to detecting the input (869)：

Meet the media cue standard according to the determination input：

26. electronic equipment (100) according to claim 25, one or more of programs include being used for following operation (870) instruction：

While playing the media item, the end of the input is detected；

27. a kind of computer readable storage medium for storing one or more programs, one or more of programs include instruction, Described instruction is when by with display, touch sensitive surface and being configured as detection and one of the intensity of the contact of the touch sensitive surface Or multiple sensors electronic equipment execute when make the electronic equipment：

In response to detecting through the input (857) caused by the contact：

Start to play the corresponding portion of the media item；And

28. computer readable storage medium according to claim 27, wherein the audio attribute packet of the media item Include volume, the cutoff frequency of low-pass filter and/or equalizer setting (858).

29. the computer readable storage medium according to any one of claim 27-28, including instruction, described instruction Make the electronic equipment when being executed by the electronic equipment：

After starting to play the corresponding portion of the media item (862)：

30. computer readable storage medium according to claim 29, including instruction, described instruction is worked as to be set by the electronics Make the electronic equipment when standby execution：In response to detecting that the input meets the media preview standard, in the display Show that the increase of the characteristic strength of the contact will be so that the equipment executes selection behaviour relative to the media item on device The instruction (863) of work.

31. the computer readable storage medium according to any one of claim 29-30, including instruction, described instruction Make the electronic equipment when being executed by the electronic equipment：

While playing the media item, the end of the input is detected；

32. computer readable storage medium according to claim 31, wherein in response to detecting described in the input Terminate and stops playing the media item including inverting the dimness of vision of the content of the expression of the neighbouring media item simultaneously And one group of one or more attribute of the audio items is gradually changed so that the media item played is gradually faded out (866)。

33. the computer readable storage medium according to any one of claim 31-32, including instruction, described instruction Make the electronic equipment (867) when being executed by the electronic equipment：

34. the computer readable storage medium according to any one of claim 27-33, including instruction, described instruction Make the electronic equipment (864) when being executed by the electronic equipment：

35. the computer readable storage medium according to any one of claim 27-34, including instruction, described instruction Make the electronic equipment when being executed by the electronic equipment：When one group of one or more audio category of the media item When property dynamically changes, the visual appearance of (859) user interface is dynamically changed.

36. computer readable storage medium according to claim 35, wherein dynamically changing the institute of the user interface State the ruler that visual appearance includes the expression for increasing (860) described media item with the increase of the characteristic strength of the contact It is very little.

37. the computer readable storage medium according to any one of claim 35-36, wherein dynamically changing described The visual appearance of user interface includes dynamically changing described in (861) as the characteristic strength of the contact changes The fuzzy quantity of a part of the expression of the neighbouring media item of user interface.

38. the computer readable storage medium according to any one of claim 27-37, wherein broadcasting on said device While putting background media, the input is arrived in detection (868), and the computer readable storage medium includes instruction, described Instruction makes the electronic equipment when being executed by the electronic equipment：

In response to detecting the input (869)：

Meet the media cue standard according to the determination input：

39. the computer readable storage medium according to claim 38, the computer readable storage medium includes instruction, Described instruction makes the electronic equipment (870) when being executed by the electronic equipment：

While playing the media item, the end of the input is detected；

40. a kind of electronic equipment, including：

Display；

Touch sensitive surface；

One or more sensors, one or more of sensors are configured as detecting strong with the contact of the touch sensitive surface Degree；With

For showing the device of the user interface of the expression including media item on the display；

What is enabled when while showing the user interface is used for described on the touch sensitive surface and media item Indicate the device inputted caused by the contact at corresponding position；

In response to detecting the device for performing the following operation by input enabling caused by the contact：

Start to play the corresponding portion of the media item；And

41. electronic equipment according to claim 40, wherein the audio attribute of the media item includes volume, low pass The cutoff frequency and/or equalizer setting (858) of filter.

42. the electronic equipment according to any one of claim 40-41, including：

After starting to play the corresponding portion of the media item (862), the device for performing the following operation of enabling：

43. electronic equipment according to claim 42, including in response to detecting that the input meets the media preview The increase of standard, the characteristic strength for showing the contact on the display of enabling will be so that the equipment phase The device of the instruction (863) of selection operation is executed for the media item.

44. the electronic equipment according to any one of claim 42-43, including：

While playing the media item, the device of the end for detecting the input of enabling；

In response to detecting the end of the input, the device for being used to stop playing the media item (865) of enabling.

45. electronic equipment according to claim 44, wherein in response to detecting the end of the input and The device of the stopping broadcasting media item includes that the dimness of vision for making the content of the expression adjacent to the media item is anti- The device turned and one group of one or more attribute for gradually changing the audio items are so that the media played Item gradually fades out the device of (866).

46. the electronic equipment according to any one of claim 44-45, including：

In response to detecting the end of the input and after stopping playing the media item, enabling for it is described It is detected at the corresponding position of the expression of media item through selection input caused by the second contact on the touch sensitive surface Device, wherein second contact, which does not have, reaches the characteristic strength of first intensity threshold；And

In response to detecting the selection input, the device for being used to execute selection operation for the media item of enabling.

47. the electronic equipment according to any one of claim 40-46, including (864)：

After starting to play the corresponding portion of the media item, enabling is used to be maintained at the touch-sensitive table in the contact While on face, the device of the increase of the characteristic strength of the contact is detected；And

In response to detecting the increase of the characteristic strength of the contact, the dress for performing the following operation of enabling It sets：

48. the electronic equipment according to any one of claim 40-47, including the device performed the following operation：Work as institute When stating one group of one or more audio attribute of media item and dynamically changing, (859) described user interface is dynamically changed Visual appearance.

49. electronic equipment according to claim 48, wherein the vision for dynamically changing the user interface The device of appearance includes the expression for increasing (860) media item for the increase with the characteristic strength of the contact Size device.

50. the electronic equipment according to any one of claim 48-49, wherein for dynamically changing user circle The device of the visual appearance in face includes changing for the characteristic strength with the contact and dynamically changing (861) The device of the fuzzy quantity of a part of the expression of the neighbouring media item of the user interface.

51. the electronic equipment according to any one of claim 40-50, wherein playing background media on said device While, the input is arrived in detection (868), and the electronic equipment includes：

In response to detecting the input (869), the device for performing the following operation of enabling：

Meet the media cue standard according to the determination input：

52. electronic equipment according to claim 51, including (870)：

In response to detecting the end of the input, enabling for restoring to play the device of the background media item, Described in one group of one or more audio attribute their value is returned to before detecting the input.

53. a kind of electronic equipment, including：

Display unit, the display unit are configured as display user interface；

Touch sensitive surface unit, the touch sensitive surface unit are configured as receiving contact；

One or more sensors unit, one or more of sensor units are configured as detection and the touch sensitive surface list The intensity of the contact of member；With

The processing unit coupled with the display unit, the touch sensitive surface unit and one or more of sensor units, The processing unit is configured as：

Display includes the user interface of the expression of media item on the display；

While showing the user interface, in position corresponding with the expression of the media item on the touch sensitive surface It sets place's detection and passes through input caused by contact；

In response to detecting through the input caused by the contact：

Start to play the corresponding portion of the media item；And

54. electronic equipment according to claim 53, wherein the audio attribute of the media item includes volume, low pass The cutoff frequency and/or equalizer setting of filter.

55. the electronic equipment according to any one of claim 53-55, wherein the processing unit is additionally configured to：

After starting to play the corresponding portion of the media item：

56. electronic equipment according to claim 55, wherein the processing unit is additionally configured to：

In response to detecting that the input meets the media preview standard, the institute of the contact is shown on the display unit The increase for stating characteristic strength will make the equipment relative to the instruction of media item execution selection operation.

57. the electronic equipment according to any one of claim 55-56, wherein the processing unit is additionally configured to：

While playing the media item, the end of the input is detected；

In response to detecting the end of the input, stop playing the media item.

58. electronic equipment according to claim 57, wherein stopping in response to detecting the end of the input Playing the media item includes that the dimness of vision of the content of the expression of the neighbouring media item is made to invert and gradually change One group of one or more attribute of the audio items is so that the media item played is gradually faded out.

59. the electronic equipment according to any one of claim 57-58, wherein the processing unit is additionally configured to：

60. the electronic equipment according to any one of claim 53-59, wherein the processing unit is additionally configured to：

After starting to play the corresponding portion of the media item, it is maintained on the touch sensitive surface unit in the contact Meanwhile detecting the increase of the characteristic strength of the contact；And

61. the electronic equipment according to any one of claim 53-60, wherein the processing unit is additionally configured to： When one group of one or more audio attribute of the media item dynamically changes, (859) described user is dynamically changed The visual appearance at interface.

62. electronic equipment according to claim 61, wherein dynamically changing the visual appearance of the user interface The size for the expression for increasing the media item including the increase with the characteristic strength of the contact.

63. the electronic equipment (100) according to any one of claim 61-62, wherein dynamically changing the user The visual appearance at interface includes dynamically changing the user interface as the characteristic strength of the contact changes The neighbouring media item the expression a part fuzzy quantity.

64. the electronic equipment according to any one of claim 53-63, wherein playing background media on said device While, detect the input, and

The processing unit is additionally configured to：

In response to detecting the input：

Meet the media cue standard according to the determination input：

65. electronic equipment according to claim 64, the processing unit is additionally configured to：

While playing the media item, the end of the input is detected；And