JP6545155B2 - Device, method and graphical user interface for synchronizing two or more displays - Google Patents

Description

Cross-reference to related applications

  This application is related to the following applications: US Provisional Application No. 61 / 793,924 entitled "Voice and Touch User Interface", filed March 15, 2013, "Pushing a Graphical User Interface to a Remote Device with Display", filed February 11, 2011 No. 13 / 032,614, entitled "Rules Provided by the Remote Device", and U.S. Application No. 12/683, entitled "Pushing a User Interface to a Remote Device", filed Jan. 6, 2010. "Pushing a User Interface to a Remote Device" filed on May 13, 2008, No. 218 No. 12 / 119,960 entitled "Pushing a User Interface to a Remote Device" filed Jul. 1, 2011 No. 13 / 175,581 entitled "Pushing a User Interface to a Remote Device", June 15, 2011 No. 13/16 1,339 entitled “Pushing a Graphical User Interface to a Remote Device with Display Rules Provided by the Remote Device”, filed on Sep. 30, 2011, “Automatically Adapting User Interfaces for September 30, 2011 No. 13 / 250,947 entitled "Hands-Free Interaction", 2 US application Ser. No. 12 / 987,982 entitled “Intelligent Automated Assistant” filed on Jan. 10, 2011, and US Provisional Application Ser. No. 61/2006 entitled “Intelligent Automated Assistant” filed on Jan. 18, 2010 No. 295,774, filed on Jun. 3, 2011, entitled "Generating and Processing Data Items that Representative Tasks to Perform", and provisionally filed US Provisional Application No. 61 / 493,201, filed on Jun. 9, 2012, "Automatically US Provisional Application No. 61 / 657,744 entitled "Adapting User Interface for Hands-Free Interaction" No. 12 / 207,316, entitled "Radio with Personal DJ", filed Sep. 9, 2008, "System and Method for Negotiation Control of a Shared Audio or Visual Application" filed Nov. 16, 2012. US Provisional Application No. 61 / 727,554 entitled “Resource”, US Application Ser. No. 61 / 832,818 entitled “Mapping Application with Several User Interfaces”, filed Jun. 8, 2013, 2013 6 Titled “Device and Method for Generating User Interfaces from a Template” U.S. Provisional Application No. 61 / 832,841, filed on June 8, 2013, entitled "Application Gateway for Providing Different User Interfaces for Limited Distraction and Non-Limited Distraction Contexts", U.S. Application No. 13 / 913,428 . These applications are incorporated herein by reference in their entirety.

  The present application relates generally to an electronic device with a touch sensitive surface, including but not limited to an electronic device with a touch sensitive surface that synchronizes two or more displays.

Background of the invention

  The use of touch sensitive surfaces as input devices for computers and other electronic computing devices has increased significantly in recent years. Exemplary touch sensitive surfaces include touch pads and touch screen displays. Such surfaces are widely used to manipulate user interface objects on a first display associated with a first electronic device.

  Exemplary operations include adjusting the position and / or size of one or more user interface objects, activating a button, opening a file / application represented by a user interface object, or other method Operating the user interface. Exemplary user interface objects include digital images, video, text, icons, control elements such as buttons, and other graphics.

  A user may have access to multiple different displays that can display different content. For example, a passenger in a car or other vehicle may have access to a smartphone and its display, as well as a car information / entertainment system and its display (s). However, when multiple different displays are available, the method for displaying content on these different displays is cumbersome and inefficient. For example, displaying the same information on different displays does not generally provide the user with useful additional information, and displaying irrelevant information on different displays is confusingly and inefficient for the user, and in some cases Will spend extra time for the user to keep displaying related content on both devices. In addition, these methods take longer than necessary, thereby wasting energy. This latter consideration is particularly important in battery powered devices.

  Thus, there is a need for an electronic device that provides more efficient methods and interfaces for synchronizing two or more displays. Such methods and interfaces optionally complement or replace conventional methods for displaying information on more than one display. Such methods and interfaces reduce the cognitive burden on the user and create a more efficient human-machine interface. For battery powered devices, such methods and interfaces conserve power and extend the time between battery charges.

  Furthermore, in many situations, controls for use in interacting with such user interfaces displayed on the display are often tightly associated with a single display. This limits the flexibility of the control, which may reduce the efficiency of the user interface. Thus, there is a need for a system that continuously maps controls and other input sources to user interfaces displayed in a flexible manner on different displays. However, when you map different controls to different displays, the controls may be mistakenly associated with the wrong display. Furthermore, taking a long time to determine the display associated with a particular control may cause delays in the user's interaction with the user interface, thereby degrading the quality of the user experience. That is, there is a need for a fast, accurate, and efficient system and method for mapping controls and other input sources to different ones of two or more displays.

  The above-described deficiencies and other issues associated with user interfaces for electronic devices with touch sensitive surfaces are reduced or eliminated by the disclosed devices. In some embodiments, the device is a desktop computer. In some embodiments, the device is portable (eg, a notebook computer, a tablet computer, or a handheld device). In some embodiments, the device comprises a touch pad. In some embodiments, the device has a touch sensitive display (also known as a "touch screen" or "touch screen display"). In some embodiments, the device includes a graphical user interface (GUI), one or more processors, a memory, and one or more modules, programs, or programs stored in the memory to perform multiple functions. Have an instruction set. In some embodiments, the user interacts with the GUI primarily through finger contact and gestures on the touch sensitive surface. In some embodiments, those functions optionally include image editing, drawing, presenting, word processing, website creation, disc authoring, spreadsheet creation, game play, making phone calls, Video conferencing, emailing, instant messaging, training support, digital photography, digital video photography, web browsing, digital music playback, and / or digital video playback. Executable instructions for performing these functions are optionally included in a non-transitory computer readable storage medium or other computer program product configured for execution by one or more processors.

  According to some embodiments, a method is performed on a first electronic device comprising a first display. The method includes displaying a first user interface on a first display, the first user interface enabling selection between a plurality of selectable objects, the first user interface being a first display Corresponding to a second user interface displayed on a second display different from the first one of the plurality of selectable objects is displayed in the second user interface as the selected object . The method displays in the first user interface information regarding a second one of the plurality of selectable objects while the first object is displayed in the second user interface as the selected object. A first object as a selected object in a second user interface in response to detecting a first input from the user, and in response to detecting the first input, corresponding to the request for Updating the first user interface on the first display to display corresponding information on the second object according to the first input while maintaining the display of Required to select the second object as the selected object after updating the It corresponds to includes detecting a second input from the user. The method updates the first user interface on the first display to display the second object as the selected object according to the second input in response to detecting the second input. And providing information to the second display that allows the second user interface on the second display to be updated to display the second object as a selected object instead of the first object Further including:

  According to some embodiments, the first electronic device comprises a first display unit configured to display a first user interface on a first display unit, the first user interface comprising The selection between a plurality of selectable objects is possible, the first user interface corresponding to a second user interface displayed on a second display unit different from the first display unit, and a plurality of selectable The first of the objects is displayed in the second user interface as the selected object. The first electronic device also includes a processing unit coupled to the first display unit. The processing unit may, while the first object is displayed as the selected object in the second user interface, of information on a second one of the plurality of selectable objects in the first user interface. Selecting a first object into a second user interface in response to detecting a first input from the user corresponding to the request to enable display and detecting the first input; Updating the first user interface on the first display unit to enable the display of the corresponding information on the second object according to the first input, while maintaining the display as the first object; After updating the first user interface according to the input of the second object with the selected object It corresponds to a request to select Te, configured to detect a second input from the user. The processing unit is responsive to detecting the second input to cause the display of the second object as the selected object according to the second input, the first on the first display unit To update the second user interface on the second display unit to update the user interface of the second display to allow the display of the second object as a selected object instead of the first object Are further configured to provide information to the second display unit.

  According to some embodiments, the method is implemented in a portable electronic device comprising a portable device display and in communication with a vehicle display of a vehicle. The method comprises obtaining a set of map search results comprising a plurality of destination candidates and displaying on the portable device display a portable device navigation interface enabling selection of a destination from the plurality of destination candidates. Transmitting to the vehicle display information enabling the vehicle display to display the vehicle navigation interface, the first destination of the plurality of possible destinations being selected in the vehicle navigation interface Is a ground, including transmitting. The method displays information about one or more destination candidates in a plurality of destination candidates different from the first destination while the first destination is a selected destination in the vehicle navigation interface According to the first input without updating the vehicle navigation interface on the vehicle display in response to detecting the first input from the user and to detecting the first input corresponding to the request for Updating the portable device navigation interface on the portable device display, and updating the portable device navigation interface according to the first input, then selecting a second destination among the plurality of possible destinations as the selected destination And detecting a second input from the user corresponding to the request. The method updates the portable device navigation interface on the portable device display to display the second destination as the selected destination according to the second input in response to detecting the second input. And transmitting information to the vehicle display enabling the vehicle display to update the vehicle navigation interface to display the second destination as the selected destination according to the second input. Further include.

  According to some embodiments, the portable electronic device comprises a portable device display unit configured to display a portable device navigation interface, and a communication interface unit configured to communicate with a vehicle display unit of a vehicle. A processing unit coupled to the portable device display unit and the communication interface unit. The processing unit is capable of obtaining a set of map search results including a plurality of potential destinations and displaying on a portable device display unit a portable device navigation interface enabling selection of destinations from a plurality of potential destinations Information to enable the vehicle display unit to enable display of the vehicle navigation interface to the vehicle display unit, the first destination of the plurality of possible destinations being selected in the vehicle navigation interface Configured to be a destination. The processing unit also relates to one or more destination candidates out of a plurality of destination candidates different from the first destination while the first destination is a selected destination in the vehicle navigation interface. A first input from a user is detected in response to a request to enable display of information, and in response to detecting the first input, without updating the vehicle navigation interface on the vehicle display unit Updating the portable device navigation interface on the portable device display unit according to the first input, and updating the portable device navigation interface according to the first input, and then selecting a second destination of the plurality of possible destinations The second input from the user, corresponding to the request to select as the destination Configured to detect. The processing unit is portable on the portable device display unit to enable the display of the second destination as the selected destination according to the second input in response to detecting the second input. Information enabling the device navigation interface to be updated and the vehicle display unit to update the vehicle navigation interface to enable the display of the second destination as a selected destination according to the second input Are further configured to transmit to the vehicle display unit.

  According to some embodiments, the method is performed at an electronic device in communication with a plurality of displays, including a first display and a second display. The method includes receiving first input information indicative of a first input detected by a first input source of the plurality of input sources, the first input information being to the first input source. A first identifier and a first input description describing a first input. The method also includes selecting a first affected display from the plurality of displays based on the first identifier and an input source mapping that maps the input source to the corresponding display. The method further includes transmitting, to the first affected display, update information that enables the first affected display to be updated according to the first input.

  According to some embodiments, an electronic device is coupled to a communication interface unit configured to communicate with a plurality of display units, including a first display unit and a second display unit. And a processing unit. The processing unit is configured to receive first input information indicative of a first input detected by a first input source of the plurality of input sources, the first input information being a first input A first identifier for the source and a first input description describing the first input. The processing unit is also configured to select the first affected display from the plurality of displays based on the first identifier and the input source mapping that maps the input source to the corresponding display. The processing unit is further configured to send update information to the first affected display enabling to update the first affected display according to the first input.

  According to some embodiments, the electronic device is a communication interface for communicating with one or more displays (eg, a first display, a second display, a portable device display, and / or a vehicle display); One or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more The program comprises instructions for performing the operations of any of the methods described herein. According to some embodiments, the non-transitory computer readable storage medium stores instructions therein, the instructions being executed by an electronic device in communication with the one or more displays. Cause the device to perform the operations of any of the methods referenced and described herein. According to some embodiments, on an electronic device in communication with one or more displays, the memory and one or more processors for executing one or more programs stored in the memory The graphical user interface includes one or more of the elements displayed in any of the methods described above, which elements are described in any of the methods described herein. As it is, it is updated according to the input. According to some embodiments, an electronic device in communication with one or more displays includes means for performing the operations of any of the methods described herein. According to some embodiments, an information processing apparatus for use in an electronic device in communication with one or more displays performs the operations of any of the methods described herein. including.

  Thus, the effectiveness, efficiency, and user satisfaction of these electronic devices with multiple displays by providing more efficient methods and interfaces for synchronizing two or more displays. The degree can be increased. Such methods and interfaces may complement or replace conventional methods for synchronizing two or more displays.

  For a better understanding of the various embodiments described, reference should be made to the following description of the embodiments in conjunction with the following drawings, like reference numerals referring to corresponding parts throughout the drawings Point to.

FIG. 1 is a block diagram illustrating a portable multifunction device with a touch sensitive display in accordance with some embodiments.

FIG. 6 is a block diagram illustrating exemplary components for event processing, according to some embodiments.

FIG. 1 illustrates a portable multifunction device with a touch screen in accordance with some embodiments.

FIG. 7 is a block diagram of an exemplary multifunction device with a display and touch sensitive surface in accordance with some embodiments.

FIG. 6 illustrates an exemplary user interface for a menu of applications on a portable multifunction device, in accordance with some embodiments.

FIG. 10 illustrates an exemplary user interface for a multifunction device with a separate touch sensing surface, in accordance with some embodiments.

FIG. 5 is a block diagram illustrating an operating environment in which a portable multifunction device communicates with an external provisioning system and / or a server, according to some embodiments.

7 is a flow chart illustrating a method of sending update information to affected displays according to some embodiments.

FIG. 10 illustrates an exemplary user interface for synchronizing two or more displays, in accordance with some embodiments. FIG. 10 illustrates an exemplary user interface for synchronizing two or more displays, in accordance with some embodiments. FIG. 10 illustrates an exemplary user interface for synchronizing two or more displays, in accordance with some embodiments. FIG. 10 illustrates an exemplary user interface for synchronizing two or more displays, in accordance with some embodiments. FIG. 10 illustrates an exemplary user interface for synchronizing two or more displays, in accordance with some embodiments. FIG. 10 illustrates an exemplary user interface for synchronizing two or more displays, in accordance with some embodiments. FIG. 10 illustrates an exemplary user interface for synchronizing two or more displays, in accordance with some embodiments. FIG. 10 illustrates an exemplary user interface for synchronizing two or more displays, in accordance with some embodiments. FIG. 10 illustrates an exemplary user interface for synchronizing two or more displays, in accordance with some embodiments. FIG. 10 illustrates an exemplary user interface for synchronizing two or more displays, in accordance with some embodiments. FIG. 10 illustrates an exemplary user interface for synchronizing two or more displays, in accordance with some embodiments. FIG. 10 illustrates an exemplary user interface for synchronizing two or more displays, in accordance with some embodiments. FIG. 10 illustrates an exemplary user interface for synchronizing two or more displays, in accordance with some embodiments. FIG. 10 illustrates an exemplary user interface for synchronizing two or more displays, in accordance with some embodiments. FIG. 10 illustrates an exemplary user interface for synchronizing two or more displays, in accordance with some embodiments. FIG. 10 illustrates an exemplary user interface for synchronizing two or more displays, in accordance with some embodiments. FIG. 10 illustrates an exemplary user interface for synchronizing two or more displays, in accordance with some embodiments. FIG. 10 illustrates an exemplary user interface for synchronizing two or more displays, in accordance with some embodiments. FIG. 10 illustrates an exemplary user interface for synchronizing two or more displays, in accordance with some embodiments. FIG. 10 illustrates an exemplary user interface for synchronizing two or more displays, in accordance with some embodiments. FIG. 10 illustrates an exemplary user interface for synchronizing two or more displays, in accordance with some embodiments. FIG. 10 illustrates an exemplary user interface for synchronizing two or more displays, in accordance with some embodiments.

8 illustrates an exemplary user interface for sending update information to affected displays, according to some embodiments. 8 illustrates an exemplary user interface for sending update information to affected displays, according to some embodiments. 8 illustrates an exemplary user interface for sending update information to affected displays, according to some embodiments. 8 illustrates an exemplary user interface for sending update information to affected displays, according to some embodiments. 8 illustrates an exemplary user interface for sending update information to affected displays, according to some embodiments. 8 illustrates an exemplary user interface for sending update information to affected displays, according to some embodiments. 8 illustrates an exemplary user interface for sending update information to affected displays, according to some embodiments. 8 illustrates an exemplary user interface for sending update information to affected displays, according to some embodiments. 8 illustrates an exemplary user interface for sending update information to affected displays, according to some embodiments. 8 illustrates an exemplary user interface for sending update information to affected displays, according to some embodiments.

5 is a flow chart illustrating a method of synchronizing two or more displays according to some embodiments. 5 is a flow chart illustrating a method of synchronizing two or more displays according to some embodiments. 5 is a flow chart illustrating a method of synchronizing two or more displays according to some embodiments. 5 is a flow chart illustrating a method of synchronizing two or more displays according to some embodiments.

5 is a flow chart illustrating a method of synchronizing two or more displays according to some embodiments. 5 is a flow chart illustrating a method of synchronizing two or more displays according to some embodiments.

7 is a flow chart illustrating a method of sending update information to affected displays according to some embodiments. 7 is a flow chart illustrating a method of sending update information to affected displays according to some embodiments. 7 is a flow chart illustrating a method of sending update information to affected displays according to some embodiments.

FIG. 7 is a functional block diagram of an electronic device, according to some embodiments.

FIG. 7 is a functional block diagram of an electronic device, according to some embodiments.

FIG. 7 is a functional block diagram of an electronic device, according to some embodiments.

  Many electronic devices update the graphical user interface on the display associated with the electronic device in response to the manipulation of the user interface object displayed on the display. The first electronic device described below comprises a second graphical interface on the second display, a first graphical user interface on the first display associated with the first electronic device Improves these methods by synchronizing. In some embodiments, a second display is associated with a second electronic device that is different from the first electronic device. In some embodiments, the first electronic device operates the user interface object on the first graphical user interface (eg, a first touch-sensitive surface or a first touch associated with the first electronic device) Synchronize the first display and the second display according to the operation of the screen display). In some other embodiments, the first electronic device operates the user interface object on the second graphical user interface (e.g., a second touch sensitive surface or a second associated with the second electronic device) Synchronize the first display and the second display in response to the operation of the touch screen display of

In the following, FIGS. 1A-1B, 2 and 3 provide a description of an exemplary device. 4A-4B and 6A-6FF illustrate exemplary user interfaces. 7A-7D and 8A-8B are flowcharts illustrating a method of synchronizing two or more displays. 9A-9C are flow diagrams illustrating a method of sending update information to affected displays. The user interfaces of FIGS. 6A-6V are used to illustrate the processes of FIGS. 7A-7D and 8A-8B. The user interfaces of FIGS. 6W-6FF are used to illustrate the processes of FIGS. 9A-9C.
Example device

  Reference will now be made in detail to the embodiments, examples of which are illustrated in the accompanying drawings. In the following detailed description, numerous specific details are set forth in order to provide a thorough understanding of the various embodiments set forth. However, it will be apparent to those skilled in the art that the various embodiments described may be practiced without these specific details. In other instances, well-known methods, procedures, components, circuits, and networks have not been described in detail so as not to unnecessarily obscure aspects of the embodiments.

  Although the terms first, second, etc. are used herein to describe various elements in some instances, these elements should not be limited by those terms. Will also be understood. These terms are only used to distinguish one element from another. For example, without departing from the scope of the various embodiments described, the first contact may be referred to as a second contact, and similarly, the second contact may be referred to as a first contact. Will be able to The first and second contacts are both contacts, but they are not the same contact.

  The terminology used in the description of the various embodiments described herein is for the purpose of describing particular embodiments only and is not intended to be limiting. As used in the various described embodiments and in the appended claims, the singular forms "a", "an" and "the" also include the plural, unless the context clearly indicates otherwise It is intended. It is also understood that the term "and / or" as used herein refers to and includes any and all possible combinations of one or more of the associated listed items. I want to be The terms "includes", "including", "comprises" and / or "comprising", as used herein, refer to the recited features, integers, steps , Specify the presence of an operation, element, and / or component, but exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof It will be further understood that not.

  As used herein, the term "if (in the case of)" is optionally depending on the context "when (when)", "upon", "in It is understood to mean response to determining (in response to a determination of) or "in response to detecting" (in response to detection of). Similarly, the phrase "if it is determined" or "if (a stated condition or event) is detected" (if the stated condition or event is detected) means the context "Upon determining", "in response to determining", and "upon detected (the stated condition or event)" ((stated condition or event). Detection) or “in response to detection (in response to detection of the stated condition or event) To mean "being optionally interpretation.

  Embodiments of electronic devices, user interfaces for such devices, and associated processes for using such devices are described. In some embodiments, the device is a portable communication device such as a mobile telephone that also includes other features, such as PDA capabilities and / or music playback capabilities. An exemplary embodiment of the portable multifunction device is, without limitation, Apple Inc. of Cupertino, California. IPhone®, iPod Touch®, and iPad® devices. Other portable electronic devices, such as laptop computers or tablet computers with touch sensitive surfaces (eg, touch screen displays and / or touch pads) are optionally used. Also, it should be understood that in some embodiments, the device is not a portable communication device but a desktop computer with a touch sensitive surface (eg, a touch screen display and / or a touch pad).

  The following discussion describes an electronic device that includes a display and a touch sensitive surface. However, it should be understood that the electronic device optionally includes one or more other physical user interface devices, such as a physical keyboard, a mouse, and / or a joystick.

  The devices are generally drawing applications, presentation applications, word processing applications, website creation applications, disc authoring applications, spreadsheet applications, gaming applications, telephone applications, video conferencing applications, email applications, instant messaging applications, training Support various applications such as one or more of support application, photo management application, digital camera application, digital video camera application, web browsing application, digital music playback application, and / or digital video playback application To over door.

  Various applications running on the device optionally use at least one common physical user interface device, such as a touch sensitive surface. One or more features of the touch sensitive surface, as well as corresponding information displayed on the device, are optionally adjusted and / or changed, per application and / or within the corresponding application. In this manner, the common physical architecture (such as the touch sensitive surface) of the device optionally supports various applications that have a user interface that is intuitive and transparent to the user.

  Attention is now directed to the embodiment of the portable device comprising the touch sensitive display. FIG. 1A is a block diagram illustrating a portable multifunction device 100 with a touch sensitive display 112 in accordance with some embodiments. Touch sensitive display 112 is sometimes referred to for convenience as a "touch screen" and may be known or referred to as a "touch sensitive display system". Device 100 includes memory 102 (optionally including one or more computer readable storage media), memory controller 122, one or more processing units (CPU) 120, peripherals interface 118, RF circuitry 108, audio circuitry. 110, a speaker 111, a microphone 113, an input / output (I / O) subsystem 106, other input or control devices 116, and an external port 124. Device 100 optionally includes one or more light sensors 164. Device 100 optionally includes one or more intensity sensors 165 for detecting the intensity of a touch on device 100 (eg, a touch sensitive surface such as touch sensitive display system 112 of device 100). Device 100 generates tactile output on device 100 (eg, generates tactile output on a touch-sensitive surface, such as touch-sensitive display system 112 of device 100 or touch pad 355 of device 300) Optionally, one or more tactile output generators 167 are included. These components optionally communicate via one or more communication buses or signal lines 103.

  As used herein and in the claims, the term "intensity" of touch on a touch sensitive surface refers to the force or pressure (per unit area) of the touch (eg, finger touch) on the touch sensitive surface Or touch force or pressure substitute (substitute) on a touch sensitive surface. The strength of the contact has a range of values, including at least four distinct values, and more typically including hundreds of distinct values (e.g., at least 256). The strength of the contact is optionally determined (or measured) using different techniques and different sensors or combinations of sensors. For example, one or more sensors underlying or adjacent to the touch sensitive surface are optionally used to measure forces at various points on the touch sensitive surface. In some implementations, force measurements from multiple force sensors are combined (e.g., weighted average) to determine an estimated contact force. Similarly, a pressure sensitive tip of the stylus is optionally used to determine the pressure of the stylus on the touch sensitive surface. Alternatively, the size of the contact area detected on the touch sensitive surface and / or its change, the capacitance of the touch sensitive surface close to the contact and / or its change, and / or the resistance of the touch sensitive surface close to the contact and And / or its change is optionally used as a substitute for the force or pressure of contact on the touch sensitive surface. In some implementations, an alternative measurement of contact force or pressure is used directly to determine whether the intensity threshold is exceeded (eg, the intensity threshold is in units corresponding to the alternative measurement) Described). In some implementations, an alternative measurement of contact force or pressure is converted to an estimated force or pressure, and this estimated force or pressure is used to determine if an intensity threshold is exceeded. (For example, the intensity threshold is a pressure threshold measured in units of pressure).

  As used herein and in the appended claims, the term "pale output" refers to the physical displacement of the device relative to the previous position of the device, which will be detected by the user at the user's touch, configuration of the device Refers to physical displacement of an element (e.g., touch sensitive surface) relative to another component of the device (e.g., housing) or displacement of the component relative to the center of mass of the device. For example, in situations where the device, or a component of the device, is in contact with a touch-sensitive user's surface (eg, a finger, palm, or other part of the user's hand), the touch produced by physical displacement. Knowledge output will be interpreted by the user as tactile sensations corresponding to perceived changes in the physical properties of the device or of the components of the device. For example, movement of the touch sensitive surface (e.g., touch sensitive display or trackpad) is optionally interpreted by the user as a "down click" or "up click" of the physical actuator button. In some cases, even if there is no movement of the physical actuator button associated with the touch sensitive surface, the user is physically pressed (e.g. displaced) by the movement of the user. You will feel a tactile sensation such as “down click” or “up click”. As another example, movement of the touch sensitive surface may optionally be interpreted by the user as “roughness” of the touch sensitive surface, even if there is no change in smoothness of the touch sensitive surface. Be sensed. Although the user's interpretation of such touch is dependent on the user's individual sensory perception, there are a lot of sensory perceptions of touch common to most users. Therefore, if the tactile output is described as corresponding to the user's particular sensory perception (e.g. "up click", "down click", "roughness"), then the touch generated unless otherwise stated The knowledge output corresponds to the physical displacement of the device or its components that causes the described sensory perception to occur to a typical (or average) user.

  Device 100 is just one example of a portable multifunction device, and device 100 optionally has more or fewer components than those shown, or optionally more than one component. It should be understood that they optionally have different configurations or arrangements of combinations or components. The various components shown in FIG. 1A are implemented in the form of hardware, software, or a combination of both hardware and software, including one or more signal processing circuits and / or application specific integrated circuits.

  Memory 102 optionally includes high-speed random access memory, and optionally also includes non-volatile memory, such as one or more magnetic disk storage devices, flash memory devices, or other non-volatile semiconductor memory devices. . Access to memory 102 by other components of device 100, such as CPU 120 and peripherals interface 118, is optionally controlled by memory controller 122.

  Peripheral interface 118 may be used to couple the input and output peripherals of this device to CPU 120 and memory 102. One or more processors 120 perform various functions related to device 100 and process data by operating or executing various software programs and / or instruction sets stored in memory 102.

  In some embodiments, peripherals interface 118, CPU 120, and memory controller 122 are optionally implemented on a single chip, such as chip 104. In some other embodiments, these are optionally implemented on separate chips.

  An RF (radio frequency) circuit 108 receives and transmits an RF signal, also called an electromagnetic signal. The RF circuitry 108 converts electrical signals to electromagnetic signals or electromagnetic signals to electrical signals and communicates with communication networks and other communication devices via the electromagnetic signals. The RF circuit 108 includes an antenna system, an RF transceiver, one or more amplifiers, a tuner, one or more oscillators, a digital signal processor, a CODEC chip set, a subscriber identity module (SIM) card, a memory, etc. Optionally includes known circuitry for performing the above functions, including but not limited to. The RF circuitry 108 may be an intranet and / or wireless such as a network such as the Internet, also referred to as the World Wide Web (WWW), a cellular telephone network, a wireless local area network (LAN), and / or a metropolitan area network (MAN). Optionally communicate with the network, as well as with other devices, by wireless communication. Wireless communication optionally uses any of a plurality of communication standards, communication protocols, and communication technologies, and as those communication standards, communication protocols, and communication technologies, a global system for mobile communication ( GSM (registered trademark), Extended Data GSM Environment (EDGE), High-Speed Downlink Packet Access (HSDPA), High-Speed Uplink Packet Access (HSUPA), Evolution, Data-Only (EV-DO), HSPA, HSPA +, Double Cell HSPA (DC-HSPDA), Long Term Evolution (LTE), Near Field Communication (NFC), Wideband Code Division Multiple Access (W-CDMA), Code Division Multiple Access (CDMA), Time Division Multiple Access (TDMA), Bluetooth (registered trademark), Wireless Fidel ty (Wi-Fi) ® (eg, IEEE 802.11a, IEEE 802.11b, IEEE 802.11g, and / or IEEE 802.11n), Voice over Internet Protocol (VoIP), Wi-MAX, electronic Protocols for mail (eg Internet Message Access Protocol (IMAP) and / or Post Office Protocol (POP)), Instant Messaging (eg Extensible Messaging and Presence Protocol (XMPP), Session Initiation Protocol for Instant Messaging and Presence Based Extensions (SIMPLE), Instant Messaging and Presence Service (IMPS), and / or Short Message Service (SMS), or Filed any other suitable communication protocol, including communication protocols not yet been developed to date document include, but are not limited to.

  Audio circuitry 110, speaker 111 and microphone 113 provide an audio interface between the user and device 100. The audio circuit 110 receives audio data from the peripheral device interface 118, converts the audio data into an electrical signal, and transmits the electrical signal to the speaker 111. The speaker 111 converts the electrical signal into a human audible sound wave. Audio circuit 110 also receives an electrical signal that has been converted from sound waves by microphone 113. Audio circuit 110 converts the electrical signal to audio data and sends the audio data to peripheral interface 118 for processing. Audio data is optionally obtained by the peripherals interface 118 from the memory 102 and / or the RF circuit 108 and / or transmitted to the memory 102 and / or the RF circuit 108. In some embodiments, audio circuit 110 further includes a headset jack (eg, 212 in FIG. 2). The headset jack is a removable audio input / output peripheral device such as an audio circuit 110 and a headset with both an output-only headphone or an output (eg, a single or double-eared headphone) and an input (eg, a microphone) To provide an interface between

  The I / O subsystem 106 couples to the peripherals interface 118 the input and output peripherals of the device 100 such as the touch screen 112 and other input control devices 116. The I / O subsystem 106 optionally includes one or more input controllers 160 for the display controller 156, the light sensor controller 158, the intensity sensor controller 159, the haptic feedback controller 161, and other input or control devices. Including. One or more input controllers 160 receive / transmit electrical signals from / to other input or control devices 116. Other input control devices 116 optionally include physical buttons (eg, push buttons, rocker buttons, etc.), dials, slider switches, joysticks, click wheels, etc. In some alternative embodiments, the input controller 160 is optionally coupled to (or not coupled to) any of a keyboard, an infrared port, a USB port, and a pointer device such as a mouse. ). One or more buttons (e.g., 208 in FIG. 2) optionally include up / down buttons for volume control of the speaker 111 and / or the microphone 113. The one or more buttons optionally include a push button (e.g., 206 of FIG. 2).

  Touch sensitive display 112 provides an input interface and an output interface between the device and the user. Display controller 156 receives and / or transmits electrical signals to and from touch screen 112. Touch screen 112 displays visual output to the user. Visual output optionally includes graphics, text, icons, videos, and any combination thereof (collectively referred to as "graphics"). In some embodiments, some or all of the visual output correspond to user interface objects.

  The touch screen 112 includes a touch sensitive surface, sensor or set of sensors that receives input from the user based on tactile and / or tactile contact. Touch screen 112 and display controller 156 (along with any associated modules and / or sets of instructions in memory 102) detect touch (and any movement or interruption of touch) on touch screen 112 and detect the detected touch. , Convert to an interaction with a user interface object (eg, one or more soft keys, icons, web pages, or images) displayed on the touch screen 112. In the exemplary embodiment, the contact points between the touch screen 112 and the user correspond to the user's finger.

  Touch screen 112 optionally uses LCD (liquid crystal display) technology, LPD (light emitting polymer display) technology, or LED (light emitting diode) technology, but in other embodiments other display technologies are used Ru. Touch screen 112 and display controller 156 are optionally, but not limited to, one or more of capacitive technology, resistive technology, infrared technology, and surface ultrasound technology, as well as touch screen 112. Contact, as well as any movement or interruption thereof, using any of a number of currently known or later developed touch sensing technologies, including other proximity sensor arrays or other elements to determine touch points. To detect In the exemplary embodiment, Apple Inc. Projected mutual-capacitive sensing techniques are used, such as those found on the iPhone (R), iPod Touch (R), and iPad (R) by (Cupertino, California).

  Touch screen 112 optionally has a video resolution greater than 100 dpi. In some embodiments, the touch screen has a video resolution of about 160 dpi. The user optionally contacts touch screen 112 using any suitable object or appendage, such as a stylus, a finger, and the like. In some embodiments, the user interface is designed to work primarily with finger-based touch and gestures and is less accurate than stylus-based input due to the large finger contact area on the touch screen is there. In some embodiments, the device translates the rough finger-based input into the correct pointer / cursor position or command to perform the action desired by the user.

  In some embodiments, in addition to the touch screen, device 100 optionally includes a touch pad (not shown) for activating or deactivating particular functions. In some embodiments, the touch pad is a touch sensitive area of the device that, unlike a touch screen, does not display visual output. The touch pad is optionally a touch sensitive surface separate from the touch screen 112 or an extension of the touch sensitive surface formed by the touch screen.

  Device 100 also includes a power system 162 for powering the various components. Power system 162 includes a power management system, one or more power supplies (eg, batteries, alternating current (AC)), recharge systems, power failure detection circuits, power converters or inverters, power status indicators (eg, light emitting diodes (LEDs)) And optionally any other components associated with power generation, management, and distribution of power in the portable device.

  Device 100 also optionally includes one or more light sensors 164. FIG. 1A shows a light sensor coupled to light sensor controller 158 in I / O subsystem 106. The light sensor 164 optionally includes a charge coupled device (CCD) or a complementary metal oxide semiconductor (CMOS) phototransistor. The light sensor 164 receives light from the environment projected through one or more lenses and converts the light into data representing an image. In conjunction with imaging module 143 (also referred to as a camera module), light sensor 164 optionally captures a still image or video. In some embodiments, the light sensor is disposed on the back of the device 100, opposite the touch screen display 112 on the front of the device, thereby allowing the touch screen display to display still and / or video images. It can be used as a viewfinder for acquisition. In some embodiments, another light sensor is placed on the front of the device so that the user's image is optionally while the user looks at other video conference participants on the touch screen display To be acquired for video conferencing.

  Device 100 also optionally includes one or more contact strength sensors 165. FIG. 1A shows a contact strength sensor coupled to the strength sensor controller 159 in the I / O subsystem 106. The contact strength sensor 165 may be one or more piezoresistive strain gauges, a capacitive force sensor, an electrical force sensor, a piezoelectric force sensor, an optical force sensor, a capacitive touch sensing surface, or another intensity sensor (eg, a touch sensing surface) Optionally, a sensor used to measure the force of contact (or pressure) above. The contact strength sensor 165 receives contact strength information from the environment (eg, pressure information or a substitute for pressure information). In some embodiments, at least one touch strength sensor is juxtaposed or close to the touch sensitive surface (eg, touch sensitive display system 112). In some embodiments, at least one contact strength sensor is disposed on the back of the device 100, which is the opposite side of the touch screen display 112 disposed on the front of the device 100.

  Device 100 also optionally includes one or more proximity sensors 166. FIG. 1A shows proximity sensor 166 coupled to peripherals interface 118. Alternatively, proximity sensor 166 is coupled to input controller 160 within I / O subsystem 106. In some embodiments, the proximity sensor turns off the touch screen 112 to disable when the multi-function device is placed close to the user's ear (eg, when the user is making a telephone call) .

  Device 100 optionally also includes one or more tactile output generators 167. FIG. 1A shows a haptic output generator coupled to haptic feedback controller 161 in I / O subsystem 106. Tactile output generator 167 optionally includes one or more electroacoustic devices, such as speakers or other audio components, and / or motors, solenoids, electroactive polymers, piezoelectric actuators, electrostatic actuators, or Includes electromechanical devices that convert energy into linear motion, such as other tactile output generating components (e.g., components that convert electrical signals to tactile output on the device). The touch intensity sensor 165 receives tactile feedback generation instructions from the tactile feedback module 133 and generates a tactile output on the device 100 that can be sensed by a user of the device 100. In some embodiments, the at least one tactile output generator is juxtaposed or proximate to the touch sensitive surface (eg, touch sensitive display system 112), optionally the touch sensitive surface Are generated vertically (eg, in / outside of the surface of device 100) or laterally (eg, back and forth in the same plane as the surface of device 100) to generate tactile output. In some embodiments, at least one tactile output generator is disposed on the back of device 100, which is the opposite side of touch screen display 112 disposed on the front of device 100.

  Device 100 also optionally includes one or more accelerometers 168. FIG. 1A shows an accelerometer 168 coupled to a peripheral interface 118. Alternatively, accelerometer 168 is optionally coupled to input controller 160 in I / O subsystem 106. In some embodiments, the information is displayed on the touch screen display in a vertical or horizontal display based on an analysis of data received from the one or more accelerometers. The device 100 optionally obtains information about the magnetometer (not shown), as well as the accelerometer (s) 168, and the position and orientation (eg, vertical or horizontal) of the device 100. GPS (or GLONASS or other global navigation system) receiver (not shown).

  In some embodiments, software components stored in memory 102 include operating system 126, communication module (or instruction set) 128, contact / move module (or instruction set) 130, graphic module (or instruction set) 132, text input module (or instruction set) 134, global positioning system (GPS) module (or instruction set) 135, and application (instruction set) 136. Further, in some embodiments, memory 102 stores device / global internal state 157, as shown in FIGS. 1A and 3. Device / global internal state 157 indicates, if present, which application is currently active, active application state, which application, view, or other information, various areas of touch screen display 112 Include one or more of: display status, including various sensors of the device and information obtained from the input control device 116, and location information regarding the location and / or attitude of the device. .

  Operating system 126 (e.g., Darwin (R), RTXC (R), LINUX (R), UNIX (R), OS X (R), WINDOWS (R), or VxWorks (R)) Embedded operating system) includes various software components and / or drivers for controlling and managing general system tasks (eg, memory management, storage device control, power management, etc.), and various hardware Facilitate communication between components and software components.

  Communication module 128 facilitates communication with other devices via one or more external ports 124 and also various software for processing data received by RF circuitry 108 and / or external ports 124. Also includes components. An external port 124 (eg, Universal Serial Bus (USB), FIREWIRE, etc.) may be coupled directly to another device, or indirectly via a network (eg, Internet, wireless LAN, etc.) It is adapted to do. In some embodiments, the external port is the same as, or similar to, and / or compatible with a 30-pin connector used on an iPod (trademark of Apple Inc.) device, multi-pin (eg, 30-pin) It is a connector. In some embodiments, the external port is the same as, or similar to, and / or compatible with an 8-pin connector (eg, Lightning connector) used on iPhone and iPod (trademark of Apple Inc.) devices There is a multi-pin (for example, 8-pin) connector.

  The touch / move module 130 optionally detects contact with the touch screen 112 (with the display controller 156) and with other touch sensing devices (e.g., a touch pad or physical click wheel). The touch / move module 130 determines whether a touch has occurred (eg, detects a finger down event), determines the strength of the touch (eg, the force or pressure of the touch, or any of the terms force or pressure of the touch). , Movement of the contact and tracking of movement across the touch sensitive surface (eg, detection of one or more finger drag events), and whether contact has ceased (eg, finger up event or interruption of contact) It includes various software components for performing various operations related to the detection of touch, such as determination of detection). The touch / move module 130 receives touch data from the touch sensitive surface. Determining the movement of the touch point, represented by a series of touch data, optionally includes the speed (magnitude), speed (magnitude and direction), and / or acceleration (magnitude and size) of the touch point. And / or determining a change in direction). These actions are optionally applied to a single touch (eg, one finger touch) or multiple simultaneous touches (eg, “multi-touch” / multiple finger touch). In some embodiments, the touch / move module 130 and the display controller 156 detect touch on the touch pad.

  In some embodiments, the contact / move module 130 may include one or more to determine whether an action has been performed by the user (eg, to determine whether the user has “clicked” on the icon). Use a set of intensity thresholds. In some embodiments, at least a subset of the intensity threshold is determined according to software parameters (eg, the intensity threshold is not determined by the activation threshold of a particular physical actuator, but rather the physical hardware of the device 100 Can be adjusted without change). For example, the mouse "click" threshold of the trackpad or touch screen display can be set to any of a wide range of predetermined thresholds without changing the trackpad or touch screen display hardware. Furthermore, in some implementations, the user of the device (e.g., by adjusting individual intensity thresholds and / or using system level click "intensity" parameters, multiple intensities at a time Software settings are provided to adjust one or more of the set of intensity thresholds) by adjusting the thresholds.

  The touch / move module 130 optionally detects gesture input by the user. Different gestures on the touch sensitive surface have different touch patterns (e.g., different movements, timings, and / or detected touch strengths). Therefore, gestures are optionally detected by detecting specific touch patterns. For example, detection of a finger tap gesture detects a finger down event followed by a finger up (lift off) event at the same position (or substantially the same position as the finger down event (e.g. at the position of the icon) Including detecting. As another example, detecting a fin gas wipe gesture on the touch sensitive surface comprises detecting a finger down event followed by detecting one or more finger drag events followed by a finger up (Lift off) detecting an event.

  Graphics module 132 may include components for changing visual effects (eg, brightness, transparency, saturation, contrast, or other visual characteristics) of the displayed graphic on touch screen 112 or other display. Includes various known software components for rendering and displaying graphics. As used herein, the term "graphic" includes any object that can be displayed to the user, including, but not limited to, text, web pages, icons (soft User interface objects including keys), digital images, videos, animations, etc.

  In some embodiments, graphics module 132 stores data representing graphics to be used. Each graphic is optionally assigned a corresponding code. The graphics module 132 receives one or more codes specifying the graphic to be displayed, along with coordinate data and other graphic characteristic data as needed, from an application or the like, for output to the display controller 156 Generate screen image data.

  The haptic feedback module 133 generates instructions used by the tactile output generator 167 to generate tactile output at one or more locations on the device 100 in response to user interaction with the device 100. To include various software components.

  The text input module 134 is optionally a component of the graphics module 132 and may include various applications (eg, contact 137, email 140, IM 141, browser 147, and any other application requiring text input) Provide a soft keyboard for entering text in).

  The GPS module 135 determines the location of the device and uses this information for use in various applications (eg, to the phone 138 for use in location based dialing) as a photo / video metadata camera. And to applications that provide location-based services, such as weather widgets, regional yellow page widgets, and map / navigation widgets.

Applications 136 optionally include the following modules (or sets of instructions), or a subset or superset thereof.
Contacts module 137 (sometimes called an address book or contact list),
Phone module 138,
● Video conferencing module 139,
● Email client module 140,
Instant messaging (IM) module 141,
● Training support module 142,
Camera module 143 for still and / or video images,
● Image management module 144,
● Browser module 147,
● Calendar module 148,
-Of the weather widget 149-1, stock widget 149-2, computer widget 149-3, alarm clock widget 149-4, dictionary widget 149-5, and other widgets acquired by the user, and user created widget 149-6 Widget module 149, optionally including one or more of
● Digital personal assistant module 150,
● Vehicle integration module 151,
A video and music playback module 152, optionally consisting of a video playback module and a music playback module,
● Memo module 153,
Map module 154 and / or online video module 155;

  Examples of other applications 136 optionally stored in memory 102 include other word processing applications, other image editing applications, drawing applications, presentation applications, JAVA enabled applications, encryption, digital These include copyright management, speech recognition, and speech duplication.

  The contact module 137, in conjunction with the touch screen 112, the display controller 156, the touch module 130, the graphic module 132, and the text input module 134, optionally includes an address book or a contact list (eg, memory 102 or memory 370). Is used to manage the application internal state 192 of the contacts module 137), and the management of the contacts module, add names to the address book, delete names from the address book, telephone Associating numbers, email addresses, physical addresses, or other information with names, associating images with names, classifying and sorting names, communicating by telephone 138, video conference 139, email 140, or IM 141 etc. Start and / or facilitate Of the eye, including the provision of a telephone number or e-mail address.

  The phone module 138 along with the RF circuit 108, the audio circuit 110, the speaker 111, the microphone 113, the touch screen 112, the display controller 156, the contact module 130, the graphic module 132, and the text input module 134 has a series of characters corresponding to the telephone number. Enter, access one or more phone numbers in address book 137, correct the phone number entered, dial the corresponding phone number, carry out the conversation, and connect when the conversation is complete Optionally used to hang up or hang up. As mentioned above, wireless communication optionally uses any of a plurality of communication standards, protocols, and techniques.

  RF circuit 108, audio circuit 110, speaker 111, microphone 113, touch screen 112, display controller 156, light sensor 164, light sensor controller 158, touch module 130, graphic module 132, text input module 134, contact list 137, and In conjunction with the telephony module 138, the video conferencing module 139 includes executable instructions for initiating, conducting and terminating video conferences between the user and one or more other participants according to the user's instructions. .

  In conjunction with the RF circuit 108, the touch screen 112, the display controller 156, the touch module 130, the graphic module 132, and the text input module 134, the email client module 140 creates an email in response to user instructions. , Executable instructions for transmitting, receiving and managing. Along with the image management module 144, the email client module 140 greatly facilitates creating and sending emails with still or video images taken with the camera module 143.

  In conjunction with the RF circuit 108, the touch screen 112, the display controller 156, the touch module 130, the graphic module 132, and the text input module 134, the instant messaging module 141 inputs or has input a string corresponding to an instant message. Or send a corresponding instant message (eg using Short Message Service (SMS) or Multimedia Message Service (MMS) protocol for phone-based instant message) Or receive and receive instant messages (using XMPP, SIMPLE, or IMPS for Internet-based instant messages) It includes executable instructions for and displays the instant message. In some embodiments, sent and / or received instant messages may be graphics, photos, audio files, video files, and video files, as supported by MMS and / or Enhanced Messaging Service (EMS). And / or optionally include other attachments. As used herein, "instant messaging" refers to telephone-based messages (eg, messages sent using SMS or MMS) and Internet-based messages (eg, using XMPP, SIMPLE, or IMPS) Means both messages).

  The training support module 142, in conjunction with the RF circuit 108, the touch screen 112, the display module 156, the touch module 130, the graphic module 132, the text input module 134, the GPS module 135, the map module 154, and the music playback module 146, (For example, regarding time, distance, and / or calorie consumption goals), communicate with training sensors (sports devices), receive training sensor data, and calibrate sensors used to monitor training And executable instructions for selecting and playing music for training, and displaying, storing and transmitting training data.

  In conjunction with the touch screen 112, display controller 156, light sensor (s) 164, light sensor controller 158, contact module 130, graphic module 132, and image management module 144, the camera module 143 may be a still image or video Includes executable instructions for capturing (including video streams) and storing them in memory 102, changing still image or video features, or deleting still images or movies from memory 102 .

  In conjunction with the touch screen 112, the display controller 156, the touch module 130, the graphics module 132, the text input module 134, and the camera module 143, the image management module 144 places and modifies still images and / or video (eg, , Editable), otherwise manipulated, labeled, deleted, presented (eg, in a digital slide show or album), and executable instructions for storage.

  In conjunction with the RF circuit 108, the touch screen 112, the display system controller 156, the touch module 130, the graphic module 132, and the text input module 134, the browser module 147 is linked to the web page or parts thereof and the web page. Include executable instructions for browsing the Internet in accordance with the user's instructions including retrieving, linking, receiving, and displaying attachments and other files.

  In conjunction with RF circuit 108, touch screen 112, display system controller 156, touch module 130, graphics module 132, text input module 134, email client module 140, and browser module 147, calendar module 148 follows the user's instructions. , Calendar, and executable instructions for creating, displaying, modifying, and storing data associated with the calendar (eg, calendar items, list of things to do, etc.).

  The widget module 149, along with the RF circuit 108, the touch screen 112, the display system controller 156, the touch module 130, the graphic module 132, the text input module 134, and the browser module 147, is a widget module that is optionally downloaded and used by the user. Application (eg, weather widget 149-1, stock widget 149-2, computer widget 149-3, alarm clock widget 149-4, and dictionary widget 149-5), or a mini application created by the user (eg, user created) It is a widget 149-6). In some embodiments, the widgets include HTML (Hypertext Markup Language, Hypertext Markup Language) files, CSS (Cascading Style Sheets, Cascading Style Sheets) files, and JavaScript files. In some embodiments, widgets include XML (extensible markup language) files and JavaScript files (eg, Yahoo! ® widgets).

  In conjunction with the RF circuit 108, the audio circuit 110, the speaker 111, the microphone 113, the touch screen 112, the display controller 156, the contact module 130, the graphic module 132, and the text input module 134, the digital personal assistant module 150 The information representing the recorded and recorded voice command is sent to a server, such as server 510 of FIG. 5A, for analysis and responds to the voice command based on the response from the server.

  The vehicle integration module 151 provides a vehicle (eg, a user interface) on a corresponding display of the vehicle information display system (eg, the display 546 of the external information providing system 540 of FIG. 5A) for a mapping application or a music application, etc. Includes executable instructions for one or more intermediary processes controlling a vehicle information display system in a car, truck, van, etc.). The vehicle integration application converts the information from the third party application into content for display by the vehicle integration application on the corresponding display of the vehicle information display system.

  In conjunction with the touch screen 112, display system controller 156, touch module 130, graphics module 132, audio circuit 110, speaker 111, RF circuit 108, and browser module 147, the video and music playback module 152 is an MP3 or AAC file, etc. Executable instructions that allow the user to download and play back recorded music or other sound files stored in one or more file formats, and connect the video (on touch screen 112 or via external port 124 The display, presentation or otherwise playable instructions on an external display). In some embodiments, device 100 optionally includes the functionality of an MP3 player such as an iPod (trademark of Apple Inc.).

  The memo module 153, together with the touch screen 112, the display controller 156, the touch module 130, the graphic module 132, and the text input module 134, is an executable instruction for creating and managing a memo, a to-do list, etc. according to the user's instructions. including.

  The map module 154 together with the RF circuit 108, the touch screen 112, the display system controller 156, the touch module 130, the graphic module 132, the text input module 134, the GPS module 135 and the browser module 147, maps and maps Receive, display, modify and store data associated with the data (eg, data about driving directions, stores at or near a particular location and other points of interest, and other data based on location) Are optionally used.

  Online video in conjunction with touch screen 112, display system controller 156, touch module 130, graphic module 132, audio circuit 110, speaker 111, RF circuit 108, text input module 134, email client module 140, and browser module 147 Module 155 allows the user to access, browse, receive (eg, by streaming and / or downloading) a particular online video, (eg, external connected on touch screen or via external port 124) H. H. H. H. H. H. H. H. H. H. H. H. H. H. H. H. H. H. H. H. H. H. H. H. H. H. H. H. H .: Replaying and Sending E-mail with a Link to Its Online Video, and; H.264 includes instructions that allow managing online video in one or more file formats such as H.264. In some embodiments, instant messaging module 141, rather than email client module 140, is used to send a link to a particular online video.

  Each of the above identified modules and applications is a set of executable instructions for performing one or more of the above described functions and methods described herein (eg, computer implemented methods and methods). The other information processing method described in These modules (ie, instruction sets) need not be implemented as separate software programs, procedures, or modules, and thus various subsets of these modules are, in various embodiments, optionally It is combined or rearranged. In some embodiments, memory 102 optionally stores a subset of the modules and data structures identified above. In addition, memory 102 optionally stores additional modules and data structures not described above.

  In some embodiments, device 100 is a device in which operations of a predetermined set of functions on the device are performed exclusively through the touch screen and / or touch pad. By using the touch screen and / or touch pad as the primary input control device for the operation of the device 100, the number of physical input control devices (push button, dial etc.) on the device 100 is optionally reduced Ru.

  The set of predefined functions, performed exclusively via the touch screen and / or touch pad, optionally includes navigation between user interfaces. In some embodiments, the touch pad navigates the device 100 from any user interface displayed on the device 100 to the main, home, or root menu when touched by the user. In such embodiments, the "menu button" is implemented using a touch pad. In some other embodiments, the menu button is not a touch pad but a physical push button or other physical input control device.

  FIG. 1B is a block diagram illustrating exemplary components for event processing in accordance with some embodiments. In some embodiments, memory 102 (FIG. 1A) or 370 (FIG. 3) may include event sorter 170 (eg, in operating system 126) and corresponding application 136-1 (eg, applications 137-13 described above), 155 and any of 380-390).

  The event sorter 170 receives event information and determines an application 136-1 and an application view 191 of the application 136-1 to which the event information is to be distributed. The event sorter 170 includes an event monitor 171 and an event dispatcher module 174. In some embodiments, application 136-1 shows an internal view of the current application view (s) displayed on touch sensitive display 112 when the application is active or running. Includes state 192. In some embodiments, device / global internal state 157 is used by event sorter 170 to determine which application is currently active, and application internal state 192 is a destination for event information. Used by event sorter 170 to determine application view 191.

  In some embodiments, application internal state 192 is displayed as, or is ready for display as, additional information, eg, resume information used when application 136-1 resumes execution, application 136-1. User interface state information indicating the current information, a queue of states to allow the user to return to the previous state or view of the application 136-1, and wait for the user to redo / undo the action previously taken Includes one or more of the matrices.

  The event monitor 171 receives event information from the peripheral device interface 118. Event information includes information of sub-events (eg, user touch on touch sensitive display 112 as part of a multi-touch gesture). Peripheral interface 118 transmits information received from sensors such as I / O subsystem 106 or proximity sensor 166 (via audio circuit 110), accelerometer (s) 168, and / or microphone 113. . The information that peripherals interface 118 receives from I / O subsystem 106 includes information from touch sensitive display 112 or a touch sensitive surface.

  In some embodiments, event monitor 171 sends requests to peripherals interface 118 at predetermined intervals. In response, peripheral interface 118 transmits event information. In other embodiments, peripheral interface 118 transmits event information only when there is a significant event (eg, receiving an input above a predetermined noise threshold and / or for a predetermined amount of time or more).

  In some embodiments, the event sorter 170 also includes a hit view determination module 172 and / or an active event recognizer determination module 173.

  The hit view determination module 172 provides a software procedure to determine where in the one or more views a sub-event occurred when the touch sensitive display 112 displays more than one view. Do. A view is composed of controls and other elements that can be viewed by the user on the display.

  Another aspect of the user interface associated with an application is a set of views, which may be referred to herein as an application view or user interface window, in which information is displayed and touch-based gestures are To be implemented. The application view (of the corresponding application) in which a touch is detected optionally corresponds to a program level in the program hierarchy or view hierarchy of the application. For example, the lowest level view in which a touch is detected is optionally referred to as a hit view, and the set of events recognized as appropriate input is at least partially the first to initiate a touch-based gesture. Optionally determined based on the hit view of the touch.

  The hit view determination module 172 receives information related to touch-based gesture sub-events. If the application has multiple views organized as a hierarchy, hit view determination module 172 identifies the hit view as the lowest view in the hierarchy to process the sub-events. In most situations, the hit view is the lowest level view in which the starting sub-event (i.e., the first sub-event in the sequence of events or sub-events forming a potential event) occurs. When a hit view is identified by the hit view determination module, that hit view typically receives all sub-events associated with the same touch or input source that were identified as a hit view.

  The active event recognizer determination module 173 determines which view in the view hierarchy should receive a particular sequence of sub-events. In some embodiments, the active event recognizer determination module 173 determines that only the hit view should receive a particular sequence of sub-events. In another embodiment, the active event recognizer determination module 173 determines that all the views including the physical location of the subevent are the views actively involved, so that all the views actively participating are , Determine that a particular sequence of sub-events should be received. In other embodiments, even if touch sub-events are completely limited to the area associated with a particular view, the upper view of the hierarchy will remain the actively engaged view.

  The event dispatcher module 174 sends event information to an event recognizer (eg, event recognizer 180). In embodiments that include the active event recognizer determination module 173, the event dispatcher module 174 delivers event information to the event recognizer determined by the active event recognizer determination module 173. In some embodiments, event dispatcher module 174 stores event information obtained by corresponding event receiver module 182 in an event queue.

  In some embodiments, operating system 126 includes an event sorter 170. Alternatively, application 136-1 includes event sorter 170. In still other embodiments, event sorter 170 is a stand-alone module or part of another module stored in memory 102, such as contact / move module 130.

  In some embodiments, the application 136-1 includes multiple event handlers 190 and one or more application views 191, each including instructions for processing touch events that occur within each view of the application's user interface. Including. Each application view 191 of the application 136-1 includes one or more event recognition units 180. Typically, each application view 191 includes a plurality of event recognizers 180. In other embodiments, one or more of the event recognizers 180 may be a user interface kit (not shown) or a higher level object such as the application 136-1 inherits methods and other properties. , Part of a separate module. In some embodiments, each event handler 190 includes one or more of data updater 176, object updater 177, GUI updater 178, and / or event data 179 received from event sorter 170. Event handler 190 optionally utilizes or invokes data updater 176, object updater 177, or GUI updater 178 to update application internal state 192. Alternatively, one or more of the application views 191 include one or more corresponding event handlers 190. Also, in some embodiments, one or more of data updater 176, object updater 177, and GUI updater 178 are included in the corresponding application view 191.

  Each event recognition unit 180 receives event information (for example, event data 179) from the event sorter 170, and identifies an event from the event information. The event recognition unit 180 includes an event reception unit 182 and an event comparison unit 184. In some embodiments, event recognizer 180 also includes at least a subset of metadata 183 and event delivery instructions 188 (which optionally includes sub-event delivery instructions).

  The event receiving unit 182 receives event information from the event sorter 170. Event information includes, for example, information on sub-events such as touch or movement of touch. Depending on the subevent, the event information also includes additional information such as the location of the subevent. If the sub-event relates to touch movement, then the event information also optionally includes the speed and direction of the sub-event. In some embodiments, the event comprises rotation of the device from one direction to another (eg, from portrait orientation to landscape orientation or vice versa), and the event information is the device's current orientation ( Containing corresponding information about the attitude of the device).

  The event comparison unit 184 compares the event information with the predetermined event or sub-event definition, and determines the event or sub-event or determines or updates the state of the event or sub-event by the comparison. In some embodiments, event comparator 184 includes event definition 186. The event definition 186 includes, for example, definitions of events (for example, a predetermined sequence of sub-events) such as event 1 (187-1) and event 2 (187-2). In some embodiments, sub-events within event 187 include, for example, touch start, touch end, touch movement, touch cancellation, and multiple touches. In one embodiment, the definition for event 1 (187-1) is a double tap on a display object. The double tap is, for example, a first touch (start of touch) for a predetermined stage on the displayed object, a first lift-off for a predetermined stage (end of touch), and a predetermined stage on the displayed object And a second lift-off (touch end) for a predetermined stage. In another embodiment, the definition of event 2 (187-2) is a drag on the displayed object. This dragging includes, for example, touch (or touch) on the display object for a predetermined stage, movement of the touch across the touch sensitive display 112, and lift off of the touch (end of touch). In some embodiments, an event also includes information about one or more associated event handlers 190.

  In some embodiments, event definitions 187 include definitions of events for each user interface object. In some embodiments, event comparator 184 performs a hit test to determine which user interface object is associated with the sub-event. For example, in an application view where three user interface objects are displayed on touch sensitive display 112, when a touch is detected on touch sensitive display 112, event comparator 184 performs a hit test to Determine which of the user interface objects are associated with that touch (sub-event). If each display object is associated with a respective event handler 190, the event comparator uses the result of its hit test to determine which event handler 190 should be activated. For example, event comparator 184 selects an event handler associated with the sub-event and object triggering the hit test.

  In some embodiments, the definition for each event 187 also includes a delay effect, which delays delivery of event information until it is determined whether the sequence of sub-events corresponds to the event type of the event recognizer. .

  When the corresponding event recognition unit 180 determines that the series of sub-events does not match any of the events in the event definition 186, the corresponding event recognition unit 180 determines that the event is in an impossible state, an event failure state, or The end event state is entered and thereafter ignores subsequent sub-events of this touch based gesture. In this situation, other event recognizers that remain active with respect to the hit view, if present, continuously track and process the sub-events of the touch-based gesture in progress.

  In some embodiments, the corresponding event recognizer 180 indicates how to perform sub-event delivery to configurable events, flags, and / or event recognizers that the event delivery system is actively involved It includes metadata 183 having a list. In some embodiments, the metadata 183 may include configurable properties, flags, and / or lists that indicate how the event recognizer interacts with or is allowed to interact with each other. Including. In some embodiments, metadata 183 includes configurable properties, flags, and / or a list that indicates whether sub-events are to be delivered to different levels in the view or program hierarchy.

  In some embodiments, the corresponding event recognizer 180 activates an event handler 190 associated with the event when one or more specific sub-events of the event are recognized. In some embodiments, each event recognizer 180 delivers to the event handler 190 event information associated with the event. Activating the event handler 190 is different from sending (and deferring sending) sub-events to their respective hit views. In some embodiments, the event recognizer 180 throws a flag associated with the recognized event, and the event handler 190 associated with the flag catches the flag and performs a predetermined process.

  In some embodiments, the event delivery instructions 188 include sub-event delivery instructions that deliver event information for sub-events without activating the event handler. Instead, the sub-event delivery instruction delivers event information to an event handler associated with a series of sub-events or a view that is actively involved. An event handler associated with a series of sub-events or views actively involved receives the event information and performs a predetermined process.

  In some embodiments, data updater 176 creates and updates data used by application 136-1. For example, data updater 176 may update the telephone number used by contacts module 137 or store video files used by video player module 145. In some embodiments, object updater 177 creates and updates objects used by application 136-1. For example, object updater 176 creates a new user interface object or updates the location of the user interface object. The GUI updater 178 updates the GUI. For example, GUI updater 178 prepares display information and sends it to graphics module 132 for display on a touch sensitive display.

  In some embodiments, event handler (s) 190 include or have access to data updater 176, object updater 177, and GUI updater 178. In some embodiments, data updater 176, object updater 177, and GUI updater 178 are included within a single module of each application 136-1 or application view 191. In other embodiments, they are included in more than one software module.

  The above discussion of user touch event processing on touch sensitive displays also applies to other forms of user input for operating multifunction device 100 using an input device, all of which are touch It will be appreciated that it does not necessarily start on the screen. For example, movement of a mouse and pressing of a mouse button, movement of a touch such as tapping on a touch pad, dragging, scrolling, pen stylus input, optionally coordinated with pressing or holding of one or more keyboards Device movement, verbal instructions, detected eye movement, biometric input, and / or any combination of these are optionally used as input corresponding to a sub-event defining an event to be recognized .

  FIG. 2 is a diagram of a portable multifunction device 100 with a touch screen 112, in accordance with some embodiments. The touch screen optionally displays one or more graphics in the user interface (UI) 200. In this embodiment, as well as other embodiments described below, the user may, for example, use one or more fingers 202 (not drawn to scale in the figures) or one or more styluses 203 (in the figures). (Not drawn to scale), it is possible to select one or more of the graphics by performing the gesture on the graphic. In some embodiments, selection of one or more graphics occurs when the user breaks contact with one or more graphics. In some embodiments, the gesture contacts one or more taps, one or more swipes (from left to right, right to left, up and / or down), and / or the device 100. Optionally includes rolling (from right to left, left to right, up and / or down) of the finger. In some implementations or situations, accidental contact with a graphic does not select that graphic. For example, if the gesture corresponding to the selection is a tap, a swipe gesture that swipes over the application icon optionally does not select the corresponding application.

  Device 100 also optionally includes one or more physical buttons, such as “home” or menu button 204. As mentioned above, menu button 204 is optionally used to navigate to any application 136 in the set of applications that are optionally run on device 100. Alternatively, in some embodiments, the menu button is implemented as a soft key in a GUI displayed on the touch screen 112.

  In one embodiment, the device 100 includes a touch screen 112, a menu button 204, a push button 206 for powering on / off the device and locking the device, a volume control button 208, a subscriber identity module (SIM) card slot 210 includes a headset jack 212 and a docking / charging external port 124. The push button 206 turns the power on / off on the device by depressing the button and holding the button depressed for a predetermined time interval, and depressing the button before the predetermined time interval has elapsed. Is optionally used to lock the device and / or unlock the device or initiate the unlocking process by releasing the button. In an alternative embodiment, device 100 also accepts verbal input via microphone 113 regarding activation or deactivation of some functions. Device 100 may also include one or more contact intensity sensors 165 for detecting the intensity of the touch on touch screen 112 and / or one or more for generating tactile output to the user of device 100. A tactile output generator 167 is also optionally included.

  FIG. 3 is a block diagram of an exemplary multifunction device with a display and touch sensitive surface in accordance with some embodiments. Device 300 need not be portable. In some embodiments, the device 300 can be a laptop computer, desktop computer, tablet computer, multimedia player device, navigation device, educational device (such as a children's learning toy), a gaming system, or a control device (eg, a home) Controller for business or business use). Device 300 typically includes one or more processing units (CPUs) 310, one or more networks or other communication interfaces 360, memory 370, and one or more for interconnecting these components. A communication bus 320 is included. Communication bus 320 optionally includes circuitry (sometimes referred to as a chip set) that interconnects and controls communication between system components. Device 300 includes an input / output (I / O) interface 330 comprising a display 340, which is typically a touch screen display (eg, touch screen display 112). The I / O interface 330 may also generate a tactile output on the keyboard and / or mouse (or other pointing device) 350 and touch pad 355, device 300 (eg, as described above with reference to FIG. 1A). Tactile output generator 357 similar to tactile output generator 167, sensor 359 (eg, light sensor, acceleration sensor, proximity sensor, touch sensing sensor, and / or contact intensity sensor described above with reference to FIG. 1A A contact strength sensor similar to 165) is also optionally included. Memory 370 includes high-speed random access memory, such as DRAM, SRAM, DDR RAM, or other random access semiconductor memory devices, and optionally, one or more magnetic disk storage devices, optical disk storage devices, flash memory It includes non-volatile memory, such as a device or other non-volatile semiconductor storage device. Memory 370 optionally includes one or more storage devices located remotely from CPU (s) 310. In some embodiments, memory 370 is similar to programs, modules, and data structures stored in memory 102 of portable multifunction device 100 (FIG. 1A), programs, modules, and data structures, or a subset thereof. Remember. Furthermore, memory 370 optionally stores additional programs, modules, and data structures that are not present in memory 102 of portable multifunction device 100. For example, memory 370 of device 300 optionally stores drawing module 380, presentation module 382, word processing module 384, website creation module 386, disk authoring module 388, and / or spreadsheet module 390, On the other hand, the memory 102 of the portable multifunction device 100 (FIG. 1A) does not optionally store these modules.

  Each of the elements of FIG. 3 identified above is optionally stored in one or more of the aforementioned memory devices. Each of the above identified modules corresponds to a set of instructions for performing the functions described above. The above-identified modules or programs (ie, instruction sets) need not be implemented as separate software programs, procedures, or modules, and thus, in various embodiments, various subsets of these modules Optionally, they are combined or otherwise rearranged. In some embodiments, memory 370 optionally stores a subset of the identified modules and data structures described above. Further, memory 370 optionally stores additional modules and data structures not described above.

  Attention is now directed to embodiments of a user interface (“UI”) that are optionally implemented on the portable multifunction device 100.

FIG. 4A illustrates an exemplary user interface for a menu of applications on portable multifunction device 100 in accordance with some embodiments. A similar user interface is optionally implemented on device 300. In some embodiments, user interface 400 includes the following elements, or a subset or superset thereof:
Signal strength indicator (s) 402 for wireless communication (s), such as cellular and Wi-Fi signals;
● Time 404,
● Bluetooth indicator 405,
● Battery status indicator 406,
A tray 408 with icons for frequently used applications, such as:
O an icon 416 for the phone module 138, labeled "phone", optionally including an indicator 414 for the number of missed calls or voice mail messages;
○ an icon 418 for the email client module 140, labeled "mail", optionally including an indicator 410 of the number of unread emails,
○ an icon 420 for the browser module 147, labeled "browser", and o a video and music playback module 152, also referred to as an iPod (trademark of Apple Inc.) module 152, labeled "iPod" Icon 422.
● Icons for other applications, such as
○ An icon 424 for the IM module 141, labeled "Text",
○ an icon 426 for the calendar module 148, labeled "calendar",
○ An icon 428 for the image management module 144, labeled “Photo”
○ An icon 430 for the camera module 143, labeled "Camera",
○ Icon 432 for the online video module 155, labeled "Online Video",
○ Icon 434 for the stock widget 149-2, labeled as "stock",
○ An icon 436 for the map module 154, labeled "map",
○ Icon 438 for the weather widget 149-1, labeled "weather",
○ Icon 440 for the Alarm Clock Widget 149-4, labeled "Clock",
○ Icon 442 for training support module 142, labeled “Training support”,
An icon 444 for the note module 153, labeled "Notes", and an icon 446 for a configuration application or module, which provides access to settings for the device 100 and its various applications 136.

  It should be noted that the label of the icon shown in FIG. 4A is merely exemplary. For example, the icon 422 for the video and music playback module 152 is labeled "music" or "music player". Other labels are optionally used for various application icons. In some embodiments, the label for the corresponding application icon includes the name of the application corresponding to the corresponding application icon. In some embodiments, the label for a particular application icon is different from the name of the application that corresponds to that particular application icon.

  FIG. 4B is exemplary on a device (eg, device 300 of FIG. 3) comprising a touch sensitive surface 451 (eg, tablet or touch pad 355 of FIG. 3) that is separate from display 450 (eg, touch screen display 112). User interface. Device 300 may also sense one or more contact intensity sensors (eg, one or more of sensors 357) and / or tactile sensations to the user of device 300 to detect the intensity of the touch on touch sensitive surface 451. One or more tactile output generators 359 are included to generate the output.

  Although some of the following examples are described with reference to inputs on touch screen display 112 (if the touch sensitive surface and the display are combined), in some embodiments, the device is As shown in FIG. 4B, the input is detected on a touch sensitive surface separate from the display. In some embodiments, the touch sensitive surface (eg, 451 in FIG. 4B) corresponds to the major axis (eg, 452 in FIG. 4B) corresponding to the major axis (eg, 453 in FIG. 4B) on the display (eg, 450) Have. According to these embodiments, the device may be configured with a touch sensitive surface 451 at a location corresponding to a corresponding location on the display (eg, 460 corresponds to 468 and 462 corresponds to 470 in FIG. 4B). Detect contacts (eg, 460 and 462 in FIG. 4B). Thus, when the touch sensitive surface is separate from the display, the user inputs (eg, contacts 460 and 462 and their movements detected by the device on the touch sensitive surface (eg, 451 in FIG. 4B) ) Is used by the device to manipulate the user interface on the display (eg, 450 of FIG. 4B) of the multifunction device. It should be understood that similar methods are optionally used for the other user interfaces described herein.

  Furthermore, while the following description is primarily described with reference to finger input (eg, finger touch, finger tap gesture, finger swipe gesture), in some embodiments those finger inputs It should be understood that one or more of them may be replaced with input from another input device (eg, mouse based input or stylus input). For example, the swipe gesture is optionally replaced with a mouse click (e.g., instead of contact) and then the movement of the cursor along the swipe path (e.g., instead of moving the touch). As another example, the tap gesture is optionally replaced by a mouse click while the cursor is placed at the location of the tap gesture (e.g. instead of stopping the detection of the touch following the detection of the touch) . Similarly, it is understood that if multiple user inputs are detected simultaneously, multiple computer mice are optionally used simultaneously, or that mouse and finger contacts are optionally used simultaneously. I want to.

  As used herein, the term "focus selector" refers to an input element that indicates the current portion of the user interface with which the user is interacting. In some implementations that include a cursor or other position marker, the cursor acts as a "focus selector" so that it is above a particular user interface element (eg, a button, window, slider, or other user interface element). The particular user interface when an input (eg, a press input) is detected on the touch sensitive surface (eg, touch pad 355 of FIG. 3 or touch sensitive surface 451 of FIG. 4B) while the cursor is on An element is adjusted according to the detected input. In some implementations, including a touch screen display (eg, touch sensitive display system 112 of FIG. 1A or touch screen 112 of FIG. 4A) that allows direct interaction with user interface elements on the touch screen display As the touch detected on the touch screen plays the role of a "focus selector", the input (e.g. touch input by touch) may be a specific user interface element (e.g. button, window, slider or) on the touch screen display. When detected at the location of another user interface element, that particular user interface element is adjusted in accordance with the detected input. In some implementations, the focus does not involve the movement of the corresponding cursor or the movement of the touch on the touch screen display (e.g., move the focus from one button to another using tab or arrow keys) By moving it from one area of the user interface to another area of the user interface, and in these implementations, the focus selector moves in accordance with the movement of focus between the various areas of the user interface. Regardless of the particular form adopted by the focus selector, the focus selector generally interacts with the user interface intended by the user (eg, a user interface element that the user intends to interact with, User interface elements (or contacts on the touch screen display) that are controlled by the user to communicate (by showing to the device). For example, the location of the focus selector (eg, cursor, touch, or selection box) on the corresponding button while the pressing input is detected on the touch sensitive surface (eg, touch pad or touch screen) Indicates that the user intends to activate the button (instead of the other user interface elements shown on the display of the device).

  FIG. 5A is a block diagram of an operating environment 500 in accordance with some embodiments. Operating environment 500 includes a server 510, one or more communication networks 505, a portable multifunction device 100, and an external information provision system 540. In some embodiments, the external information provision system 540 is an entertainment and / or navigation system implemented in a vehicle. In some embodiments, the external information provision system 540 includes one or more displays. In some embodiments, the vehicle includes a plurality of external information provision systems 540 communicatively coupled to the device 100 in the operating environment 500, each comprising a corresponding display.

The server 510 typically executes one or more processing units (CPUs) 512 for executing modules, programs and / or instructions stored in the memory 524, thereby performing processing operations. These include a network or other communication interface 520, a memory 524 and one or more communication buses 522 for interconnecting these components. Communication bus 522 optionally includes circuitry (sometimes referred to as a chip set) that interconnects and controls communication between system components. Memory 524 includes high-speed random access memory, such as DRAM, SRAM, DDR RAM, or other random access semiconductor memory devices, and also includes one or more magnetic disk storage devices, optical disk storage devices, flash memory devices, or the like. And a non-volatile memory such as a non-volatile semiconductor storage device. Memory 524 may optionally include one or more storage devices remotely located from CPU (s) 512. The memory 524 or non-volatile memory device or devices in the memory 524 comprises non-transitory computer readable storage medium. In some embodiments, memory 524 or a computer readable storage medium of memory 524 stores the following programs, modules, and data structures, or subsets thereof.
Operating system 526 including procedures for processing various basic system services, and procedures for performing hardware dependent tasks, and one or more communication network interfaces 520, and the Internet, other wide area networks, A network communication module 528 used to connect the (wired or wireless) server 510 to other computing devices via one or more communication networks 505, such as a local area network, a metropolitan area network, etc.

  The portable multifunction device 100 (sometimes referred to herein as "device 100") typically includes the components described with reference to FIGS. 1A-1B and / or FIG.

External information provision system 540 (sometimes referred to herein as “system 540”) executes modules, programs and / or instructions stored in memory 554, thereby performing processing operations 1 It includes one or more processing units (CPUs) 542, one or more networks or other communication interfaces 550, a memory 554, and one or more communication buses 552 for interconnecting these components. External information provision system 540 optionally includes one or more display devices 546 and a plurality of controls 548 (eg, jog dials, knobs, buttons, switches, touch sensitive surfaces such as touch screen displays, or other input sources) , And includes a user interface 544. In some embodiments, one or more displays 546 include a main display 546-1 (eg, a dashboard or vehicle navigation display) and an auxiliary display 546-2 (eg, a back seat or entertainment display). In some embodiments, a display of each of the one or more displays 546 can receive user touch input (eg, detect finger touch and gestures corresponding to finger touch detection and movement). It is. In some embodiments, the display of each of the one or more displays 546 is a plurality of controls 548 (eg, jog dials, knobs, buttons, switches, touch sensitive surfaces such as touch screen displays, or other input sources) Associated with one or more controls. Communication bus 552 optionally includes circuitry (sometimes referred to as a chip set) that interconnects and controls communication between system components. Memory 554 includes high speed random access memory, such as DRAM, SRAM, DDR RAM, or other random access semiconductor memory device, and optionally, one or more magnetic disk storage devices, optical disk storage devices, flash memory It includes non-volatile memory, such as a device or other non-volatile semiconductor storage device. Memory 554 may optionally include one or more storage devices remotely located from CPU (s) 542. The memory 552, or non-volatile memory device or devices in the memory 552, comprises non-transitory computer readable storage medium. In some embodiments, memory 552 or a computer readable storage medium of memory 552 stores the following programs, modules, and data structures, or a subset thereof.
Operating system 556, including procedures for processing various basic system services, and procedures for performing hardware dependent tasks, and one or more communication network interfaces 550 and the Internet, other wide area networks, local A network communication module 558 used to connect the (wired or wireless) server 540 to other computing devices via one or more communication networks 505, such as area networks, metropolitan area networks, etc.

  In some embodiments, device 100 drives one or more displays 546 of system 540. For example, device 100 transmits a video signal to system 540, and CPU 542 of system 540 renders the video signal on one or more displays 546. In some embodiments, device 100 transmits the video signal directly to one or more displays 546 and CPU 542 is not used to render the video signal (eg, device 100 uses display 546 as an auxiliary display) ). In some embodiments, the user interface displayed on touch screen 112 of device 100 is synchronized with the user interface displayed on one or more displays 546 of system 540, and in some other embodiments. In, the user interface displayed on the touch screen 112 of the device 100 is not continuously synchronized with the user interface displayed on the one or more displays 546 of the system 540, and in some cases (eg, the device 100 The user is looking at information about possible driving destinations but when he has not yet selected the driving destination), the touch screen 112 of the device 100 differs from the one displayed on the display 546 of the system 540 Display (for example, touch screen Over down 112 and the display 546 is intermittently synchronized, they are not synchronized in the period between intermittent synchronization events).

  In some embodiments, in response to detecting a user input (eg, a user touch input or a respective one of the plurality of controls 548 associated with the respective display of the one or more displays 546) The associated user input), the system 540 (or the respective display or controls of the one or more displays 546, or the respective controls of the plurality of controls 548) are input information corresponding to the user input (eg, to the input source) Sends an identifier and an input description describing the user input to the device 100. Furthermore, the device 100 may display one or more displays according to the received input information and / or the display state upon or before the user input of the user interface displayed on the respective display of the one or more displays 546. Update the display of each of 546 and / or the user interface displayed on the touch screen 112 of the device 100.

  FIG. 5B is a flow chart illustrating the process of selecting an affected display and sending updated information to the affected display. The affected display (e.g., main display 546-1) displays 560 the user interface. Each input source (e.g., jog dial 675 shown in FIGS. 6W-6FF), which is a control of system 540, detects 561 user input (e.g., rotation of jog dial 675). In some embodiments, each input source is one of the controls 548 and the user input is an interaction with one of the controls 548. For example, controls 548 include buttons, switches, dials, knobs, other mechanical affordances, touch sensitive surfaces, or other input sources. For example, the user input may be knob or dial rotation, affordance or knob depression, touch input detected on a touch sensitive surface or touch screen, or other user interaction with the control 548.

  In response to detecting a user input, each input source sends 562 input information (eg, {source ID; input description}) to the device 100. In some embodiments, the input information includes a unique identifier for the input source and input description information that describes the user input. For example, the input description information describes the magnitude and direction of rotation of the jog dial, the position and movement amount / movement direction of the contact detected on the touch sensing surface, and / or the type of gesture performed on the touch sensing surface. Raw input data such as gesture data.

  The device 100 receives 564 input information from each of the input sources. Device 100 associates the affected display by associating a unique identifier (eg, source ID) contained in the input information with a display identifier tag (eg, display ID) based at least in part on input source mapping 566. select. In some embodiments, input source mapping 566 is stored in the memory of device 100. For example, when the jog dial 675 is associated with the first display, the device 100 can rotate the jog dial 675 based on the input source mapping 566 to the first display (eg, the display 546-1 in FIGS. 6W-6CC). Associate with Alternatively, when jog dial 675 is associated with a second display (e.g., displays 546-2 in FIGS. 6DD-6FF), device 100 associates the rotation of jog dial 675 with the second display. In some embodiments, multiple input sources are mapped to the same display. However, in some embodiments, each input source is mapped to only a single display. In some embodiments, input source mapping 566 may include an input source mapping update event (eg, a vehicle associated with the first display initiating reverse to control the first display, or the input may In response to detecting that the jog dial is associated with the second display instead of the first display, and / or according to a predetermined schedule, the device 100 updates.

  After selecting the affected display, the device 100 displays the display ID (as determined based on the input source mapping 566) for the affected display based on the user interface state table 568, at least in part, on display state information (eg, By associating what kind of user interface is displayed in the display, which user interface element is displayed, and / or which control is associated with which function, such as volume control or scrolling) Determine the state of each of the user interfaces displayed on the affected display. In some embodiments, user interface state table 568 is stored in the memory of device 100. After determining the state of each of the user interfaces displayed on the affected display, the device 100 responds to the affected display according to the respective states of the user interface displayed on the affected display and the input description information. The updated user interface is generated (570), and the device 100 sends the updated user interface (or information for generating the updated user interface) to the affected display. In some embodiments, device 100 may use state information associated with a display ID corresponding to the affected display to respond to an updated user interface (eg, subsequent input information received from an input source) The user interface state table 568 is also updated to reflect the

In response to receiving an updated user interface (e.g., update information) from the device 100, the affected display may (e.g., update an image of the user interface previously provided by the device 100 by the device 100) The user interface displayed on the affected display is updated 574 to display the updated user interface, by replacing the provided user interface with the updated image of the user interface.
User interface and related processes

  Attention is now directed to embodiments of a user interface ("UI") and associated processes that can be implemented on an electronic device having a display and touch sensitive surface, such as device 300 or portable multifunction device 100.

  6A-6V illustrate exemplary user interfaces for synchronizing two or more displays according to some embodiments. The user interfaces of these figures are used to illustrate the processes described below, including the processes of FIGS. 7A-7D and 8A-8B.

  FIG. 6A shows a user interface 602 displayed on the touch screen 112 of the portable multifunction device 100 (sometimes referred to herein as “device 100”). The user interface 602 includes a plurality of user interface objects (e.g., application icons 424, 426, 428, 430, 434, 436, 446, 416, 418, 420, and 422). FIG. 6A also illustrates detecting user touch input 606 on map application icon 436 on touch screen 112.

  FIG. 6A further shows the user interface 604 displayed on the display 546-1 of the external information providing system 540 (sometimes referred to herein as "system 540"). In some embodiments, system 540 is included in a vehicle (eg, in a dashboard or steering wheel of the vehicle). In some embodiments, display 546-1 is a primary display of a plurality of displays associated with system 540. In some embodiments, display 546-1 is implemented within the dashboard of the vehicle and is visually accessible to the driver and / or passenger of the vehicle. In some embodiments, display 546-1 is a touch screen display configured to detect one or more user touch input. FIG. 6A shows the outside air temperature (eg, 82 °), the air temperature in the driver's zone of the vehicle (eg, 70 °), the air temperature in the passenger zone of the vehicle (eg, 75 °), the driver's zone of the vehicle A plurality of climate controls are displayed in the user interface 604, including blower strength (e.g. 2 at 0-10 strength) and blower strength in the passenger zone of the vehicle (e.g. 5 at 0-10 strength) And a display 546-1 is shown. FIG. 6A illustrates a plurality of user interface objects (sometimes referred to herein as “affordances” or “selectable user interface objects”) capable of adjusting the temperature within each zone of the vehicle (eg, affordances) 603a, 603b, 603c, and 603d), a plurality of user interface objects (eg affordances 603e, 603f, 603g, and 603h) capable of adjusting the blower strength in the respective zones of the vehicle, and other vehicle climates A plurality of user interface objects whose control settings can be adjusted (eg, air conditioner related affordance 605a, windshield defrosting related 605b, rear glass related defrosting 605c, and automatic climate control Display 546-1 to display the 605d) associated with the roll is also shown. FIG. 6A illustrates a plurality of mechanical affordances (eg, knobs or jog dials 607 c configured to control the volume of the vehicle's sound system or scroll one or more user interface objects on display 546-1. And a radio tuner (eg, affordances 607a, 607b, 607d, and 607e) of the vehicle.

  FIG. 6B illustrates that in response to detecting the user touch input 606 on the map application icon 436 in FIG. 6A, the touch screen 112 displays the current position 608 of the device 100 at the map application in the user interface 602 of the touch screen 112. Indicates to display. In some embodiments, detecting a user touch input on each affordance corresponds to detecting a tap gesture for each affordance, wherein the tap gesture corresponds to a touch at a position corresponding to each affordance. Detecting and detecting contact lift off within a predetermined time period (e.g., 0.05, 0.1, 0.2, 0.3, 0.4 or 0.5 seconds) . In some embodiments, the map displayed in the user interface 602 is not to scale. FIG. 6B also illustrates detecting user touch input 610 on search box 609 on touch screen 112.

  6B displays the current position 608 of the device 100 on the display 546-1 in the map displayed in the user interface 604 in response to detecting the user touch input 606 on the map application icon 436 in FIG. 6A. It further indicates to display. In some embodiments, the map displayed in the user interface 604 is not to scale. An affordance 612, also displayed within the user interface 604, causes the device to call a digital assistant (eg, Siri® from Apple Inc. of Cupertino, Calif.) When activated. FIG. 6B also shows a compass 611 indicating the orientation of the map displayed within the user interface 604 (eg, the map is oriented such that the top of the display 546-1 is north). In some implementations, the font size of the information displayed in the user interface 604 of the display 546-1 is larger than the font size of the information displayed in the user interface 602 of the touch screen 112. For example, when the system 540 is placed in a vehicle, the text on the display 546-1 indicates that the driver and passenger (s) of the vehicle are away from the display 546-1 by an arm Are made larger than on the touch screen 112 of the device 100 to improve the readability of the

  FIG. 6C illustrates displaying a user search query (eg, dry cleaning) entered in search box 609 of user interface 602. In one embodiment, the user manually enters a search query using a virtual keyboard within the user interface 602 of the device 100. In another example, the search query is voice input received from a user of device 100. FIG. 6C further illustrates maintaining the display of user interface 604 in FIG. 6B on display 546-1.

  FIG. 6D illustrates a user interface object (eg, a pin) associated with the dry cleaner represented by pins 614a and 614b near the current position 608 of device 100 in response to the input of the search query in FIG. 6C. The display in the interface 602 is shown. In some embodiments, search results (or stores) that match the user's search query may be automated based on proximity, store rating, amount of review, number of times the user has visited the store, and / or other factors. Selected (eg, ABC Cleaners). In FIG. 6D, an information box 613 corresponding to the automatically selected dry cleaner represented by pin 614 a is also displayed in the user interface 602. The information box 613 corresponds to the name of the dry cleaner (e.g. ABC Cleaners), store rating (e.g. three out of five stars), the amount of user reviews for the store (e.g. 100 reviews), and the store And an affordance 613a configured to display additional information. In some embodiments, the information box 613, when activated, causes the device to display additional information corresponding to the store in response to user touch input to any location within the information box 613. In some other embodiments, the information box 613, when activated, causes the device to display additional information corresponding to the store in response to a user touch input on the affordance 613a. FIG. 6D also illustrates detecting user touch input 615 on affordance 613 a on touch screen 112.

  FIG. 6D is from the current position 608 of the device 100 to the automatically selected search results (eg, dry cleaners represented by pins 614a corresponding to ABC Cleaners) in response to the search query input in FIG. 6C. It is further illustrated that the recommended route 616 of is displayed in the user interface 604. 6D, the user interface 604 includes a route 616, a turn-by-turn indication box 617 indicating the user's first direction (or turn point) on the route 616 and the name of the destination (eg, ABC Cleaners), and the device. Includes a “Start Indication” affordance 618 configured to initiate a turn-by-turn indication mode that provides turn-by-turn indication to ABC Cleaners using visual and / or audio guidance. FIG. 6D also shows a compass 611 indicating the orientation of the map displayed in the user interface 604 (eg, the driving direction of the vehicle indicated in the turn-by-turn instruction is oriented towards the top of the display 546-1 Thus, the map is oriented so that the top of display 546-1 is south).

  FIG. 6E illustrates displaying additional information corresponding to ABC Cleaners in the user interface 602 in response to detecting the user touch input 615 on the affordance 613a in FIG. 6D. In FIG. 6E, the user interface 602 includes information identifying the ABC Cleaners opening hours, the ABC Cleaners home page (or website), the ABC Cleaners phone number, and the ABC Cleaners address. In FIG. 6E, the user interface 602 is configured with affordances 619 configured to display instructions from the current location of the device to ABC Cleaners, and affordances configured to display the instructions from ABC Cleaners to the current location of the device. And 620. FIG. 6E also illustrates detecting user touch input 621 on affordance 619 on touch screen 112. FIG. 6E further illustrates maintaining the display of user interface 604 in FIG. 6D on display 546-1.

  FIG. 6F displays an interface in the user interface 602 for inputting the start point and the end point of the route in response to detection of the user touch input 621 on the affordance 619 on the touch screen 112 in FIG. 6E. Indicates that. In FIG. 6F, user interface 602 includes the address of the start of the route (eg, the current location of device 100) in box 622a, and the address of the end of the route (eg, ABC Cleaners, 1660 40th Avenue) in box 622b. , San Francisco, CA). In FIG. 6F, box 622a and / or box 622b can be edited using virtual keyboard 624 displayed within user interface 602, or alternatively using speech input. The affordance 622c, when activated, causes the device to replace the location entered in the box 622a with the location in the box 622b and replace the location entered in the box 622b with the location in the box 622a. In FIG. 6F, the user interface 602 includes a selectable route 623 from the current location of the device 100 to the previously entered destination. In FIG. 6F, user interface 602 further includes affordances 627a, 627b, and 627c configured to select different modes of transportation for route calculation. For example, in FIG. 6F, the "walk" mode of transport is currently selected, as indicated by the highlighted border around affordance 627b. FIG. 6F shows the “route” affordance 626a configured to calculate one or more routes from the start point in box 622a to the end point in box 622b, and the locations entered in boxes 622a and 622b. It also shows displaying a "cancel" affordance 626b configured to erase or re-display the user interface 602 displayed in FIG. 6D. FIG. 6F further illustrates detecting user touch input 628 on “root” affordance 626 a on touch screen 112. FIG. 6F further illustrates maintaining the display of user interface 604 in FIG. 6D on display 546-1.

  FIG. 6G is from the current position 608 of the device 100 to the dry cleaner represented by the pin 614a in response to detecting the user touch input 628 on the “route” affordance 626a on the touch screen 112 in FIG. 6F. It is shown that two recommended routes are displayed in the user interface 602. In FIG. 6G, the dialog box 629 shows that the currently selected first of the two recommended routes allows the user to walk from the current position 608 of the device 100 to the dry cleaner represented by pin 614a. Indicate that it will take 10 minutes and 0.2 miles. In FIG. 6G, the first route 631a is highlighted in the user interface 602, and the “route 1” affordance 631b, when activated, causes the device to select (and highlight) the route 631a. In FIG. 6G, the route 631a displayed in the user interface 602 is the same as the route 616 displayed in the user interface 604, but the orientation is different. FIG. 6G also shows displaying the route 632a in the user interface 602 and displaying the "route 2" affordance 632b configured to select (and highlight) the route 632a. In FIG. 6G, the “Start” affordance 630a, when activated, causes the device to initiate turn-by-turn instructions to the currently selected (or highlighted) route (eg, route 631a), “Erase” The affordance 630b, when activated, causes the device to clear the displayed route (eg, routes 631a and 632a) from the user interface 602 and causes the search box 609 to be displayed in the user interface 602. FIG. 6G further illustrates detecting user touch input 633 on “start” affordance 630 a on touch screen 112. FIG. 6G further illustrates maintaining the display of user interface 604 in FIG. 6D on display 546-1. In some embodiments, when the device 100 is connected to the display 546-1 and the display 546-1 is a vehicle display, the device 100 can drive the driving direction, not the walking direction, on the touch screen 112 and the display 546. -1 Display on both sides.

  FIG. 6H illustrates the user's first direction (or course change) and the name of the destination (eg, on the route 631a) in response to detecting the user touch input 633 on the “start” affordance 630a in FIG. 6G. , ABC Cleaners) is displayed in the user interface 602. In some embodiments, the turn-by-turn indication box 634 occupies the entire touch screen 112. FIG. 6H also illustrates detecting user touch input 635 on touch screen 112.

  6H further illustrates displaying a turn-by-turn instruction mode in the user interface 604 in response to the device 100 detecting the user touch input 633 on the start affordance 630a of FIG. 6G. In FIG. 6H, the current position of the device 100 is displayed in the user interface 604 as a marker 636 during turn-by-turn mode. In some embodiments, the turn-by-turn instruction mode displayed in FIG. 6H zooms in further on the map displayed in the user interface 604 than the maps displayed in FIGS. 6D-6G (eg, touch The enlargement level of the map displayed on the screen 112 is lower than the map displayed on the display 546-1.

  FIG. 6I illustrates displaying the dialog box 637 in the user interface 602 in response to detecting the user touch input 635 on the touch screen 112 in FIG. 6H. In FIG. 6I, the dialog box 637 is configured to pause the turn-by-turn instruction, “pause” affordance 638a, and the most recent search query within the map application 436 (eg, near the current location of the device 100). Includes a "back" affordance 638b configured to display the results of some dry cleaners. FIG. 6I also illustrates detecting user touch input 639 on the "back" affordance 638b. FIG. 6I further illustrates maintaining the display of user interface 604 in FIG. 6H on display 546-1.

  FIG. 6J shows the dry cleaner represented by pins 614a and 614b near the current position 608 of the device 100 in response to user touch input 639 (eg, tap input) on the “Return” affordance 638b of FIG. 6I. (E.g., corresponding to the most recent search query in map application 436 of FIG. 6C). FIG. 6J also illustrates detecting a pinch gesture that includes movement of the contacts 640 and 641 on the touch screen 112. 6J further illustrates maintaining the display of user interface 604 in FIG. 6H on display 546-1.

  FIG. 6K shows the user interface associated with the dry cleaner represented by pins 614a, 614b, 614c, and 614d near the current position 608 of device 100 in response to detecting the pinch gesture in FIG. 6J. An object (eg, a pin) is shown to be displayed in the user interface 602. For example, in FIG. 6K, the map is zoomed out (eg, displayed at a lower magnification level) in response to a pinch gesture, and a larger range of maps is displayed in the user interface 602. FIG. 6K also illustrates detecting a swipe gesture (sometimes referred to herein as a “drag gesture”) by user touch input 642 moving from position 642a to position 642b on touch screen 112. 6K further illustrates maintaining the display on the display 546-1 of the user interface 604 of FIG. 6H (eg, the user interface 604 zooms in response to detecting a pinch gesture on the touch screen display 112) Not out).

  FIG. 6L shows the user associated with the dry cleaner represented by pins 614a, 614b, 614c, 614e and 614f near the current position 608 of device 100 in response to detecting the swipe gesture in FIG. 6K. An interface object (eg, a pin) is shown to be displayed in the user interface 602. For example, in response to the swipe gesture of FIG. 6K, the map translates in the northeast direction. FIG. 6L also illustrates detecting user touch input 643 on a user interface object (eg, a pin) associated with the dry cleaner represented by pin 614 b on touch screen 112. FIG. 6L further illustrates maintaining the display of user interface 604 in FIG. 6H on display 546-1 (eg, user interface 604 is not scrolled in response to detecting a swipe gesture on touch screen display 112) ).

  FIG. 6M shows the dry being represented by pin 614b in response to detecting a user touch input 643 on a user interface object (eg, a pin) associated with the dry cleaner represented by pin 614b in FIG. 6L. A display of an information box 644 corresponding to the cleaning shop in the user interface 602 is shown. In FIG. 6M, the information box 644 contains the name of the dry cleaner (e.g., Mary Ann's Cleaners), the store rating (e.g., 5 out of 5 stars), the amount of user reviews for the store (e.g., 927). Review) and an affordance 644a that optionally includes additional information corresponding to the store. FIG. 6M also illustrates detecting user touch input 645 on affordance 644 a on touch screen 112. FIG. 6M further illustrates maintaining the display of user interface 604 in FIG. 6H on display 546-1.

  FIG. 6N illustrates displaying additional information corresponding to Mary Ann's Cleaners in the user interface 602 in response to detecting the user touch input 645 on affordance 644a in FIG. 6M. In FIG. 6N, user interface 602 identifies the hours of Mary Ann's Cleaners, the homepage (or website) of Mary Ann's Cleaners, the telephone number of Mary Ann's Cleaners, and the address of Mary Ann's Cleaners. Contains information to In FIG. 6N, user interface 602 displays affordances 646 configured to display instructions from the current location of the device to Mary Ann's Cleaners, and instructions from Mary Ann's Cleaners to the current location of the device. Also included is an affordance 647 configured as follows. FIG. 6N also illustrates detecting user touch input 648 on affordance 646 on touch screen 112. FIG. 6N further illustrates maintaining the display of user interface 604 in FIG. 6H on display 546-1.

  FIG. 6O displays an interface in the user interface 602 for inputting the start point and the end point of the route in response to detecting the user touch input 648 on the affordance 646 on the touch screen 112 in FIG. 6N. Indicates that. In FIG. 6O, the user interface 602 includes the address of the start of the route (eg, the current location of the device 100) in box 649a, and the address of the end of the route (eg, Mary Ann's Cleaners, 1291 Judah Street, San Francisco, CA). In FIG. 6O, box 649a and / or box 649b can be edited using virtual keyboard 651 displayed within user interface 602, or alternatively using voice input. The affordance 649c, when activated, causes the device to replace the location entered in the box 649a with the location in the box 649b and replace the location entered in the box 649b with the location in the box 649a. In FIG. 6O, the user interface 602 includes a selectable route 650 from the current location of the device to the previously entered destination. In FIG. 6O, user interface 602 further includes affordances 653a, 653b, and 653c configured to select different modes of transportation for route calculation. For example, in FIG. 6O, the "walk" mode of transport is currently selected, as indicated by the highlighted border around affordance 653b. FIG. 6O “root” affordance 652a configured to calculate one or more routes from the start point in box 649a to the end point in box 649b, and the locations entered in boxes 649a and 649b. It also shows displaying a "cancel" affordance 652b configured to erase or re-display the user interface 602 displayed in FIG. 6M. FIG. 6O further illustrates detecting user touch input 655 on “root” affordance 654 on touch screen 112 (eg, the affordance included in virtual keyboard 651). In some embodiments, the “route” affordance 654, when activated, calculates and / or displays one or more routes from the start point in box 649a to the end point in box 649b when activated. (Eg, “route” affordance 654 functions similar to “route” affordance 652 a). FIG. 6O further illustrates maintaining the display of user interface 604 in FIG. 6H on display 546-1.

  FIG. 6P is from the current position 608 of the device 100 to the dry cleaner represented by pin 614b in response to detecting the user touch input 655 on the “route” affordance 654 on the touch screen 112 in FIG. 6O. It is shown that two recommended routes are displayed in the user interface 602. In FIG. 6P, the dialog box 656 of the user interface 602 is a dry cleaner where the automatically selected first of the two recommended routes is represented by the user from the current position 608 of the device 100 by the pin 614b. Indicates that it takes 5 minutes and 0.4 miles to walk up. In FIG. 6P, the first route 658a is highlighted in the user interface 602, and the “route 1” affordance 658b, when activated, causes the device to select (and highlight) the route 658b. In FIG. 6P, the “Start” affordance 657a, when activated, causes the device to initiate turn-by-turn instructions to the currently selected (or highlighted) route (eg, route 658a), “Erase” The affordance 657b, when activated, causes the device to clear the displayed route (eg, routes 658a and 659a) from the user interface 602 and causes the search box 609 to be displayed in the user interface 602. FIG. 6P further illustrates detecting user touch input 660 on “route 2” affordance 659 b on touch screen 112. FIG. 6P further illustrates maintaining the display of user interface 604 in FIG. 6H on display 546-1.

  FIG. 6Q displays the route 659a (route 2) as a selected (or highlighted) route in response to the touch input 660 detected on the “route 2” affordance 659b in FIG. 6P. Show. In FIG. 6Q, the user interface 602 has 6 minutes and 0 for the second of the two recommended routes to walk the user from the current position 608 of the device 100 to the dry cleaner represented by pin 614b. Includes a dialog box 656 indicating that it will take .4 miles. FIG. 6Q further illustrates detecting a user touch input 661 on the “Start” affordance 657a on the touch screen 112. FIG. 6Q further illustrates maintaining the display of user interface 604 in FIG. 6H on display 546-1.

  FIG. 6R illustrates the user's first direction (or course change) and the name of the destination (e.g., on the route 659a) in response to detecting the user touch input 661 on the "Start" affordance 657a in FIG. 6Q. , And Mary Ann's Cleaners) are displayed in the user interface 602. In some embodiments, the turn-by-turn indication box 662 occupies the entire touch screen 112.

  FIG. 6R further illustrates displaying the turn-by-turn instruction mode in the user interface 604 in response to detecting the user touch input 661 on the “Start” affordance 657a in FIG. 6Q (eg, user interface 604). If the destination and route are selected and a turn-by-turn instruction is initiated, even if the device was previously synchronized with the user interface 602 in response to the previous input, the device will have the same route for both user interfaces Synchronize the user interface 602 and the user interface 604 to indicate an indication to the same destination using In FIG. 6R, the user interface 604 is a turn-by-turn indication box 663 indicating the route 659a, the user's first direction (or course change point) on the route 659a, and the name of the destination (eg, Mary Ann's Cleaners). And a marker 654 indicating the current position of the device 100. FIG. 6R also shows a compass 611 indicating the orientation of the map displayed in the user interface 604 (eg, the map is oriented such that the top of display 546-1 is east). FIG. 6R further illustrates detecting user touch input 665 on display 546-1.

  FIG. 6S illustrates displaying dialog box 637 in user interface 602 in response to detecting user touch input 665 on display 546-1 in FIG. 6R. In FIG. 6S, the dialog box 637 is configured to pause the turn-by-turn instruction, “pause” affordance 638a, and the latest search query within the map application 436 (eg, near the current location of the device 100). Includes a "back" affordance 638b configured to display the results of some dry cleaners.

  FIG. 6S further illustrates displaying affordance 612 in user interface 604 in response to detecting user touch input 665 on display 546-1 in FIG. 6R. The affordance 612, when activated, causes the device to call a digital assistant (eg, Siri manufactured by Apple Inc. of Cupertino, California). FIG. 6S also illustrates detecting user touch input 666 on affordance 612 on display 546-1. In response to detecting the user touch input 666 on the affordance 612, the digital assistant is called, and the digital assistant guides the user by voice "What is your business?" In response to the prompt, the user may, by voice, ask, for example, the digital assistant "tell me a nearby coffee shop". Alternatively, in some embodiments, the digital assistant is called by a predetermined voice command (e.g., "Hi, Siri").

  FIG. 6T illustrates maintaining the display of the user interface 602 in FIG. 6S on the touch screen 112. FIG. 6T further illustrates displaying the digital assistant's dialog box 667 within the user interface 604. The digital assistant's dialog box 667 indicates that the digital assistant is searching for a coffee shop in response to the user's voice request (e.g., "tell me a nearby coffee shop").

  FIG. 6U illustrates that, in response to a user voice request (eg, “tell me a nearby coffee shop”), user interface objects (eg, Is shown to be displayed in the user interface 602. In some embodiments, search results (or stores) that match the voice request are automatically based on proximity, store rating, amount of review, number of times the user has visited the store, and / or other factors. Selected (eg, Java Haus). In FIG. 6U, an information box 669 corresponding to the automatically selected coffee shop 668a is also displayed in the user interface 602. The information box 669 corresponds to the name of the coffee shop (eg, Java Haus), the shop rating (eg, 4 out of 5 stars), the amount of user reviews for the shop (eg, 320 reviews), and the shop. Includes affordances 669a configured to display additional information. Providing additional information in the user interface 602 means that the passenger of the vehicle operating the device 100 provides the additional information to the driver of the vehicle and / or to the additional information provided in the user interface 602. It is possible to select the destination based on.

  FIG. 6U shows a recommended route 671 from the current location 608 of the device 100 to the coffee shop 668a (eg, Java Haus) in response to the user's voice request (eg, “tell me a nearby coffee shop”). It further indicates that it is displayed inside. In some embodiments, routes for automatically selected search results are displayed in the user interface 604. In FIG. 6U, the user interface 604 includes a route 671, a turn-by-turn indication box 670 indicating the user's first direction (or course change point) on the route 671 and the name of the destination (e.g. Java Haus), and a turn. It includes a "Start Indication" affordance 618 that is configured to initiate the vital mode. FIG. 6U also shows a compass 611 indicating the orientation of the map displayed in the user interface 604 (eg, the map is oriented so that the top of the display 546-1 is north). FIG. 6U further illustrates detecting a user touch input 672 on the “Start indicating” affordance 618 on display 546-1.

  FIG. 6V shows the user's first direction (or course change point) and destination name (on the route 671) in response to detecting the user touch input 672 on the “Start indication” affordance 618 in FIG. 6U. For example, it indicates that a turn-by-turn instruction box 673 indicating Java Haus) is displayed in the user interface 602. In some embodiments, the turn-by-turn indication box 673 occupies the entire touch screen 112.

  FIG. 6V further illustrates displaying the turn-by-turn instruction mode in the user interface 604 in response to detecting the user touch input 672 on the “Start indication” affordance 662 on the display 546-1 in FIG. 6U. Show. In FIG. 6V, the current position of the device 100 is displayed in the user interface 604 as a marker 674 during turn-by-turn mode. In some embodiments, the turn-by-turn instruction mode displayed in FIG. 6V zooms in further on the map displayed in the user interface 604 than the map displayed in FIG. 6U.

  6W-6FF illustrate exemplary user interfaces for sending update information to affected displays in accordance with some embodiments. The user interfaces of these figures are used to illustrate the processes described below, including the processes of FIGS. 9A-9C.

  FIG. 6W illustrates an external information provision system 540 (herein “communicately” to a portable multifunction device 100 (sometimes referred to herein as “device 100”) according to some embodiments. Show two displays contained within the system 540). In some embodiments, the external information provision system 540 is an entertainment and / or navigation system implemented in a vehicle. In some embodiments, display 546-1 is associated with first external information provision system 540-1, and display 546-2 is associated with second external information provision system 540-2. In some embodiments, display 546-1 is a primary display (e.g., dashboard or vehicle navigation display) and display 546-2 is an auxiliary display (e.g., backseat or entertainment display). In some embodiments, display 546-1 and display 546-2 are touch-sensitive displays that include a touch-sensitive surface on the display and are configured to receive one or more touch inputs. FIG. 6W illustrates a jog dial 675 (eg, a pushable and / or rotatable mechanical knob associated with the system 540 configured to control one or both of the display 546-1 and the display 546-2. Or dial).

  FIG. 6W illustrates a user interface object (eg, a pin) associated with the dry cleaner represented by pins 614a and 614b near the current position 608 of the device 100 in a map / navigation application, and the user of display 546-1. The display in the interface 604 is shown. For example, these pins are displayed in the user interface 604 in response to a search query (eg, “dry clean” entered in the search box 609 in the user interface 602 of the device 100 in FIG. 6C). In some embodiments, search results (or stores) that match the user's search query are automatically based on proximity, store rating, amount of review, number of times the user has visited the store, and other factors. Selected (eg, ABC Cleaners). In FIG. 6W, destination information 678 corresponding to the automatically selected dry cleaner represented by pin 614 a is also displayed in the user interface 604. Destination information 678 includes the name of the dry cleaner (e.g. ABC Cleaners), store rating (e.g. three out of five stars), the amount of user reviews for the store (e.g. 100 reviews), and the device 100 Including the distance from the current position 608 to the store. In some embodiments, in response to a touch input to any location within the dialog box associated with the destination information 678 corresponding to the dry cleaner 514a, the currently selected search result (e.g. Turn-by-turn instructions to the dry cleaning shop) are displayed in the user interface 604. In some other embodiments, in response to depression of the jog dial 675, turn-by-turn instructions to the currently selected search result (eg, the dry cleaner represented by pin 614a) are displayed in the user interface 604 Be done. FIG. 6W also shows a compass 611 indicating the orientation of the map displayed in the user interface 604 (eg, the map is oriented so that the top of display 546-1 is north).

  FIG. 6W illustrates detecting pinch gestures by user touch inputs 679 and 680 on display 546-1. In response to detecting user touch input 679 and 680 (or pinch gesture), display 546-1 (or system 540) sends input information indicating user touch input 679 and 680 (or pinch gesture) to device 100 Do. The input information describes an identifier associated with the touch sensitive surface of display 546-1 (eg, a unique identifier associated with the touch sensitive surface of display 546-1) as well as user touch inputs 679 and 680 (or pinch gestures) Contains input description information. In some embodiments, the input description information includes the position of the user touch input 679 and 680, the magnitude of movement of the user touch input 679 and 680, the length of time the user touch input 679 and 680 is detected, the user touch It includes raw data describing user touch inputs 679 and 680, such as the surface area of inputs 679 and 680 and / or other information describing user touch inputs 679 and 680. In some other embodiments, the input description information describes a pinch gesture, such as the type of gesture performed, the position of the gesture, the magnitude of movement of the gesture, and / or other information that describes the gesture. Contains data.

  In response to receiving the input information associated with the user touch inputs 679 and 680 (or pinch gesture), the device 100 displays the identifier included in the input information on the display 546-based on the mapping of the input source to the display. Determine that it is associated with 1 (e.g., associate a touch event (or gesture) with the touch sensitive surface of display 546-1). After determining that the touch event detected in FIG. 6W is associated with the display 546-1 (eg, the affected display), the device 100 displays the display state of the display 546-1 (eg, a user shown in FIG. 6W). The interface 604 is also in a search result or destination selection mode). In some embodiments, device 100 determines the display state of the display based on the information contained in the input information. In some other embodiments, device 100 determines the display state of the display by querying system 540 for such information.

  FIG. 6W further illustrates displaying a list of songs 677 corresponding to the album in the media player application within the user interface 676 of the display 546-2.

  FIG. 6X illustrates a user interface object (eg, a pin) associated with the dry cleaner represented by pins 614a, 614b, 614c, and 614d near the current position 608 of device 100 on display 546-1. The display in the user interface 604 is shown. For example, the map displayed in the user interface 604 in FIG. 6X is a zoom out of the map displayed in the user interface 604 in FIG. 6W. Display 546-1 is an updated user interface corresponding to the pinch gesture detected on display 546-1 in FIG. 6W, and the display state of user interface 604 in FIG. 6W (eg, search results or destination selection mode) In response to receiving update information from the device 100, the zoomed out map is displayed in the user interface 604 in FIG. 6X.

  FIG. 6X also illustrates detecting rotation 681 of jog dial 675 (eg, approximately 90 degrees clockwise rotation). In response to detecting the rotation 681, the jog dial 675 (or the system 540) transmits input information indicating the rotation 681 to the device 100. The input information includes an identifier associated with the jog dial 675 (eg, a unique identifier associated with the jog dial 675) and input description information that describes the rotation 681. For example, the input description information includes raw data describing the rotation 681 such as the magnitude (90 degrees) and the direction (clockwise) of the rotation. In response to receiving the input information associated with rotation 681, device 100 determines based on the mapping of the input source to the display that the identifier contained within the input information is associated with display 546-1. (For example, associate a rotation event with the jog dial 675 currently mapped to the display 546-1). After determining that the detected rotation event in FIG. 6X is associated with the display 546-1, the device 100 displays the display state of the display 546-1 shown in FIG. 6X (e.g., search results or destination for the user interface 604). In the selection mode) is also determined. FIG. 6X further illustrates maintaining the display of the list of songs 677 in user interface 676 in FIG. 6W on display 546-2 (eg, jog dial 675 is not associated with the display on which user interface 676 is displayed). If the user interface 676 is not updated in response to the rotation of the jog dial 675).

  FIG. 6Y illustrates displaying destination information 682 corresponding to the selected dry cleaner represented by pin 614 d in user interface 604 of display 546-1. In FIG. 6Y, the display 546-1 displays the updated user interface corresponding to the rotation 681 detected by the jog dial 675 in FIG. 6X and the display status of the user interface 604 in FIG. 6X (eg, search results or destination selection mode) And the destination information 682 corresponding to the selected dry cleaner (e.g., Suzie's Cleaners) represented by the pin 614d. In some embodiments, destination information (and thus the selected destination) is scrolled according to the direction and magnitude of rotation of the jog dial 675. For example, when a search result or destination selection mode is displayed, a 90 degree rotation of the jog dial 675 selects the next search result, a clockwise rotation scrolls the search result from north to south, an counterclockwise Rotation around scrolls the search results from south to north. In FIG. 6Y, in response to detecting rotation 681 (eg, a 90 degree clockwise rotation) of the jog dial 675 in FIG. 6X, the dry cleaner represented by pin 614d is selected (or scrolled to that store) , Destination information 682 corresponding to a dry cleaner represented by pin 614 d (eg, the next dry cleaner located south of the dry cleaner represented by pin 614 a shown in FIG. 6X) It is displayed in 604.

  FIG. 6Y also illustrates detecting rotation 683 of jog dial 675 (eg, about 180 degrees of counterclockwise rotation). In some embodiments, in response to detecting rotation 683, jog dial 675 (or system 540) sends input information indicative of rotation 683 to device 100. The input information includes an identifier associated with the jog dial 675 (e.g., a unique identifier associated with the jog dial 675) and input description information describing the rotation 683. For example, the input description information includes raw data describing the rotation 683 such as the magnitude (180 degrees) and the direction (counterclockwise) of the rotation. In response to receiving the input information associated with rotation 683, device 100 determines based on the mapping of the input source to the display that the identifier included in the input information is associated with display 546-1. (For example, associate a rotation event with the jog dial 675 currently mapped to the display 546-1). After determining that the detected rotation event in FIG. 6Y is associated with the display 546-1, the device 100 displays the display state of the display 546-1 shown in FIG. 6Y (e.g., search results or destinations for the user interface 604). In the selection mode) is also determined. FIG. 6Y further illustrates maintaining a display of the list of songs 677 in user interface 676 in FIG. 6W on display 546-2.

  FIG. 6Z illustrates displaying destination information 684 corresponding to a selected dry cleaner (eg, Mary Ann's Cleaners) represented by pin 614b in user interface 604 of display 546-1. In FIG. 6Z, the display 546-1 displays the updated user interface corresponding to the rotation 683 detected by the jog dial 675 in FIG. 6Y and the display status of the user interface 604 in FIG. 6Y (e.g., search results or destination selection mode) In response to receiving update information from device 100, the destination information 684 corresponding to the selected dry cleaner represented by pin 614b is displayed. In FIG. 6Z, in response to detecting rotation 683 of jog dial 675 (eg, 180 degrees counterclockwise rotation) in FIG. 6Y, the dry cleaner represented by pin 614b is selected (or scrolls to that store) Destination information 684 corresponding to the dry cleaner represented by pin 614b (eg, two dry cleaners north of the dry cleaner represented by pin 614d shown in FIG. 6Y). It is displayed inside.

  FIG. 6Z also illustrates detecting a press or press of the jog dial 675 by the user touch input 685. In some embodiments, in response to detecting user touch input 685, jog dial 675 (or system 540) transmits input information indicative of user touch input 685 to device 100. The input information includes an identifier associated with the jog dial 675 (eg, a unique identifier associated with the jog dial 675) and input description information describing the user touch input 685. For example, the input description information includes raw data describing the user touch input 685, such as the force of the input and the amount of depression of the jog dial 675. In response to receiving the input information associated with the user touch input 685, the device 100 determines that the identifier included in the input information is associated with the display 546-1 based on the mapping of the input source to the display Determine (e.g., associate a press or press event with the jog dial 675 currently mapped to the display 546-1). After determining that the press event detected in FIG. 6Z is associated with the display 546-1, the device 100 displays the display state of the display 546-1 shown in FIG. 6Z (e.g., search results or a destination for the user interface 604). In the selection mode) is also determined. FIG. 6Z further illustrates maintaining a display of the list of songs 677 in the user interface 676 in FIG. 6W on display 546-2.

  FIG. 6AA shows a turn-by-turn instruction mode from the marker 664 indicating the current position of the device 100 to the dry cleaner (eg, Mary Ann's Cleaners) represented by the pin 614b, the user interface 604 of the display 546-1. Indicates to display within. 6AA, display 546-1 displays the updated user interface corresponding to user touch input 685 detected by jog dial 675 in FIG. 6Z and the display status of user interface 604 in FIG. 6Z (e.g., search results or destinations). In response to receiving update information including the selection mode) from the device 100, the turn-by-turn instruction mode is displayed in the user interface 604. In FIG. 6AA, in response to the detection of the user touch input 685 for pressing the jog dial 675 in FIG. 6Z, the dry cleaner store represented by the pin 614b from the marker 664 indicating the current position of the device 100 on the route 659a. For example, the turn-by-turn instruction to the one selected by the rotation 683 of the jog dial 675 in FIG. 6Y is displayed in the turn-by-turn instruction box 663 of the user interface 604. FIG. 6AA also shows a compass 611 indicating the orientation of the map displayed in the user interface 604 (eg, the map is oriented so that the top of display 546-1 is east).

  FIG. 6AA also illustrates detecting rotation 686 of jog dial 675 (eg, approximately 90 degrees of counterclockwise rotation). In response to detecting the rotation 686, the jog dial 675 (or system 540) transmits input information indicating the rotation 686 to the device 100. The input information includes an identifier associated with jog dial 675 (e.g., a unique identifier associated with jog dial 675) and input description information describing rotation 686. For example, the input description information includes raw data describing rotation 686, such as the magnitude (90 degrees) and direction (counterclockwise) of the rotation. In response to receiving the input information associated with rotation 681, device 100 determines based on the mapping of the input source to the display that the identifier contained within the input information is associated with display 546-1. (For example, associate a rotation event with the jog dial 675 currently mapped to the display 546-1). After determining that the detected rotation event in FIG. 6AA is associated with display 546-1, device 100 displays the display state of display 546-1 shown in FIG. 6AA (eg, turn-by-turn instruction mode in user interface 604) ) Is also determined. FIG. 6AA further illustrates maintaining a display of the list of songs 677 in user interface 676 in FIG. 6W on display 546-2.

  FIG. 6BB illustrates displaying a turn-by-turn instruction mode in the user interface 604 of the display 546-1. For example, the turn-by-turn instruction mode displayed in the user interface 604 in FIG. 6BB is a zoom out of the turn-by-turn instruction mode displayed in the user interface 604 in FIG. 6AA. 6BB, display 546-1 displays the updated user interface corresponding to rotation 686 detected by jog dial 675 in FIG. 6AA, and the display state (eg, turn-by-turn indication mode) of user interface 604 in FIG. 6AA. In response to receiving from the device 100 the update information that is included, the zoomed out turn-by-turn instruction mode is displayed in the user interface 604 in FIG. 6BB. FIG. 6BB further illustrates maintaining the display of the list of songs 677 in the user interface 676 in FIG. 6W on display 546-2.

  FIG. 6CC illustrates displaying the vehicle reverse mode in the user interface 604 of the display 546-1. For example, the vehicle reverse mode displayed in the user interface 604 includes a video feed from a reverse camera implemented in the vehicle. In some embodiments, the vehicle reverse mode occupies the entire display 546-1. When the system 540 is implemented in a vehicle, the display 546-1 displays the vehicle reverse mode in the user interface 604 while the vehicle reverses. In some embodiments, the vehicle reverse mode supersedes any update information received from device 100 by displays 546-1 and 546-2. In some embodiments, the vehicle reverse mode is displayed on one of the displays (e.g., display 546-1) and not on the other display (e.g., display 546-2).

  Similarly, FIG. 6 CC also shows displaying the vehicle reverse mode in the user interface 676 of the display 546-2. For example, the vehicle reverse mode displayed in the user interface 676 includes a video feed from a reverse camera implemented in the vehicle. In some embodiments, the vehicle reverse mode occupies the entire display 546-2. If the system 540 is implemented in a vehicle, the display 546-2 displays the vehicle reverse mode in the user interface 676 while the vehicle reverses. In some other embodiments, the display 546-2 does not display the vehicle reverse mode in the user interface 676 while the vehicle reverses, and maintains the display of the interface 676.

  FIG. 6DD illustrates displaying the turn-by-turn instruction mode in the user interface 604 at 6BB on display 546-1 when the vehicle no longer reverses.

  FIG. 6DD also illustrates detecting rotation 688 (eg, approximately 90 degrees clockwise) of jog dial 675. In response to detecting the rotation 688, the jog dial 675 (or the system 540) transmits input information indicating the rotation 688 to the device 100. The input information includes an identifier associated with jog dial 675 (e.g., a unique identifier associated with jog dial 675) and input description information describing rotation 688. For example, the input descriptive information includes raw data describing rotation 688, such as the magnitude (90 degrees) and direction (clockwise) of the rotation.

  Device 100 detects a change in the mapping of the input source to the display that maps jog dial 675 to display 546-2 rather than display 546-1. For example, the device 100 may communicate with the external information providing system 540 (e.g., control the display to indicate the output of the backward camera), or the user controls the user interface in the first display with their respective controls. Input source based on the input source mapping update request, such as switching the respective control from the first display mode to the second display mode in which the respective control controls the user interface in the second display Change the mapping to the display of. In response to receiving the input information associated with rotation 688, device 100 may associate an identifier included in the input information with display 546-2 based on the modified mapping of the input source to the display. (E.g., associate a rotation event with the jog dial 675 currently mapped to the display 546-2). After determining that the rotational event detected in FIG. 6DD is associated with display 546-2, device 100 displays the display state of display 546-2 shown in FIG. 6DD (eg, user interface 676 is in album song list mode). Is also determined.

  FIG. 6DD illustrates returning to displaying the list of songs 677 in the user interface 676 in FIG. 6W on the display 546-2 when the vehicle no longer reverses.

  FIG. 6EE illustrates displaying the turn-by-turn instruction mode in user interface 604 in FIG. 6BB on display 546-1. FIG. 6EE also illustrates displaying the downward scrolled list of songs 677 in user interface 676 of display 546-2. In FIG. 6EE, display 546-2 includes an updated user interface corresponding to rotation 688 detected by jog dial 675 in FIG. 6DD, and a display state (eg, album song list mode) of user interface 676 in FIG. 6DD. In response to receiving the update information from the device 100, the downward scrolling list of songs 677 is displayed in the user interface 676. In FIG. 6EE, the list of songs 677 is scrolled down in the user interface 604 in response to detecting the rotation 688 (eg, approximately 90 degrees clockwise rotation) of the jog dial 675 in FIG. 6DD.

  FIG. 6EE detects the upward swipe gesture (sometimes referred to herein as a “drag” gesture) by user touch input 689 moving from position 689a to position 689b on display 546-2. Show more. In response to detecting user touch input 689 (or a swipe gesture), display 546-2 (or system 540) transmits input information indicating touch input 689 (or a swipe gesture) to device 100. The input information describes the identifier associated with the touch sensitive surface of display 546-2 (e.g., a unique identifier associated with the touch sensitive surface of display 546-2) and input descriptive information describing touch input 689 (or a swipe gesture) including. In response to receiving the input information associated with the touch input 689 (or swipe gesture), the device 100 associates the identifier contained in the input information with the display 546-2 based on the mapping of the input source to the display (E.g., associating a touch event (or gesture) with the touch sensitive surface of display 546-2). After determining that the touch event detected in FIG. 6EE is associated with display 546-2, device 100 displays the display state of display 546-2 shown in FIG. 6EE (eg, user interface 676 is in album song list mode). Is also determined.

  FIG. 6FF shows that the turn-by-turn instruction mode in the user interface 604 in FIG. 6BB is displayed on the display 546-1. FIG. 6FF also illustrates displaying the upwardly scrolled list of songs 677 in the user interface 676 of display 546-2. In FIG. 6FF, display 546-2 displays the updated user interface corresponding to the swipe gesture detected on display 546-2 in FIG. 6EE, and the display state of user interface 676 in FIG. 6EE (e.g., album song list mode). In response to receiving update information from the device 100, the upward scrolled list of the song 677 is displayed in the user interface 604. In FIG. 6FF, the list of songs 677 is scrolled upward in the user interface 604 in response to detecting the upward swipe gesture in FIG. 6EE.

  7A-7D are flow diagrams illustrating a method 700 for synchronizing two or more displays in accordance with some embodiments. Method 700 is performed at a first electronic device (e.g., device 300 (FIG. 3) or portable multifunction device 100 (FIG. 1A)) comprising a first display and a touch sensitive surface. In some embodiments, the first display is a touch screen display (eg, touch screen 112) and the touch sensitive surface is on the first display. In some embodiments, the first display is separate from the touch sensitive surface. Some operations in method 700 are optionally combined and / or the order of some operations is optionally changed.

  As described below, method 700 provides an intuitive way to synchronize two or more displays. This method reduces the cognitive burden on the user when displaying information on more than one display, thereby creating a more efficient human-machine interface. For battery operated electronic devices, power can be saved and the interval between charging the battery by allowing the user to use the information displayed on two or more displays faster and more efficiently Increases.

  The first electronic device displays a first user interface on a first display (702). In some embodiments, the first electronic device is a portable multifunction computing device, such as a smartphone or a tablet computer. For example, FIGS. 6A-6V illustrate portable multifunction device 100 (sometimes referred to herein as “device 100”) displaying user interface 602 (eg, a first user interface) on touch screen 112. Show.

  The first user interface enables selection between multiple selectable objects (e.g., search results corresponding to previously performed search operations) (704). For example, FIG. 6D illustrates a user interface object (eg, a pin) corresponding to the dry cleaner represented by pins 614a and 614b displayed in the user interface 602 of the device 100. In FIG. 6D, the dry cleaner represented by pins 614a and 614b is displayed in the user interface 602 in response to a search query for "dry cleaning" near the current position 608 of the device 100 in FIG. 6C.

  The first user interface corresponds to a second user interface displayed on a second display different from the first display. In some embodiments, the first user interface and the second user interface are user interfaces to the same corresponding application (eg, an application running on the first electronic device). The user interface 602 (eg, the first user interface) displayed on the touch screen 112 of the device 100 is a display 546-1 of the external information providing system 540 (sometimes referred to herein as a "system 540"). It corresponds to a user interface 604 (for example, a second user interface) displayed on the (for example, touch screen display). In some embodiments, display 546-1 is a primary display of a plurality of displays associated with system 540. In some embodiments, display 546-1 is implemented within the dashboard of the vehicle and is visually accessible to the driver and / or passenger of the vehicle.

  A first object of the plurality of selectable objects is displayed in the second user interface as the selected object (eg, the first user object is currently selected in the second user interface) Object)). For example, FIG. 6D shows an information box 613 corresponding to the dry cleaner represented by the pin 614a displayed in the user interface 602. An information box 613 is displayed in the user interface 602 due to the fact that the dry cleaner represented by the pin 614a has been automatically selected based on predetermined criteria. For example, FIG. 6D further illustrates the route 616 from the current position 608 of the device 100 to the automatically selected dry cleaner represented by the pin 614a displayed in the user interface 604 of the display 546-1. .

  In some embodiments, the first object is automatically selected as the selected object based on predetermined object selection criteria (eg, initiate a search and receive one or more sets of search results) After that, the first object corresponding to the most relevant search result is automatically selected as the selected object without user intervention (706). For example, in FIG. 6D, the dry cleaner represented by pin 614a is automatically selected based on proximity, store rating, amount of review, number of times the user has visited the store, and / or other factors. .

  In some embodiments, the plurality of selectable objects correspond to a plurality of search results generated in response to a search query (e.g., typing or spoken search query) and the selected object is currently selected Corresponding to the search result (708). For example, in FIG. 6D, a user interface object (eg, a pin) corresponding to the dry cleaner represented by pins 614a and 614b is responsive to the search query “dry clean” entered in search box 609 in FIG. 6C. , Displayed in the user interface 602 of the device 100. In some embodiments, the search query is manually entered into the search box 609 using a virtual keyboard displayed within the user interface 602 of the touch screen 112. In some other embodiments, the search query is voiced into search query box 609 via voice input from the user of device 100. For example, in FIG. 6D, the dry cleaner represented by pin 614a is automatically selected from a plurality of search results including dry cleaners represented by pins 614a and 614b. Also in FIG. 6D, the user interface 604 of the system 540 displays the route 616 from the current position 608 of the user device 100 to the selected dry cleaner represented by the pin 614a.

  In some embodiments, the plurality of selectable objects correspond to the plurality of map locations, and the selected objects correspond to the currently selected destination (710). For example, in FIG. 6D, a user interface object (eg, a pin) corresponding to the dry cleaner represented by pins 614a and 614b is a map position relative to the current position 608 of device 100. Also in FIG. 6D, the user interface 604 of the system 540 displays a route 616 from the current position 608 of the user device 100 to the map position for the selected dry cleaner represented by the pin 614a.

  In some embodiments, the plurality of selectable objects correspond to the plurality of navigation routes, and the selected objects correspond to the currently selected navigation route (712). For example, FIG. 60 shows “route 1” affordance 631 b and “route 2” affordance 632 b corresponding to the routes 631 a and 631 b from the current position 608 of the user device 100 to the dry cleaner represented by pins 614 a and 614 b, respectively. Show multiple selectable user interface objects, including: Also, in FIG. 6D, the user interface 604 of the system 540 corresponds to the highlighted route 631a in the user interface 602 from the current position 608 of the user device 100 to the selected dry cleaner represented by the pin 614a. The route 616 to be displayed is displayed.

  In some embodiments, the plurality of selectable objects correspond to a plurality of content items that may be played (or provided) using the device (e.g., the plurality of selectable objects may be songs, playlists, The selected object (audiobook, radio station, etc.) corresponds to the currently selected content item (714). For example, each song in the song list is automatically based at least in part on the playback volume of each song, the ordinal position of each song in the album, the last song played, and other factors. The selected song is displayed on the user interface of the touch screen 112 of the device 100. In this example, each automatically selected song is displayed on the user interface of display 546-1 as being currently played.

  In some embodiments, the plurality of selectable objects correspond to the plurality of applications, and the selected objects correspond to the currently selected application (716).

  In some embodiments, the second display is a display coupled to a second electronic device different from the first electronic device (eg, replacing the native GUI of the second electronic device) (718). For example, FIGS. 6A-6V illustrate a display 546-1 (eg, a second display) coupled to a system 540 (eg, a second electronic device) that is different (and separate) from the device 100.

  In some embodiments, the second electronic device controls an in-vehicle entertainment system having a native graphical user interface, and the second user interface is at least a portion of the native graphical user interface of the second electronic device. Instead, it is displayed on the second display (e.g. the second user interface is generated by an application running on the first electronic device and is part of the in-vehicle entertainment system's native graphical user interface or Displayed in place of all) (720). For example, in FIGS. 6A-6V, system 540 controls an in-vehicle entertainment system that includes display 546-1. For example, the user interface 604 of FIG. 6A, which includes climate control for a vehicle, is the native graphical user interface of the in-vehicle entertainment system. For example, FIGS. 6B-6V illustrate a user interface 604 that includes a map displayed on display 546-1 instead of the native graphical user interface (eg, climate control) as shown in FIG. 6A.

  In some embodiments, the first electronic device receives the respective input from the control of the in-vehicle entertainment system (eg, a tap or swipe gesture on a touch sensitive surface associated with the second display, the second display Information corresponding to the button pressing operation on the button associated with the and / or the knob rotating operation on the knob associated with the second display) is received (722). For example, FIG. 6U illustrates that system 540 detects user touch input 672 on “Start Indication” affordance 618 in user interface 604 of display 546-1. In this example, system 540 sends information to device 100 indicating that user touch input 672 on “Start Indication” affordance 672 in user interface 604 of display 546-1 has been detected.

  In some embodiments, in response to receiving information corresponding to each input, the first electronic device updates the first user interface and the second user interface according to the respective input (724) ). For example, FIG. 6V shows the device 100 updating the user interface 602 to display the turn-by-turn indication box 673 on the touch screen 112 and the user to display the turn-by-turn mode on the display 546-1. It shows that the interface 604 is updated. In some embodiments, while the user interface 604 is in turn-by-turn mode, the map displayed on the display 546-1 is zoomed in further than when the user interface 604 is not in turn-by-turn mode.

  In some embodiments, both the first user interface and the second user interface are generated by the same application running on the first electronic device (eg, the application running on the smartphone is Generate (726) a user interface for the phone and a user interface for display on the display of the in-vehicle entertainment system of the vehicle. For example, FIG. 6A shows user touch input 606 selecting map application 436 in user interface 602. For example, FIGS. 6B-6V are implemented on device 100 and include user interface 602 (eg, a first user interface) on touch screen 112 and user interface 604 (eg, a second user) on display 546-1. A map application 436 is shown that generates both interfaces).

  While the first object is displayed in the second user interface as the selected object (728), the first electronic device can generate information about the second one of the plurality of selectable objects. A first input from a user (e.g., the first of the first electronic devices) corresponding to a request to display in one of the user interfaces (e.g., without selecting the second object as the selected object) A tap or swipe gesture on one touch sensitive surface is detected (730). For example, FIG. 6L shows that the device 100 is dry represented by pin 614b on a map including dry cleaners represented by pins 614a, 614b, 614c, 614e, and 614f displayed in the user interface 602. 15 illustrates detecting a user touch input 643 on a user interface object (eg, a pin) corresponding to a cleaner. For example, FIG. 6L displays in the user interface 604 of the display 546-1 a turn-by-turn instruction mode that the user system 540 includes a route 616 from the marker 636 to the dry cleaner represented by the pin 614a. On the other hand, it shows that the device 100 detects the user touch input 643.

  In response to detecting the first input while the first object is displayed in the second user interface as the selected object (728), the first electronic device receives the second user interface A first user on the first display to display corresponding information about the second object according to the first input, while maintaining the display of the first object therein as the selected object Update the interface (732). For example, FIG. 6M shows that device 100 has detected a user touch input 643 on a user interface object (eg, a pin) corresponding to the dry cleaner represented by pin 614b in FIG. 6L. The user interface 602 may be updated to display an information box 644 corresponding to the dry cleaner represented by 614 b. While the user interface 602 is being updated in FIG. 6M, the user interface 604 is maintained on the display 546-1 of FIG. 6M as shown prior to detecting the user touch input 643 in FIG. 6L.

  In some embodiments, a second user on the second display to display corresponding information regarding the second object in the second user interface in response to detecting the first input. The first user interface is updated without providing information to the second display that allows the interface to be updated (e.g. the first electronic device responds in response to the first input) Do not send information to the second electronic device or the second display) (734). For example, FIG. 6M shows information 546-1 (or information that allows device 100 to update user interface 604 on display 546-1 in response to detecting user touch input 643 in FIG. 6L). 14 illustrates updating the user interface 602 on the touch screen 112 without providing to the system 540). While the user interface 602 is being updated in FIG. 6M, the user interface 604 is maintained on the display 546-1 of FIG. 6M as shown prior to detecting the user touch input 643 in FIG. 6L.

  In some embodiments, the first input is to display corresponding information about the second object while maintaining the display of the first object in the second user interface as the selected object. Updating the first user interface on the first display according to the while maintaining the entire second user interface on the second display as it was displayed immediately before detecting the first input, Updating the first user interface on the first display according to the first input to display corresponding information regarding the second object (e.g., the second user interface updates the first Not updated according to) (736). For example, FIG. 6M displays on touch screen 112 such that device 100 displays information box 644 corresponding to the dry cleaner store represented by pin 614 b in response to detecting user touch input 643 in FIG. 6L. To update the user interface 602 of While the user interface 602 is being updated in FIG. 6M, the user interface 604 is maintained on the display 546-1 of FIG. 6M as shown prior to detecting the user touch input 643 in FIG. 6L.

  After updating the first user interface according to the first input while the first object is displayed in the second user interface as the selected object (728), the first electronic device is configured to: A second input from the user is detected 738 corresponding to the request to select the object of s as the selected object. For example, FIG. 6Q shows the request for the device 100 to display a turn-by-turn indication for the route 659a from the current position 608 of the device 100 to the dry cleaner represented by the pin 614b (eg, Mary Ann's Cleaners). To indicate that the user touch input 661 is to be detected.

  In response to detecting the second input (740), the first electronic device displays a second object on the first display such that the second object is displayed as the selected object according to the second input. Update 1 the user interface (742). For example, FIG. 6R shows that the device 100 detects the user touch input 661 in FIG. 6Q, the dry cleaner represented by the pin 614b from the current position 608 of the device 100 (eg, Mary Ann's Cleaners) It is shown that the user interface 602 is updated to display the turn-by-turn instruction box 662.

  In response to detecting the second input (740), the first electronic device causes the second object to be displayed on behalf of the first object as the selected object, on the second display. Information is provided 744 to the second display that enables the second user interface to be updated. In response to detecting user touch input 661 in FIG. 6Q, device 100 sends information to system 540 to enable system 540 to update user interface 604 on display 546-1. For example, FIG. 6R shows a marker 664 indicating the current position of the device 100, a route 659a to a dry cleaner (eg, Mary Ann's Cleaners) represented by the pin 614b, and a first direction of the user on the route 659a The user of display 546-1 is a turn-by-turn instruction mode (eg, from the current position 608 of device 100 to the dry cleaner represented by pin 614b) including a turn-by-turn instruction box 663 indicating (or a turn change point) The updated user interface 604 is shown displayed within the interface 604.

  In some embodiments, information enabling the second user interface on the second display to be updated to display the second object as a selected object instead of the first object After providing (746), the first electronic device enters display sync mode (eg, enters playlist play mode or driving direction mode) (748). For example, FIG. 6R displays in the user interface 602 of the touch screen 112 a turn-by-turn indication box 662 from the current position 608 of the device 100 to the dry cleaner represented by the pin 614b (eg, Display sync mode). In some embodiments, during display synchronization mode (750), the first electronic device periodically updates the first user interface over time (eg, the first electronic device is a playlist) Update the displayed cover art as you play the entire series of songs within, or update the displayed map to show the progress of the vehicle moving along the navigation route) (752) . For example, while the device 100 is displaying the turn-by-turn indication box 662 as shown in FIG. 6R, the device 100 may display a table 614b corresponding to the destination (eg, Mary Ann's Cleaners). As the updated next direction (or course change point) on route 659a, and the updated remaining distance and time are shown in the turn-by-turn instruction box 662, The user interface 602 is periodically updated. In some embodiments, during display synchronization mode (750), the first electronic device is configured to display the second user interface on the second display as the first user interface is updated over time. Provide information that allows the user to synchronize with the first user interface (eg, updated as the first electronic device plays the entire playlist, or as the vehicle travels along the navigation route) Provide cover art or map images) (754). For example, while the device 100 is displaying the turn-by-turn indication box 662 as shown in FIG. 6R, the device 100 may display a table 614b corresponding to the destination (eg, Mary Ann's Cleaners). As the dry cleaner shop is approached, the updated next direction (or turning point) on route 659a, the updated remaining distance and time, and the marker 664 (turn-by-turn instruction box 663) For example, information is provided to system 540 that enables system 540 to display updated user interface 604, including the current location of device 100).

  In some embodiments, prior to detecting the first input, the first user interface is synchronized with the second user interface (e.g., the first object as the selected object). After being responsive to the first input and displayed in both the interface and in the second user interface, and prior to detecting the second input, the first user interface is the second user interface Not synchronized (eg, the second object is displayed in the first user interface as a selected object, and at the same time the second object is displayed in the second user interface as the selected object). After responding to the second input (which remains displayed), the first user interface It is synchronized with The interface (e.g., a second object is displayed in both in the first user interface as the selected object and the second user interface) (756).

  In FIG. 6H, the first user interface 602 and the second user interface 604 are synchronized. For example, FIG. 6H illustrates a user's first direction (or turn around point) on route 631a from current position 608 of device 100 to a dry cleaner (eg, a selected first object) represented by pin 614a. And a user interface 602 on the touch screen 112 displaying a turn-by-turn instruction box 634 indicating the name of the selected first object or destination (e.g. ABC Cleaners). For example, FIG. 6H displays a turn-by-turn instruction mode using route 616 from current position 608 of device 100 to a dry cleaner (eg, a first selected object) represented by pin 614a. Also shown is the user interface 604 on 546-1.

  In FIG. 6M, the first user interface 602 and the second user interface 604 are not synchronized. For example, FIG. 6M shows a user interface 602 on touch screen 112 displaying an information box 644 corresponding to the dry cleaner store (eg, the selected second object) represented by pin 614b. For example, FIG. 6M shows a user interface 604 on display 546-1 displaying a route 616 from the current position 608 of the device 100 to the dry cleaner (eg, the selected first object) represented by the pin 614a. Also show.

  In FIG. 6R, the first user interface 602 and the second user interface 604 are synchronized. For example, FIG. 6R illustrates a user's first direction (or turnaround point) on route 659a from current position 608 of device 100 to a dry cleaner (eg, selected second object) represented by pin 614b. And a user interface 602 on the touch screen 112 displaying a turn-by-turn instruction box 662 indicating the name of the selected second object or destination (e.g., Mary Ann's Cleaners). For example, FIG. 6R displays a turn-by-turn instruction mode using route 659a from current position 608 of device 100 to a dry cleaner (eg, a selected second object) represented by pins 614b. Also shown is the user interface 604 on 546-1.

  Thus, as described above, the touch screen display 112 and the user interface displayed on the display 546-1 are intermittently synchronized. Intermittently synchronizing the touch screen display 112 and the user interface displayed on the display 546-1 may be performed by the user interface displayed on the display (eg, the display 546-1) seen by the driver. The passenger in the vehicle browses the information displayed on the touch screen display 112 without constantly updating according to the operation (for example, examines different destination candidates including contact information, reviews, explanations, etc. Make it possible). However, if the passenger determines the operation (eg, select a destination) within the user interface displayed on the touch screen display 112, the user interface on the display is synchronized, thereby selecting the driver. The user can drive to the destination and assist the user to move to the selected destination. Ru. Similarly, if the driver selects a destination in the user interface displayed on the display (eg, display 546-1) viewed by the driver, the user interface displayed on the touch screen display 112 Are updated to reflect the driver's choice so that the driver can assist the driver in moving to the selected destination. The intermittent synchronization of the user interface described above is an improvement over the situation in which the user interface displayed on the two displays is continuously synchronized (e.g. repeatedly changing the information displayed to the driver) Without allowing the passenger to see additional information). Furthermore, the intermittent synchronization of the user interface described above is also an improvement over the situation where the user interface displayed on the two displays is never synchronized (e.g. the passenger has retrieved an address on the device 100) Later, because there is no need to perform the additional step of updating the destination address in the car navigation system).

  The particular order described for the operations in FIGS. 7A-7D is exemplary only, and the order described is not intended to indicate that it is the only order in which the operations can be performed. I want you to understand. Those skilled in the art will recognize various ways to reorder the operations described herein. In addition, other process details described herein (e.g., methods 800 and 900) with respect to the other methods described herein are also described above in connection with FIGS. 7A-7D. It should be noted that the method 700 can be applied in a similar manner. For example, the contacts, gestures, user interface objects described above in connection with method 700 may be the contacts, gestures, user interface described herein in connection with other methods described herein. Optionally, one or more of the properties of the object (e.g., methods 800 and 900) are included. For the sake of brevity, these details are not repeated here.

  8A-8B are flow diagrams illustrating a method 800 for synchronizing two or more displays in accordance with some embodiments. The method 800 is implemented at a portable electronic device (eg, the device 300 of FIG. 3 or the portable multifunction device 100 of FIG. 1A) in communication with a vehicle display of a vehicle. The portable electronic device comprises a portable device display and a touch sensitive surface. In some embodiments, the portable device display is a touch screen display and the touch sensitive surface is on the portable device. In some embodiments, the portable device is separate from the touch sensitive surface. Some operations in method 800 are optionally combined and / or the order of some operations is optionally changed.

  As described below, the method 800 provides an intuitive way to synchronize two or more displays. This method reduces the user's cognitive burden when using more than one display, thereby creating a more efficient human-machine interface. With respect to battery operated electronic devices, by allowing the user to synchronize two or more displays faster and more efficiently, power is saved and the interval for charging the battery is increased.

  The portable electronic device obtains (802) a set of map search results including a plurality of potential destinations. For example, FIG. 6C illustrates a user search query (eg, dry cleaning) displayed in search box 609 in user interface 602 of touch screen 112. In response to the search query, device 100 obtains a set of map search results including multiple potential destinations by searching memory unique to device 100, memory external to device 100, the Internet, or a combination thereof. Do.

  The portable electronic device displays a portable device navigation interface on the portable device display, and the portable device navigation interface selects a destination from a plurality of potential destinations (e.g., search results corresponding to previously performed search operations) Enable (804). For example, FIG. 6D illustrates a portable device navigation interface (e.g., user interface 602) corresponding to the search query "dry clean" entered in the search query box 609 displayed on the touch screen 112 in FIG. 6C. In FIG. 6D, the user interface 602 includes selectable user interface objects (eg, pins) or affordances corresponding to the dry cleaners represented by pins 614a and 614b near the current position 608 of the device 100.

  The portable electronic device transmits information to the vehicle display enabling the vehicle display to display the vehicle navigation interface, wherein a first destination of the plurality of possible destinations is selected in the vehicle navigation interface It is a destination (806). For example, FIG. 6D shows a vehicle navigation interface (eg, user interface 604) displayed on display 546-1 of system 540. In FIG. 6D, the dry cleaners represented by pins 614a (eg, ABC Cleaners) are displayed in the user interface 604 of the display 546-1 as the selected destination, from the current position 608 of the device 100 to the pins 614a. The route 616 to the dry cleaners represented is also displayed in the user interface 604 of the display 546-1. In some embodiments, display 546-1 is a touch screen display. In some embodiments, display 546-1 is a primary display of a plurality of displays associated with system 540. In some embodiments, display 546-1 is implemented within the dashboard of the vehicle and is visually accessible to the driver and / or passenger of the vehicle.

  While the first destination is being displayed in the vehicle navigation interface as the selected destination (808), the portable electronic device is configured to select one or more of a plurality of possible destination destinations different from the first destination. A first input from a user (e.g., of the first electronic device) corresponding to a request to display information relating to a candidate of the destination (e.g. without selecting the second object as the selected object) A tap or swipe gesture on the touch sensitive surface is detected 810. For example, FIG. 6L illustrates a user interface object (e.g., a pin corresponding to a dry cleaner (e.g., Mary Ann's Cleaners) represented by pin 614b on a map where device 100 is displayed in user interface 602). ) To detect the user touch input 643 above. In FIG. 6L, the maps displayed in the user interface include the dry cleaners represented by pins 614a, 614b, 614c, 614e, and 614f. For example, in FIG. 6L, dry cleaners (eg, ABC Cleaners) represented by pins 614a are displayed in the user interface 604 of the display 546-1 as the selected destination.

  In response to detecting the first input while the first destination is the selected destination in the vehicle navigation interface (808), the portable electronic device updates the vehicle navigation interface on the vehicle display Update 812 the portable device navigation interface on the portable device display according to the first input. For example, FIG. 6M shows that device 100 has detected a user touch input 643 on a user interface object (eg, a pin) corresponding to the dry cleaner represented by pin 614b in FIG. 6L. The user interface 602 may be updated to display an information box 644 corresponding to the dry cleaner represented by 614 b. In FIG. 6M, user interface 602 is updated without providing information to display 546-1 (or system 540) that enables user interface 604 on display 546-1 to be updated. For example, in FIG. 6M, the dry cleaner represented by pin 614a is displayed in the user interface 604 of the display 546-1 as the selected destination.

  After updating the portable device navigation interface according to the first input while the first destination is displayed in the vehicle navigation interface as the selected destination (808), the portable electronic device is configured to have a plurality of destinations A second input from the user is detected 814 corresponding to the request to select the second destination of the candidates as the selected destination. For example, FIG. 6Q shows the request for the device 100 to display a turn-by-turn indication for the route 659a from the current position 608 of the device 100 to the dry cleaner represented by the pin 614b (eg, Mary Ann's Cleaners). To indicate that the user touch input 661 is to be detected.

  In response to detecting the second input (816), the portable electronic device displays the second destination as the selected destination according to the second input, the portable device on the portable device display Update the navigation interface (818). For example, FIG. 6R shows that the device 100 is responsive to the user touch input 661 detected in FIG. 6Q to include a turn-by including instructions to a dry cleaner (eg, Mary Ann's Cleaners) represented by pin 614b. The user interface 602 may be updated to display the turn indication box 662.

  In response to detecting the second input (816), the portable electronic device causes the vehicle navigation interface to display the second destination as the selected destination according to the second input. Information to allow the vehicle display to be updated (820). In response to detecting user touch input 661 in FIG. 6Q, device 100 sends information to system 540 to enable system 540 to update user interface 604 on display 546-1. For example, FIG. 6R shows an updated user interface 604, and a marker 664 indicating the current position of the device 100, a dry cleaner represented by pins 614b from the current position 608 of the device 100 (eg, Mary Ann's Cleaners). A turn-by-turn instruction mode including a turn-by-turn instruction box 663 indicating the user's first direction (or course change point) on the route 659a and the route 659a is displayed in the user interface 604 of the display 546-1. Be done.

  The particular order described for the operations in FIGS. 8A-8B is exemplary only, and the order described is not intended to indicate that it is the only order in which the operations can be performed. I want you to understand. Those skilled in the art will recognize various ways to reorder the operations described herein. In addition, other process details (eg, methods 700 and 900) described herein with respect to the other methods described herein are also described above in connection with FIGS. 8A-8B. It should be noted that the method 800 is applicable in a manner similar to the method 800. For example, the contacts, gestures, user interface objects described above in connection with method 800 may be the contacts, gestures, user interface described herein in connection with other methods described herein. Optionally, one or more of the properties of the object (e.g., methods 700 and 900) are included. For the sake of brevity, these details are not repeated here.

  9A-9C are flow diagrams illustrating a method of sending update information to an affected display in accordance with some embodiments. The method 900 is performed at an electronic device (eg, the device 300 of FIG. 3 or the portable multifunction device 100 of FIG. 1A) in communication with a plurality of displays, including a first display and a second display. In some embodiments, the electronic device comprises a display and a touch sensitive surface. In some embodiments, the display is a touch screen display and the touch sensitive surface is on the electronic device. In some embodiments, the electronic device is separate from the touch sensitive surface. Some operations in method 900 are optionally combined and / or the order of some operations is optionally changed.

  As described below, the method 900 provides an efficient way to send update information to affected displays.

  An electronic device in communication with a plurality of displays, including a first display and a second display, receives first input information indicative of a first input detected by a first input source of the plurality of input sources The first input information includes 902 a first (unique) identifier for the first input source and a first input description describing the first input. For example, the first input source description may be a raw input, such as gesture data describing the location and movement / direction of a touch detected on the touch sensitive surface, and / or the type of gesture performed on the touch sensitive surface. It is data. In some embodiments, the device communicates with a plurality of different input sources (eg, one or more buttons, knobs, jog dials, and / or touch sensitive surfaces), each input source being associated with a particular display To be mapped. In some embodiments, as described in more detail below with reference to FIGS. 6DD to 6EE, the mapping optionally maps the mapping of each input source to the first display Updated to be changed from the state to the state mapped to the second display.

  For example, FIGS. 6W-6FF illustrate device 100 in communication with displays 546-1 and 546-2. For example, displays 546-1 (e.g., primary display) and 546-2 (e.g., auxiliary display) are associated with system 540 implemented in the vehicle. For example, jog dial 675 is associated with system 540 and, when activated, causes the device to control one or both of display 546-1 and display 546-2. For example, FIG. 6X shows that the jog dial 675 detects rotation 681 (eg, approximately 90 degrees clockwise rotation). In response to detecting the rotation 681, the jog dial 675 (or the system 540) transmits input information indicating the rotation 681 to the device 100. The input information includes an identifier associated with the jog dial 675 (eg, a unique identifier associated with the jog dial 675) and input description information that describes the rotation 681. For example, the input description information includes raw data describing the rotation 681 such as the magnitude (90 degrees) and the direction (clockwise) of the rotation.

  The device selects a first affected display from the plurality of displays based on the first identifier and the input source mapping that maps the input source to the corresponding display (904). For example, in FIG. 6X, in response to receiving input information corresponding to rotation 681 from jog dial 675, device 100 is included in the input information based on input source mapping that maps the input source to the display (eg, The affected display is selected (or determined) by associating an identifier identifying the input source with the jog dial 675) with the display (e.g., display 546-1).

  In some embodiments, selecting the first affected display comprises, in the input source mapping, the first display according to the determination that the first input source is assigned to the first display, Selecting as the first affected display affected by the first input and determining that in the input source mapping, the first input source is assigned to a second display different from the first display Selecting the second display as a first affected display, affected by the first input (906). For example, in FIG. 6W, in response to receiving input information corresponding to a pinch gesture from a touch sensitive surface associated with display 546-1, device 100 may be based on input source mapping that maps the input source to the display. Select the affected display by associating with the display (eg, display 546-1) an identifier contained within the input information (eg, identifying the input source to the touch sensitive surface associated with the display 546-1) (Or judge) In another example, in FIG. 6EE, in response to receiving input information corresponding to the upward swipe gesture from the touch sensitive surface associated with display 546-2, device 100 causes the input source to be displayed. By associating an identifier contained within the input information (e.g. identifying the input source with the touch sensitive surface associated with display 546-2) based on the input source mapping to map with the display (e.g. display 546-2) , Select (or determine) the affected display.

  In some embodiments, after selecting the first affected display, the device displays a user interface displayed on the first affected display (eg, immediately before the first input information is received). , The respective state of the user interface displayed on the first affected display), and based on the respective states of the first input description and the first user interface displayed on the first display. , Generate update information (908). For example, in FIG. 6X, in response to receiving input information corresponding to rotation 681 from jog dial 675, device 100 is included in the input information based on input source mapping that maps the input source to the display (eg, The affected display is selected (or determined) by associating an identifier identifying the input source with the jog dial 675) with the display (e.g., display 546-1). In this example, after determining that the rotational event detected in FIG. 6X is associated with display 546-1, device 100 may display the display state of display 546-1 (eg, user interface 604 shown in FIG. 6X). It also determines the search result or the destination selection mode). In this example, the device 100 may have the input description information (eg, approximately 90 degrees clockwise rotation) shown in FIG. (E.g., an updated user interface displayed on display 546-1 in FIG. 6Y).

  In some embodiments, the update information is displayed on the first affected display in accordance with the determination that the state of each of the user interfaces displayed on the first affected display is the first state. Corresponding to performing the first operation for each state of the user interface (eg, each state is to display an "on playback" interface for the audio being (E.g., adjusting the volume level of audio playback associated with the user interface displayed on the first affected display according to the amount of rotation of the knob), on the first affected display That each state of the user interface displayed in the second state is a second state different from the first state. According to the determination, the update information corresponds to performing, for each state of the user interface displayed on the first affected display, a second operation different from the first operation (e.g. The state of is to display a list of selectable items, such as an audio content item, and the corresponding action is to rotate the knob within the list of selectable items on the first affected display. To navigate according to) (910). For example, FIGS. 6Y-6Z show the user interface 604 in FIG. 6Y in response to the display 546-1 detecting rotation 683 of the jog dial 675 (eg, about 180 degrees of counterclockwise rotation) in FIG. 6Y. To scroll from the south to the north between selected search results (e.g., dry cleaning shop) in the user interface 604 according to the display state (e.g., the search results or the destination selection mode) of FIG. For example, FIGS. 6AA-6BB illustrate that user interface 604 in FIG. 6AA may be responsive to display 546-1 detecting rotation 686 of jog dial 675 (eg, about 180 degrees of counterclockwise rotation) in FIG. 6AA. To indicate that the turn-by-turn instruction mode is to be zoomed out in the user interface 604 according to the display state (eg, the turn-by-turn instruction mode).

  In some embodiments, prior to receiving the first input information, each user interface has a previous state, and the device is configured to receive the user based on the first input and the previous state of each user interface. Update the interface to an updated state different from the previous state (eg, in response to a swipe gesture or knob rotation on the touch sensitive surface, the map user interface may rotate the swipe gesture's size and direction, or the knob) Is scrolled according to the size of (912). For example, FIGS. 6EE-6FF illustrate that, in response to display 546-2 detecting an upward swipe gesture by user touch input 689 on display 546-2 in FIG. 6EE, user interface 676 in FIG. A scrolling list of songs 677 in the user interface 676 is shown to be scrolled up according to the display state (eg, album song list mode).

  In some embodiments, prior to sending the update information to the first affected display, the device generates the update information based on the first input information, and the first input description generates the first input characteristic. According to the determination to describe, the device generates the first update information, and according to the determination that the first input description describes a second input characteristic different from the first input characteristic, the device is updated with the first update information. Generate different second update information (914). For example, while the list is being displayed on the first affected display, when the first input information corresponds to the rotation of the knob, the first input description is the first input being a clockwise rotation of the knob The device scrolls the list down on the affected display if indicated to include a rotation of. In contrast, if the first input description indicates that the first input includes counterclockwise rotation of the knob, the device scrolls the list upward on the affected display.

  In another example, when a search result or destination selection mode is displayed, a 90 degree rotation of the jog dial 675 selects the next search result, and a clockwise rotation scrolls the search result from north to south And counterclockwise rotation scrolls the search results from south to north. For example, FIGS. 6X-6Y show that the display 546-1 has selected search results (eg, approximately 90 degrees clockwise rotation) detected in FIG. 6X (eg, approximately 90 degrees clockwise rotation). Indicates to scroll from north to south between the dry cleaners). In contrast, for example, FIGS. 6Y-6Z are selected in response to display 546-1 detecting rotation 683 of jog dial 675 (eg, approximately 180 degrees of counterclockwise rotation) in FIG. 6Y. It shows scrolling from south to north between search results (for example, dry cleaners).

  The device sends update information to the first affected display that enables the first affected display to be updated according to the first input (916). For example, in FIG. 6X, in response to receiving, from the jog dial 675, input information corresponding to the rotation 681, the device 100 is included in the input information based on the input source mapping (e.g. And select the display 546-1 as the affected display by associating the identifier with the display 546-1. After selecting a display as the affected display, the device 100 sends updated information to the affected display 546-1 that allows the affected display 546-1 to be updated.

  In some embodiments, the update information is performed on the information corresponding to the first input (eg, the position and movement amount / direction of the touch detected on the touch sensitive surface, and / or the touch sensitive surface) Raw input data such as gesture data that describes the type of gesture) (918). In some embodiments, the update information includes a first input description. In some embodiments, the update information is generated based on the first input description (eg, the first input description is raw touch input data and the update information includes a gesture corresponding to the raw touch input data) .

  In some embodiments, the updated information is an updated user interface for display on the first affected display (eg, an image for display on the first display including a plurality of user interface objects) ) (920). In FIG. 6X, after selecting display 546-1 as the affected display corresponding to rotation 681, device 100 follows the magnitude (90 degrees) and direction (clockwise) of rotation 681 of jog dial 675 in FIG. 6X. Update information that allows the affected display 546-1 to be updated is sent to the affected display 546-1. For example, the update information may be the next dry cleaner store south of ABC Cleaners corresponding to the dry cleaner store represented by pin 614a (e.g. Suzie's Cleaners corresponding to the dry cleaner store represented by pin 614d) 6Y includes an updated user interface that scrolls search results (eg, a dry cleaning store) to be displayed in the user interface 604 as selected search results.

  In some embodiments, the device generates user interfaces to two or more of the plurality of displays and provides each user interface to a corresponding display (922). For example, device 100 may be displayed on a first display (eg, main display 546-1 or dashboard navigation display) of a vehicle information display system (eg, system 540), such as a vehicle entertainment or navigation system A second, different, for a second different display (eg, touch screen 112 of device 100 or another portable electronic device) that transmits an image representing one map / navigation user interface and is structurally integrated with the device Generate a map / navigation user interface. In some embodiments, the device is different for displaying on a third display (eg, auxiliary display 546-2 or backseat entertainment display) in a vehicle of a vehicle information display system (eg, system 540) Send an image representing the user interface.

  In some embodiments, after sending (924) the update information that enables the first affected display to be updated according to the first input, the device is different than the first input source, Receiving second input information indicating a second input detected by the second one of the input sources, the second input information being a second (unique) identifier for the second input source And a second input description describing a second input (926). For example, the second input description may be raw input data, such as gesture data that describes the location and movement / direction of the touch detected on the touch sensitive surface, and / or the type of gesture performed on the touch sensitive surface. It is. In FIG. 6X, after selecting display 546-1 as the first affected display corresponding to rotation 681, device 100 updates the first affected display 546-1 as shown in FIG. 6Y. Update information that allows to be sent to the first affected display 546-1. For example, Figure 6EE continues to show that the touch sensitive surface associated with display 546-2 detects an upward swipe gesture due to user touch input 689. In response to detecting the upward swipe gesture, the touch-sensitive surface associated with display 546-2 (or system 540) displays input information indicative of user touch input 689 (or upward swipe gesture) to device 100. Send to The input information may be an identifier associated with the touch sensitive surface associated with display 546-2 (eg, a unique identifier associated with the touch sensitive surface of display 546-2) and user touch input 689 (or an upward swipe gesture). Contains input description information describing.

  In some embodiments, after sending (924) the update information that enables the first affected display to be updated according to the first input, the device is based on the second identifier and the input source mapping. , Select a second affected display from the plurality of displays (928). In FIG. 6EE, in response to receiving input information corresponding to the upward swipe gesture from the touch sensitive surface associated with display 546-2, device 100 maps input source to input source mapping to the display. By associating an identifier contained within the input information (eg, identifying the input source with the touch sensitive surface associated with display 546-2) with the display (eg, display 546-2). Select as the second affected display.

  In some embodiments, after sending 924 the update information that enables the first affected display to be updated according to the first input (924), the device performs the second influence according to the second input. Update information that allows the receiving display to be updated is sent 930 to the second affected display. In some embodiments (eg, where both the first input source and the second input source are mapped to the same display), the first affected display is the same as the second affected display is there. In some embodiments (e.g., where the first input source and the second input source are mapped to different displays), the first affected display is different than the second affected display. After selecting the display 546-2 as the second affected display, the device 100 can update the second affected display 546-2 with information that enables the affected display 546-2 to be updated. Send to For example, in FIG. 6FF, the update information includes an updated user interface that allows the display 546-2 to display the upwardly scrolled list 677 in the user interface 676.

  In some embodiments, the device detects a change in availability of each of the plurality of displays, and in response to detecting a change in availability of each of the devices, the device determines the availability of each of the displays Update the input source mapping to indicate a change in (932). In some embodiments, changing the availability of the respective displays includes the computer system different from the electronic device (eg, a vehicle information display system such as a navigation or entertainment system) controlling the respective displays. In some embodiments, changing the availability of the respective display includes passing (or returning) control of the respective display to the electronic device different from the electronic device (e.g., the vehicle is moving backward) The vehicle information display system displays the video feed from the reverse camera on the display of the vehicle information display system, and the vehicle information display system provides the electronic device by the electronic device after the vehicle stops the reverse movement. Allow the user interface to be displayed again on the display of the vehicle information display system).

  For example, FIGS. 6W-6FF illustrate displays 546-1 and 546-2 associated with a system 540 implemented in a vehicle. For example, FIG. 6 CC shows that the display 546-1 displays the vehicle reverse mode in the user interface 604 while the vehicle reverses, and the display 546-2 is video feed from the reverse camera. Displaying in the user interface 676 a vehicle reverse mode including: While the vehicle reverses, the vehicle reverse mode replaces any updated information received from the device 100 by the displays 546-1 and 546-2.

  In some embodiments, while each input source is assigned to the first display, the device is configured to transmit from the first display to the second display for each input source of the plurality of input sources. In response to detecting a change in display assignment and detecting a change in display assignment for each input source, the device displays on the first display input information indicative of the input associated with each input source. To indicate a change in display assignment for each input source, as used to update the user interface displayed on the second display instead of being used to update the selected user interface , Update input source mapping (934). For example, from the situation where the passenger in the vehicle uses the first knob to control the user interface in the main display of the vehicle information display system, the auxiliary display of the vehicle information display system (e.g. Switching to the state of using the first knob to control the user interface in) and the input information including the identifier for the first knob is used to update the user interface displayed on the main display Instead, it is used to update the user interface displayed on the auxiliary display.

  6W-6CC, the jog dial 675, when activated, causes the device to control the display 546-1. Thus, in FIGS. 6W-6CC, the jog dial 675 is associated with the display 546-1 in mapping the input source to the display. For example, FIGS. 6AA-6BB illustrate that, in response to display 546-1 detecting rotation 686 of jog dial 675 (eg, approximately 90 degrees of counterclockwise rotation) in FIG. 15 illustrates zooming out of the turn-by-turn instruction mode in the user interface 604 according to the determination that it is mapped to one.

  In contrast, in FIGS. 6DD-6EE, the jog dial 675, when activated, causes the device to control the display 546-2. Thus, in FIGS. 6DD-6EE, the jog dial 675 is associated with the display 546-2 in a modified mapping of the input source to the display. In another embodiment, FIGS. 6DD-6EE are responsive to the display 546-2 detecting rotation 688 (eg, approximately 90 degrees clockwise rotation) of the jog dial 675 in FIG. 6DD. Shown is a scrolling down of the list 677 within the user interface 676 in accordance with the determination that it is mapped to the display 546-2.

  It is understood that the specific order described for the operations in FIGS. 9A-9C is merely exemplary and that the described order is not intended to indicate that it is the only order in which the operations can be performed. I want to be Those skilled in the art will recognize various ways to reorder the operations described herein. In addition, other process details described herein (eg, methods 700 and 800) with respect to the other methods described herein are also described above in connection with FIGS. 9A-9C. It should be noted that it is applicable in a manner similar to the existing method 900. For example, the contacts, gestures, user interface objects described above in connection with method 900 may be the contacts, gestures, user interface described herein in connection with other methods described herein. Optionally, one or more of the properties of the object (e.g., methods 700 and 800). For the sake of brevity, these details are not repeated here.

  According to some embodiments, FIG. 10 shows a functional block diagram of a first electronic device 1000 configured in accordance with the principles of the various described embodiments. The functional blocks of this device are optionally implemented by hardware, software or a combination of hardware and software to carry out the principles of the various described embodiments. Those skilled in the art will appreciate that the functional blocks described in FIG. 10 are optionally combined or separated into sub-blocks to implement the principles of the various described embodiments. Thus, the description herein optionally supports any possible combination or division or further definition of the functional blocks described herein.

  As shown in FIG. 10, the first electronic device 1000 includes a first display unit 1002 configured to display a first user interface, the first user interface comprising a plurality of selectable objects A first user interface corresponding to a second user interface displayed on a second display unit different from the first display unit, the first of the plurality of selectable objects Objects are displayed in the second user interface as selected objects. Optionally, the first electronic device 1000 is also a communication interface unit 1004 configured to communicate with the second electronic device, and a first touch configured to receive one or more touch inputs. And a sensing surface unit 1006. The first electronic device 1000 further includes a processing unit 1008 coupled to the first display unit 1002, and optionally coupled to the communication interface unit 1004 and the first touch sensitive surface unit 1006. In some embodiments, processing unit 1008 includes a detection unit 1010, an update unit 1012, a provision unit 1014, a synchronization unit 1016, and a reception unit 1018.

  While the first object is displayed in the second user interface as the selected object, the processing unit 1008 may be configured to generate information in the first user interface of information about the second one of the plurality of selectable objects. Detecting a first input from the user (e.g., using detection unit 1010) corresponding to the request to enable display on the second, and in response to detecting the first input, the second A first display unit according to a first input to enable display of corresponding information about a second object while maintaining display of the first object as a selected object in the user interface of Update the first user interface on 1002 (eg, using update unit 1012), After updating the first user interface according to the one input, the second input from the user corresponding to the request for selecting the second object as the selected object (for example using the detection unit 1010 Is configured to detect. The processing unit 1008 is responsive to detecting the second input to allow display of the second object as the selected object according to the second input, on the first display unit 1002 The second display unit to update the first user interface (e.g. using the update unit 1012) and allow the display of the second object as a selected object instead of the first object. It is further configured to provide information on the second display unit (e.g. using the providing unit 1014) that enables updating the upper second user interface.

  In some embodiments, on the second display unit, to enable display of corresponding information regarding the second object in the second user interface in response to detecting the first input. The first user interface is updated without providing the second display unit with information enabling updating the second user interface of the second.

  In some embodiments, to allow the display of corresponding information regarding the second object while maintaining the display of the first object as a selected object in the second user interface, Updating the first user interface on the first display unit 1002 according to an input of 1 causes the entire second user interface to be displayed as it was displayed just before detecting the first input. Updating the first user interface on the first display unit 1002 in accordance with the first input to allow display of corresponding information regarding the second object while maintaining the above.

  In some embodiments, the plurality of selectable objects correspond to the plurality of map locations, and the selected objects correspond to the currently selected destination.

  In some embodiments, the plurality of selectable objects correspond to the plurality of navigation routes, and the selected object corresponds to the currently selected navigation route.

  In some embodiments, the plurality of selectable objects correspond to the plurality of content items that may be played using the first electronic device 1000, and the selected objects correspond to the currently selected content item .

  In some embodiments, the plurality of selectable objects correspond to the plurality of applications, and the selected objects correspond to the currently selected application.

  In some embodiments, the plurality of selectable objects correspond to the plurality of search results generated in response to the search query, and the selected object corresponds to the currently selected search result.

  In some embodiments, the first object is automatically selected as the selected object based on predetermined object selection criteria.

  In some embodiments, before detecting the first input, the first user interface is synchronized with the second user interface, and responsive to the first input, and detects the second input. Before doing so, the first user interface is not synchronized with the second user interface, and after responding to the second input, the first user interface is synchronized with the second user interface.

  In some embodiments, the processing unit 1008 may cause the second user on the second display unit to enable displaying the second object as the selected object instead of the first object. After providing information enabling the interface to be updated, the display synchronization mode is entered (e.g. using synchronization unit 1016) and the first user interface is displayed over time during the display synchronization mode (e.g. Periodically update using the update unit 1012, and as the first user interface is updated with the passage of time, the second user interface on the second display unit is updated with the first user interface Information that makes it possible to synchronize (eg 014 Use) configured to provide.

  In some embodiments, the second electronic device controls an in-vehicle entertainment system having a native graphical user interface, and the second user interface is at least a portion of the native graphical user interface of the second electronic device. Instead, it is displayed on the second display unit.

  In some embodiments, processing unit 1008 receives information corresponding to each input from the control of the in-vehicle entertainment system (e.g., using receiving unit 1018) and receives information corresponding to each input In response, the first user interface and the second user interface are configured to update (eg, using the update unit 1012) according to the respective inputs.

  According to some embodiments, FIG. 11 shows a functional block diagram of a portable electronic device 1100 configured in accordance with the principles of the various described embodiments. The functional blocks of this device are optionally implemented by hardware, software or a combination of hardware and software to carry out the principles of the various described embodiments. Those skilled in the art will appreciate that the functional blocks described in FIG. 11 are optionally combined or separated into sub-blocks to implement the principles of the various described embodiments. Thus, the description herein optionally supports any possible combination or division or further definition of the functional blocks described herein.

  As shown in FIG. 11, the portable electronic device 1100 is configured to communicate with a portable device display unit 1102 configured to display a portable device navigation interface and a vehicle display unit of a vehicle. And optionally, a touch sensitive surface unit 1106 configured to receive one or more touch inputs, a portable device display unit 1102, a communication interface unit 1104, and optionally, a touch sensitive surface unit 1106. And a combined processing unit 1108. In some embodiments, processing unit 1108 includes an acquisition unit 1110, a display enablement unit 1112, a transmission unit 1114, a detection unit 1116, and an update unit 1118.

  The processing unit 1108 is a portable device navigation that obtains a set of map search results including multiple destination candidates (e.g., using the acquisition unit 1110) and allows selection of destinations from multiple destination candidates. A vehicle that allows information to be displayed on the portable device display unit 1102 (eg, using a display enable unit 1112) and allows the vehicle display unit to display a vehicle navigation interface The first destination of the plurality of potential destinations is configured to be transmitted to the display unit (e.g., using the transmitting unit 1114) and to be the selected destination in the vehicle navigation interface. The processing unit 1108 relates to one or more destination candidates out of a plurality of destination candidates different from the first destination while the first destination is a selected destination in the vehicle navigation interface. In response to detecting a first input from the user (e.g., using detection unit 1116) corresponding to the request to enable display of the information and detecting the first input, the vehicle display Update the portable device navigation interface on the portable device display unit 1102 according to the first input (for example using the update unit 1118) without updating the vehicle navigation interface on the unit, and according to the first input the portable device After updating the navigation interface, multiple destinations Are also configured to detect a second input from the user (eg, using detection unit 1116), corresponding to a request to select a second destination of the selected destinations as the selected destination. Ru.

  According to some embodiments, FIG. 12 shows a functional block diagram of a portable electronic device 1200 configured in accordance with the principles of the various described embodiments. The functional blocks of this device are optionally implemented by hardware, software or a combination of hardware and software to carry out the principles of the various described embodiments. Those skilled in the art will appreciate that the functional blocks described in FIG. 12 are optionally combined or separated into sub-blocks to implement the principles of the various described embodiments. Thus, the description herein optionally supports any possible combination or division or further definition of the functional blocks described herein.

  As shown in FIG. 12, the electronic device 1200 includes a communication interface unit 1202 configured to communicate with a plurality of displays, including a first display and a second display. In some embodiments, the electronic device 1200 optionally includes a display unit 1204 configured to display a user interface and a touch sensitive surface unit 1206 configured to receive one or more touch inputs. And. Electronic device 1200 further includes a processing unit 1208 coupled to communication interface unit 1202 and optionally coupled to display unit 1204 and touch sensitive surface unit 1206. In some embodiments, the processing unit 1208 includes a receiving unit 1210, a selecting unit 1212, a transmitting unit 1214, a generating unit 1216, a determining unit 1218, an updating unit 1220, a detecting unit 1222, a map updating unit And 1224.

  The processing unit 1208 is configured to receive (eg, using the receiving unit 1210) first input information indicative of a first input detected by a first input source of the plurality of input sources. The first input information includes a first identifier for the first input source and a first input description describing the first input. The processing unit 1208 may also use a first affected display from the plurality of displays (eg, using the selection unit 1212) based on the first identifier and the input source mapping that maps the input source to the corresponding display. Configuration) to select. Processing unit 1208 sends update information to the first affected display (eg, using transmitting unit 1214) that enables the first affected display to be updated according to the first input. And so on.

  In some embodiments, selecting the first affected display comprises, in the input source mapping, the first display according to the determination that the first input source is assigned to the first display, Selecting as a first affected display (e.g. using selection unit 1212) affected by the first input, and in input source mapping, the first input source is different from the first display Selecting the second display as a first affected display (e.g. using selection unit 1212) according to the determination that it is assigned to a second display; And.

  In some embodiments, the update information includes information corresponding to the first input.

  In some embodiments, the update information includes an updated user interface for display on the first affected display.

  In some embodiments, the device generates a user interface to two or more of the plurality of displays and provides each user interface to a corresponding display.

  In some embodiments, processing unit 1208 generates update information (eg, using generation unit 1216) based on the first input information prior to transmitting the update information to the first affected display. Configured to generate the first update information (eg, using generation unit 1216) according to the determination that the first input description describes the first input characteristic. Configured and the processing unit 1208 generates second update information different from the first update information according to the determination that the first input description describes a second input characteristic different from the first input characteristic (e.g. Are configured to generate (using unit 1216).

  In some embodiments, after selecting the first affected display, the processing unit 1208 determines the status of each of the user interfaces displayed on the first affected display (e.g. Using the determination and generating update information (eg, using generation unit 1216) based on the first input description and the respective state of the user interface displayed on the first affected display Configured as.

  In some embodiments, the update information is displayed on the first affected display in accordance with the determination that the state of each of the user interfaces displayed on the first affected display is the first state. And second state of each of the user interface displayed on the first affected display being different from the first state, corresponding to performing a first operation for each state of the selected user interface. According to the determination that it is a state, the update information corresponds to performing a second operation different from the first operation for each state of the user interface displayed on the first affected display.

  In some embodiments, prior to receiving the first input information, each user interface has a previous state, and processing unit 1208 is based on the first input and the previous state of each user interface. , Configured to update the user interface to an updated state different from the previous state (eg, using the update unit 1220).

  In some embodiments, processing unit 1208 detects a change in availability of each of the plurality of displays (eg, using detection unit 1220) and detects a change in availability of each of the displays In response, the input source mapping is updated (eg, using map update unit 1224) to indicate changes in the availability of the respective displays.

  In some embodiments, the processing unit 1208 may, while the respective input source is assigned to the first display, from the first display to the second display for the respective input source of the plurality of input sources. It is configured to detect a change in display assignment to (eg, using detection unit 1222). Processing unit 1208 may also include input source mapping (e.g., map update unit 1224 to indicate a change in display assignment for each input source in response to detecting a change in display assignment for each input source. ) Configured to update, whereby instead of the input information indicating the input associated with each input source being used to update the user interface displayed on the first display , Used to update the user interface displayed on the second display.

  In some embodiments, the processing unit 1208 transmits a plurality of inputs different from the first input source after sending the update information enabling the first affected display to be updated according to the first input. Configured to receive second input information (e.g., using receiving unit 1210) indicating a second input detected by a second input source of the sources, the second input information being , A second identifier for the second input source, and a second input description describing the second input. The processing unit 1208 also selects a second affected display from the plurality of displays (e.g., using the selection unit 1212) based on the second identifier and the input source mapping, the second affected according to the second input. The update information is configured to be sent to the second affected display (eg, using the transmission unit 1214) that enables the affected display to be updated.

  Operation with the above-described information processing method may be performed by one or more functional modules in an information processing device, such as a general purpose processor (e.g., as described above in connection with FIGS. 1A and 3) or an application specific chip. Is optionally implemented.

  The operations described above with reference to FIGS. 7A-7D, 8A-8B, and 9A-9C are optionally performed by the components shown in FIGS. 1A-1B or 10-12. To be implemented. For example, display operation 702, reception operation 722, detection operation 730 and 738, update operation 742, acquisition operation 802, transmission operation 806 and 820, detection operation 810 and 814, update operation 812 and 818, reception operation 902 and 928, specific operation 904 and 930, send operations 918 and 932, and / or detection operations 934 and 936 are optionally implemented by event sorter 170, event recognizer 180, and event handler 190. An event monitor 171 in the event sorter 170 detects a touch on the touch sensitive display 112 and the event dispatcher module 174 delivers the event information to the application 136-1. The corresponding event recognizer 180 of the application 136-1 queries its event information with the corresponding event definition 186, and the first contact (or device rotation) at the first location on the touch sensitive surface is: It is determined whether it corresponds to a predetermined event or sub-event, such as selection of an object on the user interface or rotation of the device from one orientation to another. When a corresponding predefined event or sub-event is detected, the event recognizer 180 activates an event handler 190 associated with the detection of the event or sub-event. Event handler 190 optionally uses or invokes data updater 176 or object updater 177 to update application internal state 192. In some embodiments, event handler 190 accesses corresponding GUI updater 178 to update what is displayed by the application. Similarly, it will be apparent to those skilled in the art how other processes are implemented based on the components shown in FIGS. 1A-1B.

  The foregoing description, for purposes of explanation, has been described with reference to specific embodiments. However, the illustrative discussions above are not intended to be exhaustive or to limit the invention to the precise forms disclosed. Many modifications and variations are possible in view of the above teachings. The principles of the present invention and its practical applications are best described and the present invention and various embodiments are best described herein by various modifications suitable for the particular application envisaged by the skilled artisan and others. These embodiments have been selected and described to enable their use in form.

Claims (12)

A portable electronic device comprising a portable device display in communication with a vehicle display of a vehicle
Obtaining a set of map search results including multiple destination candidates;
Displaying a portable device navigation interface capable of selecting a destination from the plurality of candidate destinations on the portable device display;
Selecting a first destination from the plurality of destination candidates;
A vehicle navigation interface is displayed on the vehicle display, wherein the first destination of the plurality of destination candidates is the vehicle navigation interface which is a selected destination in the vehicle navigation interface Sending information to the vehicle display that enables the
While the first destination is the selected destination in the vehicle navigation interface,
Detecting a first input from a user corresponding to a request to display information about one or more destination candidates other than the first destination among the plurality of destination candidates;
The portable device navigation on the portable device display according to the first input, without the vehicle navigation interface on the vehicle display being updated in response to detecting the first input Updating the interface,
After updating the portable device navigation interface according to the first input, the user responds to a request for selecting a second destination of the plurality of destination candidates as the selected destination. Detecting the second input of the
In response to detecting the second input,
Updating the portable device navigation interface on the portable device display to display the second destination as the selected destination according to the second input;
Transmitting, to the vehicle display, information enabling the vehicle display to update the vehicle navigation interface according to the second input to display the second destination as the selected destination and that,
Transmitting the information enabling the vehicle navigation interface on the vehicle display to be updated to display the second destination as the selected destination instead of the first destination After
Entering display sync mode and
During the display sync mode,
Updating the mobile device navigation interface periodically over time;
Providing information enabling the vehicle navigation interface on the vehicle display to be synchronized with the mobile device navigation interface as the mobile device navigation interface is updated over time. ,
Method, characterized in that it comprises.   In response to the detection of the first input, the vehicle navigation interface on the vehicle display is information on one or more destination candidates other than the first destination among the plurality of destination candidates. The method according to claim 1, characterized in that the portable device navigation interface is updated without providing the vehicle display with information enabling it to be updated to display. Updating the portable device navigation interface on the portable device display according to the first input without the vehicle navigation interface on the vehicle display being updated;
Displaying information about the second destination according to the first input while maintaining the entire vehicle navigation interface on the vehicle display as displayed immediately before detecting the first input The method according to claim 1 or 2, comprising updating the portable device navigation interface on the portable device display.   In selecting the first destination from the plurality of destination candidates, the first destination is automatically selected based on a predetermined destination selection criterion. The method according to any one of Items 1 to 3. Before detecting the first input, the portable device navigation interface is synchronized with the vehicle navigation interface,
After responding to the first input and before detecting the second input, the portable device navigation interface is not synchronized with the vehicle navigation interface,
5. A method according to any one of the preceding claims, characterized in that the portable device navigation interface is synchronized with the vehicle navigation interface after responding to the second input. The vehicle display is characterized in that the a portable electronic device different from the second display coupled to the electronic device, the method according to any one of claims 1 to 5. The second electronic device controls an in-vehicle entertainment system having a native graphical user interface;
The method of claim 6 , wherein the vehicle navigation interface is displayed on the vehicle display in place of at least a portion of the native graphical user interface of the second electronic device. The method according to claim 6 , characterized in that both the mobile device navigation interface and the vehicle navigation interface are generated by the same application running on the mobile electronic device. A portable electronic device,
Mobile device display,
With one or more processors,
With memory
And one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, the one or more programs are:
Get a set of map search results including multiple destination candidates,
A portable device navigation interface capable of selecting a destination from the plurality of destination candidates is displayed on the portable device display,
Select a first destination from the plurality of destination candidates;
In a vehicle display of a vehicle with which the portable electronic device is in communication, the vehicle display is a vehicle navigation interface, and the first destination among the plurality of destination candidates is within the vehicle navigation interface Transmitting information enabling to display the vehicle navigation interface which is the selected destination;
While the first destination is the selected destination in the vehicle navigation interface,
Detecting a first input from a user corresponding to a request to display information about one or more destination candidates other than the first destination among the plurality of destination candidates;
The portable device navigation on the portable device display according to the first input, without the vehicle navigation interface on the vehicle display being updated in response to detecting the first input・ Update the interface,
After updating the portable device navigation interface according to the first input, the user responds to a request for selecting a second destination of the plurality of destination candidates as the selected destination. Detects the second input of
In response to detecting the second input,
Updating the portable device navigation interface on the portable device display to display the second destination as the selected destination according to the second input;
The second destination to display as the selected destination, and transmits the information to the vehicle display makes it possible to update the vehicle navigation interface in accordance with said second input to the vehicle display ,
Transmitting the information enabling the vehicle navigation interface on the vehicle display to be updated to display the second destination as the selected destination instead of the first destination After
Enter display sync mode,
During the display sync mode,
Periodically updating the portable device navigation interface over time;
Providing information enabling the vehicle navigation interface on the vehicle display to be synchronized with the mobile device navigation interface as the mobile device navigation interface is updated over time <br A portable electronic device, characterized in that it comprises instructions for />. 10. A portable electronic device according to claim 9, wherein the one or more programs comprise instructions for performing the method according to any one of claims 2 to 8 . A computer program, said computer program comprising instructions, said instructions comprising a portable device display, said instruction being executed by a portable electronic device in communication with a vehicle display of a vehicle, said portable electronic device ,
Get a set of map search results including multiple destination candidates,
A portable device navigation interface capable of selecting a destination from the plurality of destination candidates is displayed on the portable device display,
Select a first destination from the plurality of destination candidates,
A vehicle navigation interface is displayed on the vehicle display, wherein the first destination of the plurality of destination candidates is the vehicle navigation interface which is a selected destination in the vehicle navigation interface Sending information to the vehicle display to enable
While the first destination is the selected destination in the vehicle navigation interface,
Detecting a first input from the user corresponding to a request for displaying information on one or more destination candidates other than the first destination among the plurality of destination candidates,
The portable device navigation on the portable device display according to the first input, without the vehicle navigation interface on the vehicle display being updated in response to detecting the first input・ Update the interface,
After updating the portable device navigation interface according to the first input, the user responds to a request for selecting a second destination of the plurality of destination candidates as the selected destination. Detect the second input of
In response to detecting the second input,
Updating the portable device navigation interface on the portable device display to display the second destination as the selected destination according to the second input;
Transmitting to the vehicle display information enabling the vehicle display to update the vehicle navigation interface according to the second input to display the second destination as the selected destination ,
Transmitting the information enabling the vehicle navigation interface on the vehicle display to be updated to display the second destination as the selected destination instead of the first destination After
Enter display sync mode,
During the display sync mode,
Updating the portable device navigation interface periodically over time,
Provide information enabling the vehicle navigation interface on the vehicle display to be synchronized with the mobile device navigation interface as the mobile device navigation interface is updated over time <br A computer program characterized by A computer program according to claim 11 , characterized in that the one or more programs comprise instructions for performing the method according to any one of claims 2 to 8 .

Images