JP5771243B2 - Portable touch screen device, method and graphic user interface for using emoji characters - Google Patents

Description

  The present invention relates generally to portable electronic devices that generate pictogram characters for electronic messages and documents, and more particularly, portable electronic devices with touch screen displays that generate pictogram characters for electronic messages and documents. Concerning.

  As portable electronic devices become increasingly compact and the number of functions performed by a given device increases, it becomes a significant challenge to design a user interface that allows users to easily interact with multifunction devices. I came. This challenge is particularly significant for handheld portable devices with touch sensitive displays whose screen is much smaller than a desktop or laptop computer. This situation is inconvenient. This is because the user interface is a gateway through which the user not only receives content but also receives responses to user actions or behaviors, including user attempts to access device features, tools and functions. Certain portable communication devices (eg, mobile phones, sometimes referred to as mobile phones, cell phones, cellular phones, etc.) are more able to allow users to access, store and manipulate data. Rely on adding many push buttons, increasing push button density, overloading push button functionality, or using complex menu systems. These conventional user interfaces often require the user to learn complex key sequences and menu hierarchy keys.

  Also, many conventional user interfaces, such as those that include physical push buttons, are not flexible. This may prevent the user interface from being configured and / or adapted by an application or user running on the portable device. This inflexibility is irritating to most users when coupled with the time-consuming request to learn a large number of key sequences and menu hierarchy keys and the difficulty of activating the desired push button.

  Recently, portable electronic devices that support the use of pictogram characters have been developed. Emoji characters are often used in web forums, email messages, instant messages (eg, SMS text messages), notes, and online games. There are a large and increasing number of emoji characters. However, displaying, organizing, and using multiple pictogram characters in existing portable electronic devices with a small display screen is quite cumbersome.

  Accordingly, there is a need for a portable electronic device with a touch screen display that has a more transparent and efficient user interface for using pictogram characters. Such an interface allows tasks involving pictogram characters to be performed more quickly and efficiently by the user, thereby conserving power and extending the time between battery charging of portable electronic devices. Furthermore, such an interface enhances the effectiveness of portable electronic devices and user satisfaction with such devices.

The disadvantages and other problems associated with the user interface for generating and using pictograms in portable devices are alleviated by the portable multifunction device disclosed herein. In some embodiments, the apparatus performs a touch-sensitive display (also known as a “touch screen”) with a graphic user interface (GUI), one or more processors, memory, and multiple functions. And one or more modules, programs or instruction sets stored in the memory. In some embodiments, the user interacts with the GUI primarily through finger touches and gestures on the touch-sensitive display. In some embodiments, the functions include telephone calls, video conferencing, email, instant messaging, blogs, digital photos, digital video, web browsing, digital music playback, and / or digital video playback. Instructions for performing these functions are included in a computer program product configured to be executed by one or more processors (eg, a computer readable storage medium having instructions stored thereon).

  According to certain embodiments, the computer-implemented method is performed in a portable electronic device with a touch screen display. The computer-implemented method includes simultaneously displaying a character input area operable to display text character input and pictogram character input selected by a user, a keyboard display area, and a plurality of pictogram category icons. A “text character” as used herein and in the claims refers to alphanumeric characters, sinographs, Japanese kanji or kana symbols, and / or other human written language characters. In addition, the “pictogram character” used in the present specification and claims refers to stationary and animated pictograms that are not text characters in human writing languages. The pictogram category icon corresponds to a plurality of pictogram categories. The computer-implemented method also detects a gesture in each pictogram category icon of the plurality of pictogram category icons, and in response to detecting a gesture in each pictogram category icon, the computer-implemented method includes a first emoji character key for each pictogram category in the keyboard display area. Displaying a subset and a plurality of subset sequence mark icons for each pictogram category at the same time, each pictogram category being configured to be displayed individually in the keyboard display area as a sequence of subsets of pictogram character keys It contains multiple subsets of emoji character keys, and the subset sequence mark icon shows the number of subsets in each emoji category and the emoji character key support in the keyboard display area. It gives information about the position of the subset is displayed in the set of sequences. Also, while displaying the first subset of pictogram character keys for each pictogram category in the keyboard display area, the computer-implemented method detects gestures in the keyboard display area and is responsive to detecting gestures in the keyboard display area. The display of the first subset of pictogram character keys for each pictogram category is replaced with the display of the second subset of pictogram character keys for each pictogram category in the keyboard display area, and the information provided by the subset sequence mark icon Updating to reflect the replacement of the emoticon character keys with the displayed first subset by the second subset.

  According to certain embodiments, the portable electronic device includes a touch screen display, one or more processors, memory, and one or more programs. The one or more programs are stored in a memory and configured to be executed by one or more processors. The one or more programs include a character input area operable to display a text character input and pictogram character input selected by a user, a keyboard display area, and instructions for displaying a plurality of pictogram category icons simultaneously, The pictogram category icon corresponds to a plurality of pictogram categories. In addition, the one or more programs detect a gesture in each pictogram category icon of a plurality of pictogram category icons, and respond to detection of a gesture in each pictogram category icon in response to detection of a gesture in each pictogram category icon And an instruction for simultaneously displaying a plurality of subset sequence mark icons for each pictogram category, each pictogram category being displayed individually in the keyboard display area as a sequence of a subset of pictogram character keys. Contains a large number of subsets of emoji character keys, and the subset sequence mark icon indicates the number of subsets in each emoji category and the emoticons in the keyboard display area. It gives information about the position of the subset is displayed in the sequence of a subset of the character keys. The program also detects gestures in the keyboard display area while displaying the first subset of emoji character keys for each pictogram category in the keyboard display area, and in response to detecting gestures in the keyboard display area, The display of the first subset of pictogram character keys for the pictogram category is replaced with the display of the second subset of pictogram character keys for each pictogram category in the keyboard display area, and the information provided by the subset sequence mark icon is replaced with the first of the pictogram character keys. Instructions are also included for updating to reflect the replacement of the displayed subset of pictogram character keys by the two subsets.

  According to one embodiment, a computer-readable storage medium displays text character input and pictogram character input selected by a user when executed on a portable electronic device with a touch screen display. A character input area, a keyboard display area, and a plurality of pictogram category icons that can be operated simultaneously are displayed. The pictogram category icon corresponds to a plurality of pictogram categories, and each pictogram category icon of the plurality of pictogram category icons A first subset of pictogram character keys for each pictogram category in the keyboard display area and each pictogram in response to detecting a gesture at each pictogram category icon Multiple subset sequence mark icons for a category are displayed simultaneously, and each emoji category includes multiple subsets of emoji character keys configured to be displayed individually in the keyboard display area as a sequence of emoji character key subsets. , And the subset sequence mark icon gives information about the number of subsets in each pictogram category and the position of the displayed subset in the pictogram character key subset sequence in the keyboard display area, and the pictogram for each pictogram category Detecting a gesture in the keyboard display area while displaying the first subset of the character keys in the keyboard display area, and in response to detecting the gesture in the keyboard display area, The display of the first subset of pictogram character keys for the character category is replaced with the display of the second subset of pictogram character keys for each pictogram category in the keyboard display area, and the information provided by the subset sequence mark icon is replaced with the first of the pictogram character keys. Instructions to be updated to reflect the replacement of the pictogram character keys by the two subsets with the displayed first subset are stored.

  According to an embodiment, a graphic user interface of a portable electronic device with a touch screen display includes a character input area operable to display a text character input and an emoji character input selected by a user, a keyboard display area, and a plurality of Equipped with emoji category icons. The pictogram category icon corresponds to a plurality of pictogram categories. Each pictogram category includes multiple subsets of pictogram character keys that are configured to be individually displayed in the keyboard display area as a sequence of subsets of pictogram character keys. The graphic user interface also includes a first subset of pictogram character keys for each pictogram category, a second subset of pictogram character keys for each pictogram category, and a plurality of subset sequence mark icons for each pictogram category. This subset sequence mark icon provides information regarding the number of subsets in each pictogram category and the position of the displayed subset in the pictogram character key subset sequence in the keyboard display area. A character input area, a keyboard display area, and a plurality of pictogram category icons are displayed simultaneously. In response to detecting a gesture in each pictogram category icon of the plurality of pictogram category icons, a first subset of pictogram character keys for each pictogram category in the keyboard display area is displayed simultaneously with a plurality of subset sequence mark icons for each pictogram category. . In response to detecting a gesture in the keyboard display area while displaying the first subset of pictogram character keys for each pictogram category in the keyboard display area, the display of the first subset of pictogram character keys for each pictogram category is displayed on the keyboard. It is replaced with the display of the second subset of pictogram character keys for each pictogram category in the display area, and the information provided by the subset sequence mark icon is the first subset in which the pictogram character keys are displayed by the second subset of pictogram character keys. Updated to reflect the replacement.

According to certain embodiments, a portable electronic device includes a touch screen display, a character input area operable to display a text character input and an emoji character input selected by a user, a keyboard display area, and a plurality of pictogram category icons. The pictogram category icon corresponds to a plurality of pictogram categories. The apparatus also includes means for detecting a gesture in each pictogram category icon of a plurality of pictogram category icons and a pictogram character key for each pictogram category in the keyboard display area in response to detection of a gesture in each pictogram category icon. And a means for simultaneously displaying a plurality of subset sequence mark icons for each pictogram category, wherein each pictogram category is individually displayed in the keyboard display area as a sequence of a subset of pictogram character keys. Includes a number of subsets of emoji character keys configured, and the subset sequence mark icon indicates the number of subsets in each emoji category and the emoji character keys in the keyboard display area. It gives information about the position of the subset is displayed in the sequence of subsets. The apparatus also responds to means for detecting gestures in the keyboard display area and detecting gestures in the keyboard display area while displaying the first subset of pictogram character keys for each pictogram category in the keyboard display area. The display of the first subset of pictogram character keys for each pictogram category is replaced with the display of the second subset of pictogram character keys for each pictogram category in the keyboard display area, and the information provided by the subset sequence mark icon is replaced with the pictogram character. Means for updating to reflect the replacement of the emoticon character key with the displayed first subset by the second subset of keys.

  According to certain embodiments, a computer-implemented method is performed in a portable electronic device with a touch screen display while the portable electronic device is in a user interface lock mode. The method receives an incoming message from a party to a user of a portable electronic device while the touch screen display is in a power save state, and in response to receiving the incoming message, powers on the touch screen display; and Displaying at least a portion of a message coming from the party to the user on the touch screen display. The method also detects one or more default finger gestures on the touch screen display corresponding to the emoji character, detects a default user action to initiate transmission of a response to the incoming message, and It also includes sending a response to the incoming message in response to detecting a predetermined user action to initiate sending a response to the incoming message. The response includes an emoji character corresponding to one or more predetermined finger gestures detected on the touch screen display.

  According to certain embodiments, the portable electronic device includes a touch screen display, one or more processors, memory, and one or more programs. The one or more programs are stored in the memory and are configured to be executed by the one or more processors. One or more programs receive an incoming message from a party to a user of the portable electronic device while the touch screen display is in a power saving state while the portable electronic device is in user interface lock mode, and receive the incoming message. In response to receiving, the touch screen display is powered on, and at least a portion of a message coming from the party to the user is displayed on the touch screen display, and one or more predefined ones on the touch screen display corresponding to the emoji character A default user action for detecting a finger gesture, detecting a default user action for initiating transmission of a response to an incoming message, and for initiating transmission of a response to the incoming message Out in response to, including instructions for sending a response to the incoming message. The response includes an emoji character corresponding to one or more predetermined finger gestures detected on the touch screen display.

  According to certain embodiments, a computer readable storage medium is displayed on a touch screen display when executed by the portable electronic device with a touch screen display while the portable electronic device is in a user interface lock mode. Stores instructions that cause it to receive incoming messages from the party to the user of the portable electronic device while the device is in the power saving state. The instruction also powers on the touch screen display and comes from the party to the user in response to the portable electronic device receiving the incoming message while it is in user interface lock mode. A default user action for displaying at least a portion of the message on the touch screen display, detecting one or more default finger gestures on the touch screen display corresponding to the emoji character, and initiating a response to the incoming message And in response to detecting a default user action to initiate transmission of a response to the incoming message, a response to the incoming message is transmitted. The response includes an emoji character corresponding to one or more predetermined finger gestures detected on the touch screen display.

  According to an embodiment, a graphic user interface of a portable electronic device with a touch screen display includes at least a portion of a message coming from a party to a user of the portable electronic device. While the portable electronic device is in user interface lock mode, the touch screen display is powered on in response to receiving an incoming message from the party to the user of the portable electronic device while the touch screen display is in a power saving state. And at least a portion of a message coming from the party to the user is displayed on the touch screen display. While the portable electronic device is in the user interface lock mode, one or more predefined finger gestures on the touch screen display corresponding to the emoji character are detected. While the portable electronic device is in the user interface lock mode, it transmits a response to the incoming message in response to detecting a default user action to initiate transmission of a response to the incoming message. The response includes an emoji character corresponding to one or more predetermined finger gestures detected on the touch screen display.

  According to one embodiment, the portable electronic device arrives at the user of the portable electronic device from the party while the touch screen display is in a power saving state while the touch screen display and the portable electronic device are in user interface lock mode. Means for receiving a message, means for powering on the touch screen display in response to receiving the incoming message, means for displaying at least a portion of the incoming message from the party to the user on the touch screen display, and an emoji character Means for detecting one or more default finger gestures on the touch screen display corresponding to the means for detecting, and means for detecting a default user action for initiating transmission of a response to an incoming message; In response to detecting a predefined user action to initiate the transmission of a response to the message, and means for transmitting a response to the incoming message. The response includes an emoji character corresponding to one or more predetermined finger gestures detected on the touch screen display.

  According to certain embodiments, a computer-implemented method is provided in a portable electronic device with a touch screen display from a party while the portable electronic device is in a user interface lock mode and the touch screen display is in a power save state. Incoming message to the user of the device and in response to receiving the incoming message, the touch screen display is powered on and at least a portion of the incoming message from the party to the user is displayed on the touch screen display and the incoming message Display a plurality of default responses to the default response, the default response includes a plurality of pictogram characters, and further detects and arrives at a default user action for selecting a default response among the plurality of default responses. Do Detect a default user action to start sending a selected default response to a message, and respond to detecting a default user action to start sending a selected default response to an incoming message Sending a selected default response to the incoming message. The selected default response includes a pictograph character among the plurality of pictograph characters.

  According to certain embodiments, a portable electronic device comprises a touch screen display, one or more processors, a memory, and one or more programs, the one or more programs stored in the memory, It is configured to be executed by one or more processors. One or more programs receive an incoming message from a party to the user of the portable electronic device while the touch screen display is in a power save state while the portable electronic device is in user interface lock mode, and receive an incoming message In response to powering on the touch screen display, displaying at least a portion of the incoming message from the party to the user on the touch screen display, displaying a plurality of predefined responses to the incoming message, the default response being: Detecting a default user action for selecting a default response among a plurality of default responses and initiating transmission of the selected default response to an incoming message. Default user action Out, and in response to the detection of the default user action to start sending default response that is selected for the incoming messages, including instructions for sending the default response that is selected for the incoming message. The selected default response includes a pictograph character among the plurality of pictograph characters.

  According to certain embodiments, a computer readable storage medium is displayed on a touch screen display when executed by the portable electronic device with a touch screen display while the portable electronic device is in a user interface lock mode. Receiving an incoming message from the party to the user of the portable electronic device while the device is in a power saving state, and in response to receiving the incoming message, powers on the touch screen display and at least a message coming from the party to the user A portion is displayed on the touch screen display, and multiple default responses to incoming messages are displayed, the default response includes multiple pictogram characters, and a default response is selected among the multiple default responses in order to To detect a predetermined user action, detect a default user action to initiate transmission of a selected default response to an incoming message, and initiate transmission of a selected default response to an incoming message In response to detecting the default user action, instructions are stored that cause a selected default response to be sent for an incoming message. The selected default response includes a pictograph character among the plurality of pictograph characters.

  According to an embodiment, a graphic user interface of a portable electronic device with a touch screen display includes at least a portion of a message coming from a party to a user of the portable electronic device and a plurality of predefined responses to the incoming message. . The prescribed response includes a plurality of pictogram characters. While the portable electronic device is in user interface lock mode, the touch screen display is powered on in response to receiving an incoming message from the party to the user of the portable electronic device while the touch screen display is in a power saving state. , Display at least a portion of the incoming message from the party to the user on the touch screen display, display multiple default responses to the incoming message, and select a default response among the multiple default responses In response to detecting a user action and detecting a default user action for initiating transmission of a selected default response to an incoming message, a selected default response for the incoming message is transmitted. The selected default response includes a pictograph character among the plurality of pictograph characters.

  According to one embodiment, the portable electronic device arrives at the user of the portable electronic device from the party while the touch screen display is in a power saving state while the touch screen display and the portable electronic device are in user interface lock mode. Means for receiving a message to respond, means for powering on the touch screen display in response to receiving the incoming message, means for displaying at least a portion of the message coming from the party to the user on the touch screen display Means for displaying a plurality of default responses to a message, wherein the default response includes a plurality of pictogram characters, and a default user for selecting a default response among the plurality of default responses Detect action And means for detecting a default user action for initiating transmission of a selected default response for an incoming message; and a default user for initiating transmission of a selected default response for an incoming message. Means for transmitting a selected default response to the incoming message in response to detecting the action. The selected default response includes a pictograph character among the plurality of pictograph characters.

  According to certain embodiments, a computer-implemented method is provided in a portable electronic device with a touch screen display from a party while the portable electronic device is in a user interface lock mode and the touch screen display is in a power save state. In response to receiving an incoming message to the user of the device and in response to receiving the incoming message, the touch screen display is powered on and at least a portion of the incoming message from the party to the user is displayed on the touch screen display A default user action for initiating display of a plurality of default responses to the message is detected, the default response including a plurality of pictogram characters, and a plurality of default responses to the incoming message. In response to detecting a default user action to start displaying the default user action to display multiple default responses and to select a default response among the multiple default responses Default user action for detecting and detecting a default user action for initiating transmission of a selected default response to an incoming message and for initiating transmission of a selected default response for an incoming message In response to detecting a predetermined default response to the incoming message. The selected default response includes a pictograph character among the plurality of pictograph characters.

  According to an embodiment, the portable electronic device comprises a touch screen display, one or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory, and It is configured to be executed by one or more processors. One or more programs receive an incoming message from a party to a user of the portable electronic device while the touch screen display is in a power saving state while the portable electronic device is in user interface lock mode, and receive the incoming message. In response to receiving, the touch screen display is powered on, at least a portion of the message coming from the party to the user is displayed on the touch screen display, and a default for initiating display of multiple default responses to the incoming message In response to detecting a default user action for initiating display of a plurality of default responses to an incoming message. Multiple default responses Displays a default user action for detecting a default user action for selecting a default response among multiple default responses and starting sending the selected default response for an incoming message. Instructions for sending a selected default response to an incoming message in response to detecting and detecting a default user action to initiate transmission of the selected default response to the incoming message including. The selected default response includes a pictograph character among the plurality of pictograph characters.

  According to certain embodiments, a computer readable storage medium is displayed on a touch screen display when executed by the portable electronic device with a touch screen display while the portable electronic device is in a user interface lock mode. Receives an incoming message from the party to the user of the portable electronic device while the device is in a power saving state, and in response to receiving the incoming message, powers on the touch screen display and Displaying at least a portion on the touch screen display and detecting a default user action for initiating display of a plurality of default responses to an incoming message, the default response including a plurality of emoji characters; In response to detecting a default user action to initiate display of multiple default responses to incoming messages, display multiple default responses, and display a default response in multiple default responses Detects default user action to select, detects default user action to initiate transmission of selected default response to incoming message, and transmits selected default response to incoming message In response to detecting a default user action for initiating a message, instructions are stored for sending a selected default response to the incoming message. The selected default response includes a pictograph character among the plurality of pictograph characters.

According to an embodiment, a graphic user interface of a portable electronic device with a touch screen display includes at least a portion of a message coming from a party to a user of the portable electronic device and a plurality of predefined responses to the incoming message. . The prescribed response includes a plurality of pictogram characters. While the portable electronic device is in user interface lock mode, the touch screen display is powered on in response to receiving an incoming message from the party to the user of the portable electronic device while the touch screen display is in a power saving state. Responsive to detecting a default user action to display at least a portion of an incoming message from the party to the user on the touch screen display and to initiate display of a plurality of default responses to the incoming message; Default response to view default response, detect default user action to select default response among multiple default responses, and start sending selected default response to incoming message In response to detecting a user action, Sending a default response that is selected for the message to be. The selected default response includes a pictograph character among the plurality of pictograph characters.

  According to one embodiment, the portable electronic device arrives at the user of the portable electronic device from the party while the touch screen display is in a power saving state while the touch screen display and the portable electronic device are in user interface lock mode. Means for receiving a message to respond, means for powering on the touch screen display in response to receiving the incoming message, means for displaying at least a portion of the message coming from the party to the user on the touch screen display Means for detecting a default user action for initiating display of a plurality of default responses to a message, wherein the default response includes a plurality of pictogram characters; and a plurality of default responses to an incoming message A means for displaying multiple default responses in response to detecting a default user action for initiating display of responses, and a default for selecting a default response among the multiple default responses Means for detecting a user action, means for detecting a default user action for initiating transmission of a selected default response to an incoming message, and initiating transmission of a selected default response for an incoming message Means for transmitting a selected default response to an incoming message in response to detecting a default user action for the. The selected default response includes a pictograph character among the plurality of pictograph characters.

  According to an embodiment, a computer-implemented method detects in a portable electronic device with a touch screen display a user-defined gesture on the touch screen display while the portable electronic device is in a user-defined gesture recording mode. Generating a representation of the defined gesture, associating the representation of the user-defined gesture with the pictogram character, and storing the representation of the user-defined gesture and the association of the user-defined gesture representation with the pictogram character.

  According to certain embodiments, a portable electronic device comprises a touch screen display, one or more processors, a memory, and one or more programs, the one or more programs stored in the memory, It is configured to be executed by one or more processors. One or more programs detect a user-defined gesture on the touch screen display while the portable electronic device is in a user-defined gesture recording mode, generate a representation of the user-defined gesture, and display the representation of the user-defined gesture. It includes instructions for associating with the pictogram character and storing the user-defined gesture representation and the relevance between the user-defined gesture representation and the pictogram character.

  According to one embodiment, the computer-readable storage medium is displayed on a touch screen display when the portable electronic device is in a user-defined gesture recording mode when executed by the portable electronic device with a touch screen display. Detecting a user-defined gesture above, generating a representation of the user-defined gesture, associating the representation of the user-defined gesture with a pictogram character, and expressing the user-defined gesture and the representation of the user-defined gesture with the pictogram character Instructions for memorizing relevance are stored.

  According to an embodiment, the portable electronic device includes a touch screen display, means for detecting a user-defined gesture on the touch screen display while the portable electronic device is in a user-defined gesture recording mode, and the user-defined gestures. Means for generating an expression; means for associating the expression of the user-defined gesture with the pictogram character; and means for storing the expression of the user-defined gesture and the relationship between the expression of the user-defined gesture and the pictogram character. .

  According to certain embodiments, a computer-implemented method includes receiving an electronic message at a portable electronic device with a touch screen display. The electronic message includes instructions for starting the geographic map application, location data representing a location on the geographic map, and pictographic data representing a pictographic character. The pictographic data is associated with the position data. The method also displays an electronic message on the touch screen display, detects a default user action for the displayed electronic message to start the geographic map application, and sets a default for starting the geographic map application. In response to detecting the user action, displaying a geographic map in the geographic map application, and further displaying an emoji character at a location on the geographic map corresponding to the location data.

  According to certain embodiments, a portable electronic device comprises a touch screen display, one or more processors, a memory, and one or more programs, the one or more programs stored in the memory, It is configured to be executed by one or more processors. The one or more programs include instructions for receiving an electronic message, the electronic message including instructions for starting a geographic map application, location data representing a location on the geographic map, and pictographic data representing a pictographic character. The pictographic data is associated with the position data. One or more programs also display an electronic message on the touch screen display, detect a default user action for the displayed electronic message to start the geographic map application, and start the geographic map application. In response to detecting a default user action for displaying a geographic map in the geographic map application, and further including instructions for displaying an emoji character at a location on the geographic map corresponding to the location data.

According to certain embodiments, a computer-readable storage medium stores instructions that, when executed by a portable electronic device with a touch screen display, cause the device to receive an electronic message. The electronic message includes instructions for starting the geographic map application, location data representing a location on the geographic map, and pictographic data representing a pictographic character. The pictographic data is associated with the position data. The instruction also causes the device to display an electronic message on the touch screen display, detect a default user action for the displayed electronic message to initiate the geographic map application, and initiate the geographic map application. In response to detecting a default user action for displaying a geographic map in the geographic map application, and further displaying an emoji character at a location on the geographic map corresponding to the location data.

  According to an embodiment, a graphic user interface of a portable electronic device with a touch screen display includes an electronic message, a geographic map, and an emoji character. The electronic message is received by the portable electronic device. The electronic message includes instructions for starting the geographic map application, location data representing a location on the geographic map, and pictographic data representing a pictographic character. The pictographic data is associated with the position data. The electronic message is displayed on the touch screen display. In response to detecting a default user action for that displayed electronic message to initiate the geographic map application, a geographic map in the geographic map application is displayed, and the location on the geographic map corresponding to the location data A pictogram character is displayed.

  According to an embodiment, the portable electronic device comprises a touch screen display and means for receiving an electronic message, the electronic message including instructions for initiating a geographic map application, location data representing a location on the geographic map, and It includes pictographic data representing a pictographic character, and the pictographic data is associated with position data. The apparatus also includes means for displaying an electronic message on the touch screen display, means for detecting a default user action for the displayed electronic message to initiate the geographic map application, and starting the geographic map application. Means for displaying a geographic map in the geographic map application in response to detecting a default user action for, and means for displaying an emoji character at a location on the geographic map corresponding to the location data. .

  Accordingly, the present invention provides a simple, efficient and easy-to-use interface for using pictograms in portable electronic devices with touch sensitive displays. Such an interface conserves power by allowing the user to complete tasks faster and extend the time between battery charges. Such an interface also increases user satisfaction with portable electronic devices.

  For a better understanding of the above and additional embodiments of the present invention, a detailed description of the embodiments will be given with reference to the accompanying drawings, wherein like reference numerals designate corresponding parts throughout.

1 is a block diagram illustrating a portable multifunction device with a touch-sensitive display according to some embodiments. FIG. 1 is a block diagram illustrating a portable multifunction device with a touch-sensitive display according to some embodiments. FIG. 1 illustrates a portable multifunction device having a touch screen according to certain embodiments. 6 illustrates an example user interface for unlocking a portable electronic device according to some embodiments. FIG. 4 illustrates an example user interface for a menu of applications in a portable multifunction device according to an embodiment. FIG. 4 illustrates an example user interface for a menu of applications in a portable multifunction device according to an embodiment. FIG. 6 illustrates an example user interface for providing and organizing pictogram character keys on a portable electronic device equipped with a touch-sensitive display according to some embodiments. FIG. 6 illustrates an example user interface for providing and organizing pictogram character keys on a portable electronic device equipped with a touch-sensitive display according to some embodiments. FIG. 6 illustrates an example user interface for providing and organizing pictogram character keys on a portable electronic device equipped with a touch-sensitive display according to some embodiments. FIG. 6 illustrates an example user interface for providing and organizing pictogram character keys on a portable electronic device equipped with a touch-sensitive display according to some embodiments. FIG. 6 illustrates an example user interface for providing and organizing pictogram character keys on a portable electronic device equipped with a touch-sensitive display according to some embodiments. FIG. 6 illustrates an example user interface for providing and organizing pictogram character keys on a portable electronic device equipped with a touch-sensitive display according to some embodiments. FIG. 6 illustrates an example user interface for providing and organizing pictogram character keys on a portable electronic device equipped with a touch-sensitive display according to some embodiments. FIG. 4 illustrates an example user interface for navigating between pictogram character subsets of a particular pictogram category and entering certain pictogram characters in a portable electronic device equipped with a touch-sensitive display according to some embodiments. FIG. 4 illustrates an example user interface for navigating between pictogram character subsets of a particular pictogram category and entering certain pictogram characters in a portable electronic device equipped with a touch-sensitive display according to some embodiments. FIG. 4 illustrates an example user interface for navigating between pictogram character subsets of a particular pictogram category and entering certain pictogram characters in a portable electronic device equipped with a touch-sensitive display according to some embodiments. FIG. 4 illustrates an example user interface for navigating between pictogram character subsets of a particular pictogram category and entering certain pictogram characters in a portable electronic device equipped with a touch-sensitive display according to some embodiments. FIG. 4 illustrates an example user interface for navigating between pictogram character subsets of a particular pictogram category and entering certain pictogram characters in a portable electronic device equipped with a touch-sensitive display according to some embodiments. FIG. 6 illustrates an example user interface for entering emoticon characters and transmitting an electronic message including the emoticon characters in a portable electronic device equipped with a touch-sensitive display according to some embodiments. FIG. 6 illustrates an example user interface for entering emoticon characters and transmitting an electronic message including the emoticon characters in a portable electronic device equipped with a touch-sensitive display according to some embodiments. FIG. 6 illustrates an example user interface for entering emoticon characters and transmitting an electronic message including the emoticon characters in a portable electronic device equipped with a touch-sensitive display according to some embodiments. FIG. 6 illustrates an example user interface for entering emoticon characters and transmitting an electronic message including the emoticon characters in a portable electronic device equipped with a touch-sensitive display according to some embodiments. FIG. 6 illustrates an example user interface for entering emoticon characters and transmitting an electronic message including the emoticon characters in a portable electronic device equipped with a touch-sensitive display according to some embodiments. 6 illustrates an example user interface for reconfiguring emoji character keys of a software keyboard in a portable electronic device equipped with a touch sensitive display according to some embodiments. 6 illustrates an example user interface for reconfiguring emoji character keys of a software keyboard in a portable electronic device equipped with a touch sensitive display according to some embodiments. 6 illustrates an example user interface for reconfiguring emoji character keys of a software keyboard in a portable electronic device equipped with a touch sensitive display according to some embodiments. FIG. 4 illustrates an example user interface for responding to a message with pictograms while a portable electronic device equipped with a touch-sensitive display is in a user interface lock mode according to an embodiment. FIG. 4 illustrates an example user interface for responding to a message with pictograms while a portable electronic device equipped with a touch-sensitive display is in a user interface lock mode according to an embodiment. FIG. 4 illustrates an example user interface for responding to a message with pictograms while a portable electronic device equipped with a touch-sensitive display is in a user interface lock mode according to an embodiment. FIG. 4 illustrates an example user interface for responding to a message with pictograms while a portable electronic device equipped with a touch-sensitive display is in a user interface lock mode according to an embodiment. FIG. 4 illustrates an example user interface for responding to a message with pictograms while a portable electronic device equipped with a touch-sensitive display is in a user interface lock mode according to an embodiment. FIG. 4 illustrates an example user interface for responding to a message with pictograms while a portable electronic device equipped with a touch-sensitive display is in a user interface lock mode according to an embodiment. FIG. 4 illustrates an example user interface for responding to a message with pictograms while a portable electronic device equipped with a touch-sensitive display is in a user interface lock mode according to an embodiment. FIG. 4 illustrates an example user interface for responding to a message with pictograms while a portable electronic device equipped with a touch-sensitive display is in a user interface lock mode according to an embodiment. FIG. 4 illustrates an example user interface for assigning user-defined gestures to emoji characters in a portable electronic device equipped with a touch-sensitive display according to some embodiments. FIG. 4 illustrates an example user interface for assigning user-defined gestures to emoji characters in a portable electronic device equipped with a touch-sensitive display according to some embodiments. FIG. 4 illustrates an example user interface for assigning user-defined gestures to emoji characters in a portable electronic device equipped with a touch-sensitive display according to some embodiments. FIG. 4 illustrates an example user interface for assigning user-defined gestures to emoji characters in a portable electronic device equipped with a touch-sensitive display according to some embodiments. FIG. 4 illustrates an example user interface for assigning user-defined gestures to emoji characters in a portable electronic device equipped with a touch-sensitive display according to some embodiments. FIG. 5 illustrates an example user interface for using an emoji character as a position indicator on a map via an electronic message according to an embodiment. FIG. 5 illustrates an example user interface for using an emoji character as a position indicator on a map via an electronic message according to an embodiment. 6 is a flowchart illustrating a method of operating an emoji character keyboard in a portable electronic device equipped with a touch-sensitive display according to an embodiment. 6 is a flowchart illustrating a method of operating an emoji character keyboard in a portable electronic device equipped with a touch-sensitive display according to an embodiment. 6 is a flowchart illustrating a method of operating an emoji character keyboard in a portable electronic device equipped with a touch-sensitive display according to an embodiment. 6 is a flowchart illustrating a method of operating an emoji character keyboard in a portable electronic device equipped with a touch-sensitive display according to an embodiment. 6 is a flowchart illustrating a method of operating an emoji character keyboard in a portable electronic device equipped with a touch-sensitive display according to an embodiment. 6 is a flowchart illustrating a method of operating an emoji character keyboard in a portable electronic device equipped with a touch-sensitive display according to an embodiment. 6 is a flowchart illustrating a method for responding to a message with pictograms while a portable electronic device equipped with a touch-sensitive display is in a user interface lock mode according to an embodiment. 6 is a flowchart illustrating a method for responding to a message with pictograms while a portable electronic device equipped with a touch-sensitive display is in a user interface lock mode according to an embodiment. 6 is a flowchart illustrating a method for responding to a message with pictograms while a portable electronic device equipped with a touch-sensitive display is in a user interface lock mode according to an embodiment. 4 is a flowchart illustrating a method for specifying a user-defined gesture for a pictograph in a portable electronic device equipped with a touch-sensitive display according to an embodiment. 6 is a flowchart illustrating a method for using a pictograph as a position indicator on a map via an electronic message according to an embodiment.

  Embodiments shown in the accompanying drawings will be described in detail below. In the following detailed description, numerous specific details are set forth in order to provide a thorough understanding of the present invention. However, it will be apparent to one skilled in the art that the present invention may be practiced without these specific details. In other instances, well-known methods, procedures, components, circuits, and networks have not been described in detail in order not to unnecessarily obscure aspects of the embodiments.

  Also, although the terms first, second, etc. are used herein to describe various elements, it should also be understood that these elements are not limited by these words. These terms are only used to distinguish one element from another. For example, a first gesture may be referred to as a second gesture, and, similarly, a second gesture may be referred to as a first gesture without departing from the scope of the present invention.

  The terminology used in the description of the present invention is merely for describing particular embodiments and is not intended to limit the present invention in any way. In the description of the invention and in the claims, the singular forms “a”, “an”, and “the” are intended to include the plural forms as well, unless expressly stated otherwise. Also, as used herein, the term “and / or” should be understood to refer to and encompass any and all possible combinations of one or more of the associated items listed. Further, as used herein, the terms “comprises” and / or “comprising” identify the presence of the described feature, integer, step, action, element, and / or component. The presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof is not excluded.

  As used herein, the term “if” refers to “when” or “upon” or “in response to a decision” or “detection” depending on the situation. Is interpreted to mean "in response to". Similarly, the phrases “when determined” or “when [stated condition or event] is detected” are used in the context of “when determining” or “determining” depending on the circumstances. "In response to detecting" or "in response to detecting [describing condition or event]" or "in response to detecting [describing condition or event]".

  A portable multifunction device, a user interface for such a device, and associated processes for using such a device are described. In some embodiments, the device is a portable communication device such as a mobile phone that also includes other functions, such as PDA and / or music player functions. Exemplary embodiments of portable multifunction devices include, but are not limited to, iPhone® and iPod Touch® devices available from Apple Computer, Cupertino, California.

  The user interface can include a physical click wheel in addition to the touch screen or a virtual click wheel displayed on the touch screen. A click wheel is a user interface device that provides navigation commands based on the angular displacement of the wheel or the point of contact of the device by the user with the wheel. The click wheel is also used, for example, to provide a user command corresponding to the selection of one or more items when the user of the device presses at least a portion of the wheel or the center of the wheel. Alternatively, blocking contact with the click wheel image on the touch screen surface indicates a user command corresponding to the selection. For simplicity, in the following description, a portable multifunction device that includes a touch screen is used as an exemplary embodiment. However, it should be understood that some of the user interface and associated processes can be applied to other portable devices.

  The device can support various applications, for example, one or more of the following applications. Phone applications, video conferencing applications, email applications, instant messaging applications, blog applications, photo management applications, digital camera applications, digital video camera applications, web browsing applications, digital music player applications, and / or digital video player applications .

  Various applications that can run on the device use at least one common physical user interface device, such as a touch screen. One or more functions of the touch screen and the corresponding information displayed on the device are adjusted and / or changed from one application to the next and / or within each application. In this way, the common physical architecture of devices (such as touch screens) can support a variety of applications with an intuitive and transparent user interface.

  The user interface includes a soft keyboard embodiment that is operable to input or operate to input text characters and pictogram characters. The soft keyboard embodiment includes a standard (QWERTY) configuration and / or a non-standard configuration symbol on the keyboard display icon, for example, “Keyboards For Portable Electronic Devices” filed on July 24, 2006. US patent application Ser. No. 11 / 459,606, filed July 24, 2006, and US Patent Application No. 11 / 459,615 filed July 24, 2006, entitled “Touch Screen Keyboards For Portable Electronic Devices” The contents of these patent applications are incorporated herein by reference as they are. Keyboard embodiments include a small number of icons (or soft keys) for the number of keys in an existing physical keyboard, such as for a typewriter. This facilitates the user to select one or more icons and thus one or more corresponding symbols on the keyboard. This keyboard embodiment is adaptive. For example, the display icon can be changed based on a user action such as selecting one or more icons and / or one or more corresponding symbols. One or more applications in the portable device can use common and / or different keyboard embodiments. Thus, the keyboard embodiment used can be tailored for at least some applications. In some embodiments, one or more keyboard embodiments can be adjusted for each user. For example, one or more keyboard embodiments can be tailored for each user based on each user's word usage history (dictionary, slang, individual usage). When selecting one or more icons, and thus selecting one or more symbols when using a soft keyboard embodiment, adjust some keyboard embodiments to reduce the probability of user error. be able to. A soft keyboard that inputs or operates to input pictogram characters is described in detail below.

  Attention will be directed to device embodiments. 1A and 1B are block diagrams illustrating a portable multifunction device 100 with a touch-sensitive display 112 according to an embodiment. Touch-sensitive display 112 is also referred to as a “touch screen” for convenience, and is also known or referred to as a touch-sensitive display system. The apparatus 100 includes a memory 102 (including one or more computer-readable storage media), a memory controller 122, one or more processing units (CPUs) 120, a peripheral interface 118, an RF circuit 108, and audio. A circuit 110, a speaker 111, a microphone 113, an input / output (I / O) subsystem 106, another input or control device 116, and an external port 124 are provided. The apparatus 100 includes one or more optical sensors 164. These components can communicate via one or more communication buses or signal lines 103.

  The device 100 is only one example of a portable multifunction device 100, and the device 100 may have more or less components shown, may combine two or more components, or It will be apparent that different configurations or arrangements may be provided. The various components shown in FIGS. 1 and 1B may be implemented in hardware, software, or a combination of both, including one or more signal processing circuits and / or application specific integrated circuits.

  Memory 102 may include high speed random access memory, and may also include non-volatile memory, such as one or more magnetic disk storage devices, flash memory devices, or other non-volatile solid state memory devices. Access to the memory 102 by other components of the device 100, such as the CPU 120 and the peripheral interface 118, can be controlled by the memory controller 122.

  Peripheral interface 118 couples device input and output peripheral devices to CPU 120 and memory 102. One or more processors 120 run or execute various software programs and / or instruction sets stored in memory 102 to perform various functions of device 100 and process data.

  In some embodiments, peripheral interface 118, CPU 120, and memory controller 122 are implemented in a single chip, such as chip 104. In other embodiments, they may be embodied in separate chips.

  The RF (high frequency) circuit 108 receives and transmits an RF signal, also called an electromagnetic signal. The RF circuit 108 converts electrical signals to electromagnetic signals and vice versa, and communicates with communication networks and other communication devices via electromagnetic signals. The RF circuit 108 includes well-known circuits for performing these functions, including but not limited to antenna systems, RF transceivers, one or more amplifiers, tuners, one or more oscillators, digital signal processors. , CODEC chipset, subscriber identity module (SIM) card, memory, etc. The RF circuit 108 is a network such as the Internet, also referred to as the World Wide Web (WWW), an intranet and / or a wireless network, such as a cellular telephone network, a wireless local area network (LAN), and / or a metropolitan area network (MAN). ), And other devices. Wireless communication uses any of a number of communication standards, protocols, and technologies that include the Global System for Mobile Communications (GSM), Enhanced Data GSM Environment (EDGE), High Speed Downlink Packet Access (HSDPA), Wideband code division multiple access (W-CDMA), code division multiple access (CDMA), time division multiple access (TDMA), Bluetooth, wireless fidelity (Wi-Fi) (eg, IEEE 802.11a, IEEE 802.11b, IEEE 802. 11g and / or IEEE 802.11n), Voice over Internet Protocol (VoIP), Wi-MAX, e-mail protocol (eg, Internet Message Access Protocol (IMAP) and / or Post Office protocol (POP)), instant messaging (eg, extensible messaging and presence protocol (XMPP), session initiation protocol for instant messaging and presence leverage extension (SIMPLE), instant messaging and presence service (IMPS)), and / or Or other suitable communication protocols including, but not limited to, Short Messaging Service (SMS), or communication protocols not yet developed on the filing date of this application.

  The audio circuit 110, the speaker 111, and the microphone 113 form an audio interface between the user and the device 100. The audio circuit 110 receives audio data from the peripheral interface 118, converts the audio data into an electrical signal, and transmits the electrical signal to the speaker 111. The speaker 111 converts an electrical signal into a sound wave that can be heard by humans. The audio circuit 110 also receives an electrical signal converted from a sound wave by the microphone 113. The audio circuit 110 converts the electrical signal into audio data and transmits the audio data to the peripheral interface 118 for processing. Audio data is retrieved from and / or transmitted to memory 102 and / or RF circuit 108 by peripheral interface 118. In some embodiments, the audio circuit 110 also includes a headset jack (eg, 212 in FIG. 2). The headset jack provides both audio circuitry 110 and removable audio input / output peripheral devices, such as output-only headphones, or both output (eg, one or both ear headphones) and input (eg, microphone). Interface with the accompanying headset.

  The I / O subsystem 106 couples the input / output peripherals of the device 100, such as the touch screen 112 and other input / output devices 116, to the peripheral interface 118. The I / O subsystem 106 includes a display controller 156 and one or more input controllers 160 for other inputs or control devices. One or more input controllers 160 receive / send electrical signals to / from other inputs or control device 116. Other input / control devices 116 include physical buttons (eg, push buttons, rocker buttons, etc.), dials, slider switches, joysticks, click wheels, etc. In certain other embodiments, the input controller 160 is coupled to (or not coupled to) any of the following: That is, a pointer device such as a keyboard, an infrared port, a USB port, and a mouse. One or more buttons (eg, 208 in FIG. 2) include up / down buttons for volume control of the speaker 111 and / or microphone 113. The one or more buttons include a push button (eg, 206 in FIG. 2). A quick press of the push button releases the lock on the touch screen 112 or the process of unlocking the device using a gesture on the touch screen begins, which is hereby incorporated by reference as of December 23, 2005. This is described in US patent application Ser. No. 11 / 322,549, entitled “Unlocking a Device by Performing Gestures on an Unlock Image”. Pressing and holding a push button (eg, 206) can turn on or off power to the device 100. The user can customize the function of one or more buttons. The touch screen 112 is used to implement a virtual or soft button and one or more soft keyboards.

  Touch sensitive touch screen 112 provides an input interface and an output interface between the device and the user. Display controller 156 receives / transmits electrical signals from / to touch screen 112. Touch screen 112 displays visual output to the user. Visual output includes graphics, text, icons, video, and combinations thereof (collectively referred to as “graphics”). In some embodiments, some or all of the visual output corresponds to user interface objects.

  Touch screen 112 has a touch-sensitive surface, sensor, or set of sensors that accepts input from a user based on haptic and / or tactile touch. The touch screen 112 and display controller 156 (and associated modules and / or set of instructions in the memory 102) detect contact (and movement or disconnection of the contact) on the touch screen 112, and the detected contact is Translating into interaction with user interface objects (eg, one or more soft keys, icons, web pages or video) displayed on the touch screen. In the exemplary embodiment, the point of contact between the touch screen 112 and the user corresponds to the user's finger.

  The touch screen 112 uses LCD (liquid crystal display) technology, or LPD (light emitting polymer display) technology, although other display technologies may be used in other embodiments. The touch screen 112 and the display controller 156 can detect touch and its movement or disconnection using any of a plurality of touch sensing technologies now known or developed in the future. , Resistive, infrared, and surface acoustic wave technologies, and other proximity sensor arrays, or other elements for determining one or more points of contact with the touch screen 112, but are not so limited. In an exemplary embodiment, a projected mutual capacitance sensing technique such as found in iPhone® and iPod Touch® available from Apple Computer, Inc. of Cupertino, California is used.

  The touch sensitive display in certain embodiments of the touch screen 112 is similar to the multi-touch sensitive tablet described in the following US patents, each of which is incorporated herein by reference. 6,323,846 (Mr. Westerman, etc.), 6,570,557 (Mr. Westerman, etc.), and / or 6,677,932 (Mr. Westerman), and / or US Patent Publication No. 2002 / 0015024A1. However, touch screen 112 displays visual output from portable device 100, but touch sensitive tablets do not provide visual output.

  The touch sensitive display in one embodiment of the touch screen 112 is described in the following patent application. (1) US Patent Application No. 11 / 381,313 entitled “Multipoint Touch Surface Controller” filed on May 2, 2006; (2) “Multipoint Touch screen” filed on May 6, 2004; (3) US Patent Application No. 10 / 903,964, filed July 30, 2004, entitled “Gestures For Touch Sensitive Input Devices”; US Patent Application No. 11 / 048,264 entitled “Gestures For Touch Sensitive Input Devices” filed on January 31, 2005; (5) “Mode-Based Graphical User” filed on January 18, 2005; US Patent Application No. 11 / 038,590 entitled “Interfaces For Touch Sensitive Input Devices”; (6) entitled “Virtual Input Device Placement On A Touch Screen User Interface” filed on September 16, 2005; (7) US Patent Application No. 11 / 228,700 entitled “Operation Of A Computer With A Touch Screen Interface” filed on September 16, 2005; 8) US Patent Application No. 11 / 228,737 entitled “Activating Virtual Keys Of A Touch-Screen Virtual Keyboard” filed on September 16, 2005; and (9) filed on March 3, 2006. No. 11 / 367,749 entitled “Multi-Functional Hand-Held Device”. All of these patent applications are incorporated herein by reference in their entirety.

  The touch screen 112 has a resolution of 100 dpi or more. In an exemplary embodiment, the touch screen has a resolution of about 160 dpi. The user can touch the touch screen 112 using a suitable object or accessory, such as a stylus, finger, etc. In some embodiments, the user interface is designed primarily to work with finger-based touches and gestures, which is significantly less accurate than stylus-based input due to the large finger contact area to the touch screen. In some embodiments, the device converts finger-based rough input into a precise pointer / cursor position or command for performing the action desired by the user.

  In some embodiments, in addition to the touch screen, device 100 includes a touch pad (not shown) for activating or deactivating specific functions. In some embodiments, the touchpad is a touch sensitive area of the device that does not display visual output, unlike a touch screen. The touch pad may be a touch-sensitive surface that is separate from the touch screen 112 or may be an extension of the touch-sensitive surface formed by the touch screen.

  In some embodiments, device 100 includes a physical or virtual click wheel as input control device 116. The user displays on the touch screen 112 by rotating the click wheel or moving the contact point with the click wheel (eg, the amount of movement of the contact point is measured by its angular displacement relative to the center point of the click wheel). Can be navigated and interacted with one or more graphical objects (hereinafter referred to as icons). The click wheel may also be used to select one or more of the displayed icons. For example, the user can press at least a portion of the click wheel or associated button. User commands and navigation commands provided by the user via the click wheel can be processed by the input controller 160 and one or more of the modules and / or instruction sets in the memory 102. In the case of a virtual click wheel, the click wheel and click wheel controller are part of the touch screen 112 and display controller 156, respectively. In the case of a virtual click wheel, the click wheel is either an opaque or translucent object that appears and disappears on the touch screen display in response to user interaction with the device. In some embodiments, the virtual click wheel is displayed on the touch screen of the portable multifunction device and is manipulated by user contact with the touch screen.

  The apparatus 100 also includes a power supply system 162 for powering various components. The power system 162 includes a power management system, one or more power sources (eg, battery, alternating current (AC)), recharging system, power failure detection circuit, power converter or inverter, power status indicator (eg, light emitting diode (LED)). ), And other components related to power generation, management and distribution in portable devices.

The device 100 also includes one or more optical sensors 164. FIGS. 1A and 1B show an optical sensor coupled to an optical sensor controller 158 in the I / O subsystem 106. The optical sensor 164 includes a charge coupled device (CCD) or a complementary metal oxide semiconductor (CMOS) phototransistor. Optical sensor 164 receives light from the environment projected through one or more lenses and converts the light into data representing the image. In conjunction with the imaging module 143 (also referred to as a camera module), the optical sensor 164 can capture still images or video. In some embodiments, the optical sensor is located on the back of the device 100, as opposed to the touch screen display 112 in front of the device 100, and the touch screen display is used as a viewfinder for still and / or video image acquisition. It can be used. In some embodiments, an optical sensor is placed in front of the device to obtain the user's video for the video conference while allowing the user to view other video conference participants on the touch screen display. In some embodiments, the position of the optical sensor 164 may be changed by the user (eg, by rotating the lens and sensor in the device housing) to make the single optical sensor 164 a video conference and a stationary and / or video image. Allows use with a touch screen display for both acquisition.

  The device 100 also includes one or more proximity sensors 166. 1A and 1B show the proximity sensor 166 coupled to the peripheral interface 118. Alternatively, proximity sensor 166 may be coupled to input controller 160 of I / O subsystem 106. Proximity sensor 166 is US patent application Ser. No. 11 / 241,839 entitled “Proximity Detector In Handheld Device”, incorporated herein by reference in its entirety, and 11 / 240,788 entitled “Proximity Detector In Handheld Device”; 11 / 620,702 entitled “Using Ambient Light Sensor To Augment Proximity Sensor Output”; 11 / 586,862 entitled “Automated Response To And Sensing Of User Activity In Portable Devices”; and “Methods And Systems For Automatic It functions as described in 11 / 638,251 entitled “Configuration Of Peripherals”. In some embodiments, the proximity sensor turns off and disables the touch screen 112 when the multifunction device is placed near the user's ear (eg, when the user places a phone call). In some embodiments, the proximity sensor keeps the screen off when the device is in the user's pocket, handbag, or other dark area to prevent unnecessary battery consumption when the device is locked. To do.

  The device 100 also includes one or more accelerometers 168. 1A and 1B show an accelerometer 168 coupled to the peripheral interface 118. Alternatively, the accelerometer 168 may be coupled to the input controller 160 of the I / O subsystem 106. Accelerometers 168 are entitled US Patent Publication No. 20050190059 entitled “Acceleration-based Theft Detection System for Portable Electronic Devices” and “Methods And Apparatuses For Operating A Portable Device Based On An Accelerometer” which is incorporated herein by reference in its entirety. It functions as described in US Patent Publication No. 20060017692. In certain embodiments, the information is displayed on the touch screen display in portrait view or landscape view based on analysis of data received from one or more accelerometers.

  In some embodiments, software components stored in memory 102 include operating system 126, communication module (or instruction set) 128, touch / motion module (or instruction set) 130, graphics module (or instruction set) 132, A text input module (or instruction set) 134, a global positioning system (GPS) module (or instruction set) 135, and an application (or instruction set) 136 are included.

  Operating system 126 (eg, Darwin, RTXC, LINUX, OS X, WINDOWS, or embedded operating system, eg, VxWorks) performs common system tasks (eg, memory management, storage device control, power management, etc.) It includes various software components and / or drivers for controlling and managing and facilitates communication between various hardware and software components. Communication module 128 facilitates communication with other devices through one or more external ports 124 and also includes various software components for handling data received by RF circuit 108 and / or external port 124. . The external port 124 (eg, universal serial bus (USB), FIREWIRE, etc.) is directly connected to other devices or indirectly connected via a network (eg, the Internet, wireless LAN, etc.). In some embodiments, the external port is the same, similar and / or compatible multi-pin (eg, 30-pin) connector as the 30-pin connector used in iPod (Apple Computer ™) devices. It is.

  The contact / motion module 130 can detect contact with the touch screen 112 (associated with the display controller 156) and other touch sensing devices (eg, a touchpad or physical click wheel). The contact / motion module 130 determines if a contact has occurred (eg, detects a finger down event), determines if there is a touch movement, and tracks the movement across the touch screen 112 (eg, one or more). A variety of operations to perform various operations related to contact detection, such as detecting finger drag events), determining whether contact has stopped (eg, detecting a finger up event or contact disconnection), etc. Includes software components. Contact / motion module 130 receives contact data from touch screen 112. Determining the movement of a contact point represented by a series of contact data can include the speed (size), speed (size and direction), and / or acceleration (change in size and / or direction) of the contact point. Including deciding. These operations apply to a single contact (eg, one finger contact) or multiple simultaneous contacts (eg, “multi-touch” / multiple finger contacts). In some embodiments, contact / motion module 130 and display controller 156 also detect contact on the touchpad. In some embodiments, contact / motion module 130 and controller 160 detect contact on the click wheel.

  The contact / motion module 130 can detect a gesture input by a user. Different gestures on the touch screen 112 have different contact patterns. Therefore, a gesture can be detected by detecting a specific contact pattern. For example, detecting a finger tap gesture can be performed following detection of a finger down event at the same position (or substantially the same position) as the finger down event (eg, as a text character key or an emoji character key on a soft keyboard). Detecting a finger up event (at the location of a simple button type user interface object). As another example, detecting a finger swipe gesture on a touch screen includes detecting one or more finger drag events followed by a finger up event following detection of a finger down event. In some embodiments, the contact / movement module 130 detects user-defined gestures.

  The graphics module 132 includes various known software components for rendering and displaying graphics on the touch screen 112, including components that change the brightness of the displayed graphics. As used herein, the term “graphic” includes objects that can be displayed to the user, including text, web pages, icons (like user interface objects that include softkeys), digital video, video, animation, etc. It is not limited to this.

  In some embodiments, the graphics module 132 stores data representing the graphics used. Each graphic is assigned a corresponding code. The graphics module 132 receives one or more codes identifying the graphics to be displayed, optionally from the application, along with coordinate data and other graphics property data, and generates screen video data to display Output to the controller 156.

  A text input module 134, a component of the graphics module 132, for entering text in various applications (eg, contacts 137, e-mail 140, IM 141, browser 147, and other applications that require text input). Give a soft keyboard. The text input module 134 is also configured to provide an emoji soft keyboard for entering emoji characters in various applications (e.g., e-mail 140, IM 141, and other applications that use emoji character input). The In some embodiments, the text input module 134 stores data representing a soft keyboard and an emoji soft keyboard for entering text.

  The GPS module 135 determines the location of the device and uses this information for various applications (eg, phone 138 for use in local base dialing, camera 143 as photo / video metadata, as well as weather widgets, local yellow pages Application for providing location-based services such as widgets and map / navigation widgets).

The application 136 includes the next module (or instruction set), or a subset or superset thereof.
A contact module 137 (also referred to as an address book or contact list);
A telephone module 138;
Video conferencing module 139;
E-mail client module 140;
An instant messaging (IM) module 141;
-Training support module 142;
A camera module 143 for still and / or video images;
A video management module 144;
A video player module 145;
A music player module 146;
Browser module 147;
Calendar module 148;
Widget module 149, which includes weather widget 149-1, stock widget 149-2, calculator widget 149-3, alarm clock widget 149-4, dictionary widget 149-5, and other widgets obtained by the user, and Including a user-generated widget 149-6;
A widget generation module 150 for creating a user generated widget 149-6;
Search module 151;
A video and music player module 152, which combines a video player module 145 and a music player module 146;
・ Note module 153;
A map module 154; and / or an online video module 155.

  Other applications 136 stored in memory 102 include, for example, other word processing applications, JAVA enabled applications, encryption, digital rights management, voice recognition, and voice copying.

  In connection with the touch screen 112, display controller 156, contact module 130, graphics module 132, and text input module 134, the contact module 137 is used to manage an address book or contact list, which is an address Add name (s) to book, delete name (s) from address book, phone number (s), e-mail address (es), physical Associating a specific address (s) or other information with a name, associating a video with a name, categorizing and segmenting the name, initiating communication by phone 138, video conference 139, e-mail 140 or IM 141 Provide a phone number or email address to facilitate and / or facilitate Including the.

  In connection with the RF circuit 108, audio circuit 110, speaker 111, microphone 113, touch screen 112, display controller 156, contact module 130, graphics module 132, and text input module 134, the telephone module 138 corresponds to a telephone number. Enter a sequence of characters, access one or more phone numbers in the address book 137, change the phone numbers entered, dial each phone number, engage in a conversation, and disconnect or disconnect when the conversation is complete Used to hang up. As described above, wireless communication may use any of a plurality of communication standards, protocols, and technologies.

  RF circuit 108, audio circuit 110, speaker 111, microphone 113, touch screen 112, display controller 156, optical sensor 164, optical sensor controller 158, contact module 130, graphic module 132, text input module 134, contact list 137 , And in conjunction with the telephone module 138, the video conferencing module 139 is used to initiate, execute and end a video conference between the user and one or more other participants.

  In connection with the RF circuit 108, touch screen 112, display controller 156, contact module 130, graphics module 132, and text input module 134, the email client module 140 generates, sends, and receives emails. And used to manage. In connection with the video management module 144, the e-mail module 140 makes it very easy to generate and send e-mail along with still or video images taken by the camera module 143. Further, the text input module 134 functions in conjunction with the email client module 140 to facilitate the insertion of pictographs into the email.

  In connection with the RF circuit 108, touch screen 112, display controller 156, contact module 130, graphics module 132, and text input module 134, the instant messaging module 141 inputs a sequence of characters corresponding to the instant message, previously Change the entered character and send each instant message (for example, using the Short Message Service (SMS) or Multimedia Message Service (MMS) protocol protocols for phone-based instant messages, or Internet-based instant messages For messages (using XMPP, SIMPLE, or IMPS), receive and receive instant messages It was used to see the instant message. In some embodiments, instant messages sent and / or received include graphics, photos, audio files, video files, and / or other attachments supported by MMS and / or Enhanced Messaging Service (EMS). . As used herein, “instant messaging” refers to phone-based messages (eg, messages sent using SMS or MMS) and Internet-based messages (eg, sent using XMPP, SIMPLE, or IMPS). Message). Further, the text input module 134 functions in conjunction with the instant messaging module 141 to facilitate the insertion of pictographs into each instant message.

  In connection with the RF circuit 108, touch screen 112, display controller 156, contact module 130, graphics module 132, text input module 134, GPS module 135, map module 154, and music player module 146, the training support module 142 is a training module. Generate (e.g., have a time, distance and / or calorie burning target), communicate with a training sensor (sport equipment), receive training sensor data, calibrate the sensor used to monitor training, It is used to select and play music for training and to display, store and transmit training data.

  In connection with the touch screen 112, display controller 156, optical sensor 164, optical sensor controller 158, contact module 130, graphics module 132, and video management module 144, the camera module 143 is a still image or video (video stream). Are captured and stored in the memory 102, used to change the characteristics of the still image or video, or to delete the still image or video from the memory 102.

  In connection with the touch screen 112, display controller 156, contact module 130, graphics module 132, text input module 134, and camera module 143, the video management module 144 arranges and modifies still and / or video images, and so on. If not, it is used to manipulate, label, delete, present (eg, in a digital slide show or album), and store.

  In connection with the touch screen 112, display controller 156, contact module 130, graphics module 132, audio circuit 110 and speaker 111, the video player module 145 is connected to video (eg, to the touch screen or via an external port 124. (Displayed on an external display) used to display, present, or play.

  In connection with the touch screen 112, display system controller 156, contact module 130, graphics module 132, audio circuit 110, speaker 111, RF circuit 108, and browser module 147, the music player module 146 includes recorded music and one The user can download and play back other sound files stored in the above file format, for example, MP3 or AAC files. In some embodiments, the device 100 includes the functionality of an MP3 player, such as iPod (a trademark of Apple Computer).

  In conjunction with the RF circuit 108, touch screen 112, display system controller 156, contact module 130, graphics module 132, and text input module 134, the browser module 147 includes a web page or portion thereof, and attachments linked to the web page. And browsing the Internet, including searching, linking to, receiving and displaying it, and other files.

  In conjunction with the RF circuit 108, touch screen 112, display system controller 156, contact module 130, graphics module 132, text input module 134, email module 140, and browser module 147, the calendar module 148 can be used for calendars and calendars. Used to create, display, modify, and store relevant data (eg, calendar entries, to-do lists, etc.). Further, the text input module 134 functions in conjunction with the calendar module 148 to facilitate the insertion of pictographs into calendar entries, to-do lists, etc.

  In conjunction with the RF circuit 108, touch screen 112, display system controller 156, contact module 130, graphics module 132, text input module 134, and browser module 147, the widget module 149 is a mini-application that is downloaded and used by the user. (E.g., weather widget 149-1, stock widget 149-2, calculator widget 149-3, alarm clock widget 149-4, and dictionary widget 149-5) or a user-generated mini application (e.g. , User generated widget 149-6). In one embodiment, the widget includes an HTML (Hyper Text Markup Language) file, a CSS (Cascading Style Sheet) file, and a JavaScript file. In one embodiment, the widget includes an XML (Extensible Markup Language) file and a JavaScript file (eg, Yahoo! Widget).

  In conjunction with the RF circuit 108, touch screen 112, display system controller 156, contact module 130, graphics module 132, text input module 134, and browser module 147, the widget generation module 150 is used to generate a widget by the user. (Eg, converting a user-specified portion of a web page into a widget).

  In connection with touch screen 112, display system controller 156, contact module 130, graphics module 132, and text input module 134, search module 151 may include one or more search criteria (eg, one or more user-specified search terms). Is used to search memory 102 for text, music, sound, video, video, and / or other files that match

  In conjunction with touch screen 112, display controller 156, contact module 130, graphics module 132, and text input module 134, note module 153 is used to generate and manage notes, to-do lists, and the like. In addition, the text input module 134 functions in conjunction with the note module 153 to facilitate the insertion of pictographs into notes, to-do lists, and the like.

  In connection with RF circuit 108, touch screen 112, display system controller 156, contact module 130, graphics module 132, text input module 134, GPS module 135, and browser module 147, map module 154 is associated with maps and maps. It is used to receive, display, modify and store data (eg, driving directions, data about a particular location or nearby stores and other intersections, and other data based on location). Further, the text input module 134 functions in conjunction with the map module 154 to facilitate the insertion of pictographs into the map.

  Online in connection with touch screen 112, display system controller 156, contact module 130, graphics module 132, audio circuit 110, speaker 111, RF circuit 108, text input module 134, email client module 140, and browser module 147 The video module 155 allows the user to access, browse, receive (eg, by streaming and / or download), play (eg, on a touch screen or an external display connected via the external port 124), In addition, an e-mail is sent with a link to a specific online video, and other online videos are It can be managed in one or more file formats such as H.264. In one embodiment, the instant messaging module 141 is used instead of the email client module 140 to send a link to a particular online video. Additional description of online video applications can be found in US Provisional Patent Application No. 60 / 936,562, entitled “Portable Multifunction Device, Method, and Graphical User Interface for Playing Online Videos,” filed June 20, 2007, And US patent application Ser. No. 11 / 968,067 entitled “Portable Multifunction Device, Method, and Graphical User Interface for Playing Online Videos” filed on Dec. 31, 2007. Incorporated herein by reference.

  Each of the modules and applications described above corresponds to a set of instructions for performing one or more functions described above. These modules (i.e., sets of instructions) need not be embodied as separate software programs, procedures or modules, thus combining various subsets of these modules in various embodiments, otherwise, Can be rearranged. For example, video player module 145 can be combined with music player module 146 into a single module (eg, video and music player module 152 of FIG. 1B). In some embodiments, the memory 102 may store a subset of the modules and data structures described above. In addition, the memory 102 can store additional modules and data structures not described above.

  In some embodiments, the device 100 is a device in which a predefined set of functions is performed exclusively through the touch screen 112 and / or touchpad. Using the touch screen and / or touchpad as a primary input / control device for operation of the device 100 may reduce the number of physical input / control devices (push buttons, dials, etc.) in the device 100. it can.

  The default set of functions performed exclusively through the touch screen and / or touch pad includes navigation between user interfaces. In some embodiments, the touchpad navigates the device 100 from a user interface displayed on the device 100 to a main, home or root menu when the user touches. In such an embodiment, the touchpad is referred to as a “menu button”. In other embodiments, the menu button is a physical push button or other physical input / control device rather than a touchpad.

  FIG. 2 illustrates a portable multifunction device 100 having a touch screen 112 according to an embodiment. The touch screen displays one or more graphics within a user interface (UI) 200. In this and other embodiments, the user can select one or more graphics, for example, by touching or touching the graphics with one or more fingers 202 (not drawn to scale in the illustration). Can be selected. In some embodiments, the selection of one or more graphics occurs when the user disconnects contact with the one or more graphics. In some embodiments, the contact may include one or more taps that contact the device 100, one or more swipes (left to right, right to left, up and / or down directions), and / or finger rotation (right To left, left to right, up direction and / or down direction). In some embodiments, the graphic is not selected by accidental contact with the graphic. For example, when the gesture corresponding to the selection is a tap, the corresponding application is not selected in the swipe gesture that sweeps over the application icon.

  The device 100 also includes one or more physical buttons such as a “Home” or menu button 204. As described above, the menu button 204 is used to navigate to any application 136 in the set of applications executed on the device 100. Alternatively, in some embodiments, the menu button is embodied as a GUI soft key on the touch screen 112.

  In one embodiment, the device 100 includes a touch screen 112, a menu button 204, a push button 206 for turning the device on and off and locking the device, a volume adjustment button 208, a subscriber identity module ( SIM) card slot 210, headset jack 212, and docking / charging external port 124. Push button 206 is pressed and held in a pressed state for a predetermined time interval to turn the device on and off, and this button is pressed and released before the predetermined time interval elapses to release the device. Used to lock and / or unlock the device or initiate an unlock process. In another embodiment, the device 100 can also accept verbal input to activate or deactivate certain functions through the microphone 113.

FIG. 3 illustrates an example user interface for unlocking a portable electronic device according to an embodiment. In some embodiments, the user interface 300 includes the following elements, or a subset or superset thereof:
An unlock video 302 that moves with a finger gesture to unlock the device;
An arrow 304 giving a visual cue to the unlock gesture;
A channel 306 that provides additional cues for unlock gestures;
・ Time 308;
・ Day of the week 310;
• Date 312; and • Wallpaper image 314.

  In some embodiments, the device detects contact with the touch-sensitive display (eg, a user's finger touches or near unlocked image 302) while it is in a user interface locked state. The device moves the unlock video 302 based on the contact. The device transitions to the user interface unlocked state if the detected contact corresponds to a predefined gesture that moves the unlocked video across the channel 306. Conversely, the device keeps the user interface locked if the detected contact does not correspond to a predefined gesture. As described above, the process of unlocking a device using a gesture on a touch screen is described in US patent application Ser. No. 11 entitled “Unlocking A Device By Performing Gestures On An Unlock Image” filed on Dec. 23, 2005. / 322,549; and 11 / 322,550 entitled “Indication Of Progress Towards Satisfaction Of A User Input Condition” filed on December 23, 2005, which are hereby incorporated by reference Is used as is.

4A and 4B illustrate an exemplary user interface for a menu of applications in a portable multifunction device according to an embodiment. In some embodiments, user interface 400A includes the following elements, or a subset or superset thereof:
A signal strength indicator 402 for wireless communications, such as cellular and Wi-Fi signals;
・ Time 404;
A battery status indicator 406;
A tray 408 with icons for frequently used applications such as:
A phone 138 that includes an indicator 414 of the number of missed calls or voicemail messages;
An email client 140 that includes an indicator 410 of the number of unread emails;
Browser 147; and music player 146; and icons for other applications such as:
・ IM141;
・ Video management 144;
-Camera 143;
A video player 145;
・ Weather 149-1;
-Strain 149-2;
-Training support 142;
Calendar 148;
-Calculator 149-3;
Alarm clock 149-4;
A dictionary 149-5; and a user-generated widget 149-6.

In some embodiments, user interface 400B includes the following elements, or a subset or superset thereof:
-402, 404, 406, 141, 148, 144, 143, 149-3, 149-2, 149-1, 149-4, 410, 414, 138, 140 and 147 described above;
・ Map 154;
・ Note 153;
A setting 412 that gives access to the settings of the device 100 and its various applications 136, as described below;
A video and music player module 152, also referred to as iPod (trademark of Apple Computer) 152; and an online video module 155, also referred to as YouTube (trademark of Google) module 155.

  In some embodiments, UI 400A or 400B provides integrated access to both widget-based and non-widget-based applications. In some embodiments, all widgets are displayed on UI 400A or 400B, whether user generated or not. In other embodiments, activating the user generated widget 149-6 icon leads to a user generated widget or another UI containing an icon corresponding to the user generated widget.

  In one embodiment, the user is described, for example, in US patent application Ser. No. 11 / 459,602, entitled “Portable Electronic Device With Interface Reconfiguration Mode,” filed July 24, 2006, which is incorporated herein by reference in its entirety. The process can be used to re-arrange icons in UI 400A or 400B. For example, the user can use a finger gesture to put or remove application icons in the tray 408.

  FIGS. 5A-5G illustrate an example user interface for providing and organizing pictogram character keys on a portable electronic device 100 equipped with a touch-sensitive display 112 according to some embodiments. An outline of these figures is given here.

  FIG. 5A shows a user interface 500A in the instant messaging application 141. FIG. As described in this example, a character input area 5000 is provided that can be operated to display or display text character input and pictogram character input selected by the user. As mentioned above, text characters used in the description and claims refer to alphanumeric characters, sinographs, Japanese kanji or kana symbols, and / or other human written language characters, An emoji character refers to a stationary and animated pictogram that is not a text character in human writing languages. The user interface 500A is also provided with a keyboard display area 5005 including a plurality of pictogram category icons 5010. The pictogram category icon 5010 corresponds to a plurality of pictogram categories, such as those most recently and frequently used, people, nature, objects, travel, symbols, and / or seasonal pictogram categories. In the exemplary embodiment displayed on the UI 500A, the pictogram category icon 5010-1 corresponds to the most recently and frequently used, the pictogram category icon 5010-2 corresponds to a person, and the pictogram category icon 5010. -3 corresponds to nature, pictogram category icon 5010-4 corresponds to an object, pictogram category icon 5010-5 corresponds to travel, and pictogram category icon 5010-6 corresponds to a symbol.

  UI 500A also includes a normative pictogram character key 5015 in keyboard display area 5005 for each pictogram category to be displayed. In the normative UI 500A, the displayed emoji character keys (ie, 5015-1, 5015-2 and 5015-3) correspond to the emoji category “most recently and frequently used”. This is because the pictogram category icon 5010-1, the most recently and frequently used one, is currently selected. The currently selected pictogram category icon 5010 is typically visually different from the unselected pictogram category icon, for example by highlighting or changing its background color relative to the unselected pictogram category icon. Differentiated.

  In one embodiment, the emoji character keys of the emoji category corresponding to the emoji category icon 5010-1, the most recent and frequently used emoji characters, analyze the up-to-date and frequency of use of the emoji characters used. Based on the algorithm to be selected for display. This algorithm generates a history list of pictogram character usage. Each entry in the history list is given a relative weight based on the position of the entry in the history list. For example, the most recently used emoji character gets the highest weight, the second most recently used emoji character gets a lower weight than the most recently used emoji character, and the third most recently used emoji character Obtains a smaller weight than the second most recently used pictogram character, and so on. Then, for each individual pictogram character, the overall score is determined by combining (eg, summing) the relative weights of that pictogram character. For example, in the case of a pictogram character occupying the second, tenth and twentieth positions in the history list, the overall score of the pictogram character is given to the second, tenth and twentieth positions in the history list. It is the sum of relative weights. The emoji character with the highest overall score is then displayed in the emoji category “most recently and frequently used” (eg, in order of decreasing score).

  In this exemplary embodiment, UI 500A also includes a keyboard selection icon 5020. In some embodiments, the keyboard selection icon 5020 is displayed simultaneously with multiple pictogram category icons 5010, for example, in a predetermined area 5009 along the bottom of the touch screen display. Keyboard select icon 5020 is used in some embodiments to replace the display of pictogram character keys with the display of a text character input keyboard (eg, 5055 in FIG. 5G) in keyboard display area 5005. UI 500A also includes a second icon 5025, which initiates transmission of an electronic message entered in area 5000 when activated (eg, by a finger tap gesture). The UI 500A also includes a delete icon 5011 that deletes one or more characters in the input area 5000 when activated (eg, by a finger tap gesture).

  UI 500B (FIG. 5B) includes a subset sequence mark icon 5040 for each emoji category, here a person who is instructed to be visible when selected by highlighting emoji category icon 5010-2. A keyboard display area 5005 is shown. Subset sequence mark icon 5040 provides information regarding the number of subsets in the currently displayed pictogram category and the position of the displayed subset in the sequence of pictogram character key subsets in keyboard display area 5005. Here, the subset sequence mark icon 5040 indicates that there are six subsets of pictogram character keys in the human pictogram category and that the first subset of the human pictogram category is currently displayed. For another example, see the description below for the subset sequence mark icon 6000 of FIGS. 6A-6C.

  UI 500C, UI 500D, UI 500E, and UI 500F, shown in FIGS. 5C-5F, respectively, show user interfaces with other normative emoji categories and an emoji character keyboard.

  A UI 500G (FIG. 5G) shows a text character input keyboard 5055 in the keyboard display area 5005. In some embodiments, the keyboard 5055 includes a text character input category icon 5060, which when associated with a numeric key and other text character input keys when activated (eg, by a finger tap gesture on the icon). Display the text character keyboard. In some embodiments, the keyboard 5055 includes a hybrid purpose key 5062 that operates to input a space character and also operates to confirm automatic completion and / or automatic correction upon text character input. For example, when inputting Japanese text, in some embodiments, speech text can be used. When the device evaluates a text character entered by the user, one or more recommended word completion candidates are presented for the text already entered. For example, when the user types the character “ra”, “rain” is recommended as a completion candidate. Thereafter, while the “rain” completion candidate is displayed (not shown), the hybrid purpose key 5062 is activated (eg, by a finger tap gesture on the key 5062) and the text “rain” is intended for input. The apparatus inserts the character string “rain” into the character input area 5000. In some embodiments, a sinograph or Japanese kana character may be presented as a completion candidate. In some embodiments, the keyboard 5055 includes a hybrid purpose key 5063 that operates to enter a newline character and can also operate to confirm auto-completion for the action. For example, if the user types the character “goo”, the device recommends proceeding to the Google search engine as an action completion candidate. Thereafter, the hybrid purpose key 5063 is activated (eg, by a finger tap gesture on the key 5063), causing the device to open the browsing module 147 to display the Google search engine web page. The keyboard 5055 also includes a keyboard selection icon 5020. In addition, the keyboard 5055 includes text character input keys 5065, such as 5065-1 (Q), 5065-2 (A), and 5065-3 (S), among others.

  6A-6E illustrate an example user interface for navigating between emoticon character subsets of a particular emoji category and inputting certain emoji characters in portable electronic device 100 equipped with touch-sensitive display 112 according to some embodiments. . An outline of these figures is given here.

  UI 600A (FIG. 6A) is a subset sequence mark for the currently displayed pictogram category, here, the natural that is instructed to be visible when selected by highlighting the pictogram category icon 5010-3. Icons 6000-1, 6000-2 and 6000-3 are shown. The subset sequence mark icons 6000-1, 6000-2 and 6000-3 indicate the number of subsets in the currently displayed pictogram category and the displayed subsets in the sequence of the subset of pictogram character keys in the keyboard display area 5005. Give information about location and. For example, in the UI 600A, the number of mark icons 6000 indicates that there are three subsets in this pictogram category (natural). Further, visually highlighting icon 6000-1 for icons 6000-2 and 6000-3 indicates that the first subset 6015 of this category is displayed.

  In some embodiments, upon detecting a gesture (eg, gesture 6010 (FIG. 6A), horizontal or substantially horizontal finger swipe gesture) in the keyboard display area 5005, the first subset 6015 of emoji character keys is displayed in the keyboard display area 5005. Are replaced with the second subset 6020 (FIG. 6B) of pictogram character keys. Further, in some embodiments, the information provided by the subset sequence mark icon is updated to reflect that the displayed first subset 6015 of emoji character keys has been replaced with the second subset 6020 of emoji character keys. . In this example, the subset sequence mark icon 6000-1 is highlighted in the UI 600A. This is because the first subset 6015 of pictogram character keys in the nature category is displayed. When the first subset 6015 of pictogram character keys is replaced with the second subset 6020 (FIG. 6B) of pictogram character keys, the subset sequence mark icon 6000-2 is highlighted. In some embodiments, a subset sequence mark icon corresponding to a currently displayed subset of emoji character keys is highlighted or otherwise visually distinguished while a subset of displayed emoji character keys. Highlighting or otherwise visually distinguishing another subset sequence mark icon corresponding to.

  UI 600B (FIG. 6B), which has already been described to some extent, illustrates the detection of another gesture in keyboard display area 5005 (eg, gesture 6025, a horizontal or substantially horizontal finger swipe gesture). In response to detection of gesture 6025, the second subset 6020 of pictogram character keys is replaced with the display of the third subset 6040 of pictogram character keys (FIG. 6C) in the keyboard display area 5005. Correspondingly, as shown in UI 600C (FIG. 6C), subset sequence mark icon 6000-3 is highlighted instead of subset sequence mark icon 6000-2.

  UI 600C (FIG. 6C) illustrates the detection of another gesture in keyboard display area 5005 (eg, gesture 6030, a horizontal or substantially horizontal finger swipe gesture that is substantially opposite to the direction of gesture 6025). In response to detection of gesture 6030, the third subset 6040 of pictogram character keys is replaced with the display of the second subset 6020 (FIG. 6B) of pictogram character keys in the keyboard display area 5005. Correspondingly, subset sequence mark icon 6000-2 is highlighted instead of subset sequence mark icon 6000-3 (FIG. 6B).

  UI 600C (FIG. 6C) also shows a user gesture 6045 (eg, a finger tap gesture) on pictograph key 6050. UI 600D (FIG. 6D) shows a key selection icon 6060 that is displayed while a finger touch of user gesture 6045 is still on the emoji key 6050 (shown in FIG. 6C). The key selection icon 6060 indicates a key that is selected when the user's finger is lifted (stops contact) from the touch screen display at the current position of finger contact on the touch screen display. In an embodiment, the selected pictograph character, in this case, the pictograph character image 6061 corresponding to the pictograph key 6050 is displayed in the key selection icon 6060.

  UI 600E (FIG. 6E) shows a pictographic character key image 6070 corresponding to pictographic key 6050 displayed in character input area 5000 in response to detection of a selected pictographic character, in this case gesture 6045.

  7A-7E illustrate an exemplary user interface for entering emoticon characters and sending an electronic message including the emoticon characters in portable electronic device 100 equipped with touch-sensitive display 112 according to some embodiments. An outline of these figures is given here.

  UI 700A to UI 700C indicate user selection of pictogram characters in the same manner as described above. In UI 700A, pictogram key 7000 is selected in response to detection of user gesture 7001 (eg, finger tap gesture). UI 700 </ b> B (FIG. 7B) shows a key selection icon 7010 displayed while the finger used for gesture 7001 is still in contact with the area corresponding to pictogram key 7000.

  UI 700C (FIG. 7C) shows an image 7025 of the selected pictogram character displayed in character input area 5000 in response to detection of gesture 7001. In response to detection of a user gesture (eg, finger tap gesture 7026 of FIG. 7D) in send icon 5025 (FIG. 7D), the text character and pictogram character entered in area 5000 are different parties (eg, FIGS. 7D and 7E). John appleseed) as an electronic message.

  In this example, pictogram characters are input and transmitted in an instant message. However, those skilled in the art will appreciate that emoji characters may be entered and transmitted in other forms of electronic messages and documents, such as e-mail 140.

  8A-8C illustrate an exemplary user interface for reconfiguring the emoji character keys of a software keyboard in portable electronic device 100 equipped with touch sensitive display 112 according to some embodiments. This process is simple and efficient for the user to re-arrange the emoji character keys on the portable electronic device (eg, to optimize the location of the emoji character keys for the user to input). Give the mechanism.

  UI 800A (FIG. 8A) shows a keyboard display area 5005 with a plurality of pictogram character keys 8002 including pictogram keys 8001-1 and 8001-2. The user interface is configured to detect a default user action (eg, pressing the pictogram character key 8001 with a finger and holding the finger contact 8005 for a predetermined time or more) to initiate a default user interface reconfiguration process.

  In one embodiment, the UI 800B (FIG. 8B), when detecting a predetermined user action 8005, the pictogram character key 8001 in the keyboard display area 5005 is substantially centered on the average position of each pictogram character key. It indicates that the region vibrates (for example, vibrations 8020-1, 8020-2, 8020-3, etc.). These vibrations give a simple visual indication that the device is currently in interface reconfiguration mode and that the vibration icons are rearranged.

  Also, as shown in UI 800B, in some embodiments, the device may provide a touch point 8050 with the touch screen display 112 at a first position corresponding to the first pictogram character key 8001-2 of the plurality of pictogram character keys 8002. It is detected that the contact point 8050 is moved to the second position 8025 on the touch screen display 112. In some embodiments, the user interface indicates that the first emoji character key 8001-2 is moved on the touch screen display 112 to the second position 8025 based on the detected movement of the contact point 8050.

  In UI 800C (FIG. 8C), the user interface is responsive to detecting a second default user action (eg, detecting activation of the home button 204) to end the default user interface reconfiguration process. The position of the first pictogram character key 8001-2 is fixed to the second position. In the example shown here, the net result is that the pictogram character keys 8001-1 and 8001-2 have been swapped in the UI 800C when compared to the UI 800A.

  9A-9H illustrate an exemplary user interface for responding to messages with pictograms while the portable electronic device 100 equipped with the touch-sensitive display 112 is in user interface lock mode, according to an embodiment. An outline of these figures is given here.

  UI 900 (FIG. 9A) shows an example user interface in user interface lock mode. See also the description of FIG. 3 above. In the user interface unlock mode, the device 100 is in its normal operating state and detects and responds to user input corresponding to interaction with the user interface. The device 100 in the unlock mode will be described as the unlocked device 100. The unlocked state device 100 detects and responds to user input navigating between user interfaces, data entries, and functional activation or deactivation. In contrast to the user interface unlock mode, in the user interface lock mode, only a limited subset of control icons are displayed and the device 100 responds only to a limited set of finger gestures. For example, only the unlock video 302, the arrow 304, and the channel 306 are displayed together with the wallpaper video 314 (UI 900A). In addition, as described below, a small number of predefined responses including pictogram characters are displayed and / or a small number of gestures are used to respond to messages received while in user interface lock mode.

  UI 900 (FIG. 9B) shows receiving a message arriving from a party 2210 (here “Jane Doe”) to the user of the portable electronic device while the user interface is in user interface lock mode. At least a portion of the message 2212 to be displayed to the user on the touch screen display 112.

  UI 900C (FIG. 9C) allows the user interface to detect one or more predefined finger gestures (eg, tap gestures 9100-1, 9100-2, and swipe gesture 9100-3) corresponding to the emoji character on touch screen display 112. It shows that In some embodiments, the one or more default finger gestures and pictograms are designated by designating the emoticon character with one or more default finger gestures, as described below with reference to FIGS. 10A-10E and FIG. A correspondence with the character is formed. In some embodiments, the user interface also detects a default user action (eg, a swipe gesture 2213) to initiate transmission of a response to the incoming message 2212 and then transmits a response to the incoming message 2212. The response includes pictographic characters corresponding to one or more predefined finger gestures (eg, 9100-1, 9100-2, and 9100-3) detected on the touch screen display 112.

  UI 900D (FIG. 9D) illustrates that in some embodiments, the user interface displays a plurality of default responses 9110 for incoming messages 2212, such as 9110-1, 9110-2 and 9110. -3) includes a plurality of pictogram character keys (for example, 9111-1, 911-2, and 9111-3). In the UI 900D, each default response 9110 includes a single pictogram character 9111. More specifically, other predefined responses 9110 include two or more emoji characters, and certain predefined responses 9110 may not include emoji characters (not shown).

  UI 900E (FIG. 9E) illustrates detection of a user action (eg, tap gesture 9120) to select a default response (eg, response 9110-1). The user can quickly make a selection from a list of predefined responses containing pictogram characters and send the selected response without unlocking the device. In certain embodiments, upon detection of selection of default response 9110, transmission of the default response is initiated without detecting further action by the user. In some embodiments, after detecting selection of a default response 9110, further default user actions to initiate transmission of the response are detected, for example, activation of the home button 204 is detected, and a “Send” button ( Activation (not shown) is detected, or a sliding finger gesture 2213 that apparently moves the unlocked image 302 across the channel 306 is detected (in this case, the message “slide to see” in the channel 306 is shown in FIG. Message “slide to send response” or similar message).

  UI 900F (FIG. 9F) shows the user interface in a messaging application after the device is subsequently unlocked, which shows a default response 9110-1 that was sent while the device was in the locked state. .

  UI900G and UI900H (FIGS. 9G and 9H, respectively) are similar to UI900E and UI900F (FIGS. 9E and 9F, respectively), but the default responses 9110-4, 9110-5, and 9110-6 are text characters and pictogram characters. It is composed only of pictogram characters, not a combination of Similarly, the default response transmission 9110-4 in FIG. 9H corresponds to the default response 9110-4 in FIG.

  FIGS. 10A-10E illustrate an example user interface for assigning user-defined gestures to pictogram characters in portable electronic device 100 equipped with touch-sensitive display 112 according to some embodiments. An outline of these figures is given here.

  UI 1000A (FIG. 10A) shows an example user interface for recording user-defined gestures. The UI 1000A includes a record icon 1001, an association icon 1010, a storage icon 1015, and a cancel icon 1016. When a user gesture 1020 is detected in the recording icon 1001, the user interface enters a gesture recording mode. UI 1000B (FIG. 10B) shows user-defined gesture 1030 detected on touch screen 112 while in gesture recording mode. In some embodiments, in response to detecting a gesture in association icon 1010 (eg, finger tap gesture 1040 in FIG. 10C), a process is initiated to associate a representation of user-defined gesture 1030 with an emoji character, which is an emoji character. Including displaying one or more user interfaces (eg, UI 1000D of FIG. 10D) with character keys. In some embodiments, the user interface with emoji character keys is similar to UI 500A-500F (FIGS. 5A-F), but the user interface now selects the emoji character for input into an electronic message or document. Rather than being used to associate pictograms with user-defined gestures 1030. In response to detecting a gesture on the emoji character key (eg, a finger tap gesture 1070 on the emoji key 1060 in FIG. 10D), the representation of the user-defined gesture 1030 is associated with the emoji character corresponding to the emoji character key 1060 ( (Alternatively, the user confirmation is put on hold). In some embodiments, the confirmation UI (eg, UI 1000E of FIG. 10E) may include an emoji character 1075 associated with the user-defined gesture 1030 when activation of the storage icon 1015 is detected (eg, with the finger tap gesture 1080 of FIG. 10E). Is displayed. In response to activation of the cancel icon 1016 (eg, by a finger tap gesture), the process of designating a user-defined gesture as an emoji character is terminated.

  FIGS. 11A-11B illustrate an example user interface for using an emoji character as a position indicator on a map via an electronic message according to an embodiment. An outline of these figures is given here.

  UI 1100A (FIG. 11A) shows an electronic message 2216 displayed on instant messaging application 141. The electronic message includes instructions for starting the geographic map application, location data representing a location on the geographic map, and pictographic data representing a pictographic character. The pictographic data is associated with the position data. In response to detecting a predefined user action on the message (eg, a finger tap gesture on the link 2220 to Lake Tahoe), the device responds in the geographic map application 154 with a geographic map (eg, map 1115 in FIG. 11B). And the apparatus displays a pictographic character 1120 (FIG. 11B) at a position on the geographic map 1115 corresponding to the position data.

  The processes shown in FIGS. 12A-12F and FIGS. 13-17 are described below using some of the user interfaces in FIGS. 5A-11C.

  12A-12F are flowcharts illustrating a method of operating an emoji character keyboard in a portable electronic device equipped with a touch sensitive display according to some embodiments. The method 1200 is performed in a portable electronic device (eg, portable multifunction device 100) having a touch screen display. This method provides a more efficient way to organize and select emoji characters in portable electronic devices, thereby conserving power, extending the time between battery charges, and at the device Increase user satisfaction.

  The portable electronic device simultaneously displays a character input area (eg, area 5000 in FIG. 6A), a keyboard display area (eg, area 5005 in FIG. 6A), and a plurality of pictogram category icons (eg, icon 5010 in FIG. 6A). (1202). The character input area 5005 can be operated to display or display a text character input and a pictogram character input selected by the user.

  The pictogram category icons correspond to a plurality of pictogram categories (for example, 5010-1 to 5010-6 in FIG. 6A). In some embodiments, a plurality of pictogram category icons are used for recently and frequently used pictogram category icons corresponding to a plurality of pictogram characters recently and frequently selected by the user (eg, the recent and frequently used pictogram category icons of FIG. 6A). (1204). In some embodiments, the plurality of pictogram category icons include seasonal pictogram character icons corresponding to a plurality of pictogram characters associated with a particular season (eg, spring, summer, autumn or winter) (1206). In some embodiments, the plurality of pictogram category icons includes holiday pictogram category icons corresponding to a plurality of pictogram characters associated with a plurality of holidays (eg, Christmas or New Year) (1208).

  In some embodiments, a plurality of pictogram category icons 5010 are displayed (1210) in the keyboard display area 5005 (FIG. 6A).

  The apparatus detects a gesture in each pictogram category icon 5010 of the plurality of pictogram category icons (1212). In some embodiments, the gesture is a finger tap gesture (eg, tap gesture 5007 in category icon 5010-3 of FIG. 5A).

  In response to detecting a gesture in each pictogram category icon, the device may include a first subset of pictogram character keys for each pictogram category in keyboard display area 5005 (eg, subset 6015 in FIG. 6A) and a plurality of subsets for each pictogram category. A sequence mark icon (for example, the subset sequence mark icon 6000 in FIG. 6A) is simultaneously displayed (1214).

  Each pictogram category includes multiple subsets of pictogram character keys configured to be displayed separately as a sequence of subsets of pictogram character keys in the keyboard display area. For example, the natural pictogram categories of FIGS. 6A-6C are three subsets of pictogram character keys configured to be displayed separately as a sequence of subsets of pictogram character keys in the keyboard display area 5005, ie, the first subset 6015 of FIG. 6A. , The second subset 6020 of FIG. 6B, and the third subset 6040 of FIG. 6C.

  The subset sequence mark icon provides information about the number of subsets in each pictogram category and the position of the displayed subset in the pictogram character key subset sequence within the keyboard display area. For example, in FIGS. 6A-6C, subset sequence mark icons 6000-1, 6000-2 and 6000-3 indicate that there are three subsets of pictogram character keys in natural pictogram category 5010-3. Further, icon 6000-1 is highlighted when the first subset 6015 is displayed (FIG. 6A) and icon 6000-2 is highlighted when the second subset 6020 is displayed (FIG. 6B). ), And icon 6000-3 is highlighted when the third subset 6040 is displayed (FIG. 6C). In some embodiments, the subset sequence mark icon has a simple geometric shape such as a circle, square, rectangle or star.

  In some embodiments, the emoji character keys in at least one emoji category are ordered based on the emoji character usage data (1216). In some embodiments, the emoji character usage data includes emoji character usage data from a software application running on the portable electronic device (1218). In some embodiments, the emoji character usage data includes (2020) emoji character usage data received (eg, downloaded) from a portable electronic device service provider. In some embodiments, the emoji character keys in at least one emoji category are ordered based on the emoji character popularity data (1222).

  In some embodiments, a subset sequence mark icon is displayed 1224 in the keyboard display area (eg, subset sequence mark icon 6000 is displayed in area 5005, FIG. 6A).

  While displaying the first subset of pictogram character keys for each pictogram category in the keyboard display area, the device detects a gesture in the keyboard display area (1226). For example, the device detects a horizontal or substantially horizontal finger swipe gesture 6010 in area 5005 (FIG. 6A).

  In response to detecting a gesture in the keyboard display area, the device replaces the display of the first subset of pictogram character keys of each pictogram category with the display of the second subset of pictogram character keys of each pictogram category in the keyboard display area; and The information provided by the subset sequence mark icon is updated to reflect the replacement of the pictograph character key with the displayed first subset by the second subset of pictograph character keys (1228). For example, in response to detecting a horizontal finger swipe gesture 6010 in the keyboard display area 5005, the device displays a display of the first subset 6015 of the pictogram character keys (FIG. 6A) of the natural pictogram category in the natural pictogram category in the keyboard display area 5005. Replace the display of the second subset 6020 of pictogram character keys of FIG. 6B and replace the information provided by the subset sequence mark icon 6000 with the first subset 6015 of pictogram character keys displayed by the second subset 6020 of pictogram character keys. Update to reflect the replacement. Updating the information provided by the subset sequence mark icon highlights or visually distinguishes the subset sequence mark icon corresponding to the second subset of pictogram character keys, while the first subset of pictogram character keys. Stop highlighting or visually distinguishing another corresponding subset sequence mark icon (eg, highlight icon 6000-2 while highlighting icon 6000-1 in FIG. 6B) ), But is not limited to this.

  In some embodiments, replacing the display of the first subset of pictogram character keys of each pictogram category with the display of the second subset of pictogram character keys of each pictogram category in the keyboard display area moves away from the touch screen display. Displaying an animation of a first subset of emoji character keys (1230). For example, between FIGS. 6A and 6B, an animation of subset 6015 that deviates from touch screen display 112 is displayed.

  In some embodiments, replacing the display of the first subset of pictogram character keys of each pictogram category with the display of the second subset of pictogram character keys of each pictogram category in the keyboard display area is a pictogram character moving to a touch screen display. Displaying an animation of the second subset of keys (1232). For example, an animation of subset 6020 moving to touch screen display 112 is displayed between FIGS. 6A and 6B.

  In some embodiments, replacing the display of the first subset of pictogram character keys of each pictogram category with the display of the second subset of pictogram character keys of each pictogram category in the keyboard display area moves away from the touch screen display. Displaying an animation of the first subset of emoji character keys and displaying an animation of the second subset of emoji character keys moving to the touch screen display (1234). For example, between FIGS. 6A and 6B, animations of subset 6015 moving away from touch screen display 112 and subset 6020 moving to touch screen display 112 are displayed.

  In some embodiments, the device detects a gesture in the keyboard display area while displaying a second subset of the emoji character keys for each emoji category in the keyboard display area. For example, the device detects a horizontal or substantially horizontal finger swipe gesture 6025 in area 5005 while displaying subset 6020 (FIG. 6B). In response to detecting a gesture (eg, gesture 6025) in the keyboard display area, the device displays a second subset of the pictogram character keys of each pictogram category and the first of the pictogram character keys of each pictogram category in the keyboard display area. Replace with the display of 3 subsets (1236). Also, in response to detecting a gesture (eg, gesture 6025) in the keyboard display area, the device displays the information provided by the subset sequence mark icon for the pictogram character key displayed by the third subset of pictogram character keys. Update to reflect replacement with the second subset (1236). For example, in response to detecting a horizontal finger swipe gesture 6025 in the keyboard display area 5000, the device naturally displays in the keyboard display area 5005 the display of the second subset 6020 of the emoji character keys of the natural emoji category (FIG. 6B). The display is replaced with the display of the third subset 6040 of the pictogram character keys of the pictogram category (FIG. 6C), and the information provided by the subset sequence mark icon 6000 is replaced with the second displayed pictogram character key by the third subset 6040 of the pictogram character keys. Update to reflect replacement with subset 6020. Updating the information highlights or visually distinguishes the subset sequence mark icon corresponding to the third subset of emoji character keys, while another subset sequence mark icon corresponding to the second subset of emoji character keys. Including, but not limited to, highlighting or visual differentiation (eg, in FIG. 6C, highlighting icon 6000-3 while highlighting icon 6000-2 is stopped). Not.

  In some embodiments, the gesture detected while displaying the second subset of emoji character keys is a first direction swipe gesture (eg, a right-to-left swipe gesture 6025, FIG. 6B). Responsive to the device detecting a second swipe gesture (eg, left-to-right swipe gesture 6030, FIG. 6C) on the touch screen display in the keyboard display area in a direction substantially opposite to the first direction. The device replaces the display of the third subset of the pictogram character keys of each pictogram category with the display of the second subset of the pictogram character keys of each pictogram category in the keyboard display area, and the information provided by the subset sequence mark icon, Update to reflect replacement of the displayed third subset of pictogram character keys with the second subset of pictogram character keys (eg, highlight the subset sequence mark icon corresponding to the second subset of pictogram character keys) While distinguish Luke or visual, to stop the differentiation of highlighting or visual on another subset sequence mark icon corresponding to a third subset of pictograms character key) (1238). For example, in response to detecting a swipe gesture 6030 (FIG. 6C) for the touch screen display in the keyboard display area 5005 in a direction substantially opposite to the direction of the gesture 6025 (FIG. 6B), the device The display of the third subset of the pictogram character key 6040 of the category is replaced with the display of the second subset of the pictogram character key 6020 of the natural pictogram category in the keyboard display area 5005, and the information provided by the subset sequence mark icon 6000 is replaced with the pictogram character key. Updated to reflect the replacement of the displayed emoticon character key with the displayed third subset by the second subset of e.g. (e.g., the subset corresponding to the second subset 6020 of emoji character keys) Sequence while distinguishing mark icon 6000-2 whether or visual to highlight, to stop the differentiation of highlighting or visual subset sequence mark icon 6003 corresponding to the third subset 6040 of the pictograph character keys).

  In some embodiments, the device displays a keyboard selection icon (eg, icon 5020, FIG. 6B). In some embodiments, the keyboard selection icon 5020 displays multiple pictogram category icons 5010 simultaneously (eg, in a predetermined area 5009 along the bottom of the touch screen display). The device detects a gesture on the keyboard selection icon (eg, finger tap gesture 6055 of FIG. 6B). After detecting the gesture on the keyboard selection icon, the apparatus replaces the display of the second subset of the pictogram character keys of each pictogram category with the display of the text character input keyboard in the keyboard display area (1240). For example, after detecting the tap gesture 6055 on the keyboard selection icon 5020, the device displays the second subset 6020 of the pictogram character keys (FIG. 6B) in the natural pictogram category as text characters in the keyboard display area 5055 (FIG. 5G). Replace with the display of the input keyboard 5055. In some embodiments, the second subset 6020 of emoji character keys is replaced with a text character input keyboard 5055 in response to detecting gesture 6055. In some embodiments, one or more intermediate text character input keyboard selection UIs are selected before a particular text character input keyboard (eg, an alphanumeric keyboard, kana keyboard, or kanji keyboard) is selected and displayed in area 5005. Is shown.

  In some embodiments, the device receives one or more words entered by a user with a text character input keyboard (eg, keyboard 5055 of FIG. 5G). The apparatus displays one or more words input to the character input area 5000 by the user. In response to receiving one or more words entered by the user with the text character input keyboard, the device determines a recommended emoji character based at least in part on the one or more words entered by the user, and the The recommended pictogram character is displayed on the touch screen display (1242). The determination of the recommended pictogram character is based on the word input by the user, not the punctuation marks input by the user. For example, when the user types the word “rain”, an umbrella and / or raindrop pictogram character is recommended. Similarly, if the user types the word “fly”, an aircraft pictogram character is recommended. In some embodiments, a single emoji character is recommended. In other embodiments, a plurality of pictogram characters are recommended. In some embodiments, the device detects a gesture for a touch screen display (eg, a tap gesture for a space bar of a text character input keyboard or a recommended gesture character tap character). In response to detecting the gesture, the apparatus inserts and displays a recommended pictogram character after one or more words input to the character input area by the user (1244).

In some embodiments, the device displays a keyboard selection icon 5020 while the text character input keyboard 5055 is displayed in the keyboard display area 5055 (FIG. 5G). Next, after detecting a gesture on the keyboard selection icon 5020, for example, a finger tap gesture, the apparatus displays the text character input keyboard 5055 (FIG. 6B) with the pictogram character keys (FIG. 6B) of each pictogram category in the keyboard display area. 6B) and the simultaneous display of the second subset 6020 and the corresponding subset sequence mark icon 6000 (1246). In some embodiments, the text character input keyboard 5055 is replaced with a second subset of emoji character keys in response to detecting a gesture. In some embodiments, one or more intermediate pictogram keyboard selection UIs are shown before a particular subset of pictogram character keys are selected and displayed in area 5005. In some embodiments, in response to detecting a gesture on the keyboard selection icon 5020, the device toggles between the display of the text character input keyboard and the most recently displayed subset of emoji character keys.

  In some embodiments, the apparatus detects a gesture for each pictogram character key in the second subset of pictogram character keys. In response to detecting a gesture for each pictogram character key in the second subset of pictogram character keys, the device inserts and displays the corresponding pictogram character in the character input key (1248). For example, in response to detecting a tap gesture for a pictograph character key in the second subset 6020 of pictograph character keys, the apparatus inserts and displays a corresponding pictograph character in the character input area 5000. 6C-6E illustrate similar insertion and display of pictogram characters from the third subset 6040 of pictogram characters to the input area 5000. FIG. In response to detecting a gesture for an additional pictogram character key, an additional pictogram character may be inserted into the pictogram input area 5000 and displayed.

  In some embodiments, the device detects a default user action to initiate a default user interface reconfiguration process (eg, pressing an emoji character key with a finger and holding finger contact for longer than a predetermined time). In response to detecting the predetermined user action, the device changes the position of two or more emoji character keys in the second subset of emoji character keys. This change includes changing the position of each pictogram character of two or more pictogram character keys around each average position different from each average position of other pictogram character keys of two or more pictogram character keys ( 1249). For example, in response to detecting a finger touch on an emoji character key 8001 (FIG. 8A) that is longer than a predetermined time (eg, 1 second), the device may detect a subset 8002 of the emoji character keys (FIG. 8A, which is shown in FIG. 6B). Change the positions of all pictogram character keys in the sub-set 6020).

  In one embodiment, each pictogram character key of two or more pictogram character keys vibrates in a region substantially centered at each average position of each pictogram character key (shown in FIG. 8B) (1250).

  In some embodiments, the device detects that the user forms a contact point with the touch screen display at a first position corresponding to the first emoji character key in the second subset of emoji character keys and on the touch screen display. The movement of the contact point with respect to the second position is also detected. In response to these two detections, the device displays the movement of the first pictogram character key relative to the second position on the touch screen display based on the detected touch point movement (1252). For example, it is detected that the user forms a contact point 8050 (FIG. 8B) with the touch screen display at a position corresponding to the emoji character key 8001-2, and the contact point 8050 with respect to the second position 825 on the touch screen display. In response to detecting the movement, the apparatus displays the movement of the pictogram character key 8001-2 relative to the second position 825 on the touch screen display based on the detected touch point movement.

  In some embodiments, the device is in a second position in a second position in response to detecting a second default user action (eg, detecting activation of the home button 204) that terminates the default user interface reconfiguration process. The position of one pictogram character key (for example, 8000-2 in FIG. 8C) is fixed (1254).

  FIG. 13 is a flowchart illustrating a method for responding to a message with a pictogram while the portable electronic device 100 equipped with the touch-sensitive display 112 is in a user interface lock mode according to an embodiment. This method 1300 provides a simple and intuitive way for a user to quickly respond to incoming messages while the device is in a locked state, thereby conserving power and at the time of battery charging and battery charging. Extends the period of time.

  While the portable electronic device 100 is in the user interface lock mode (1301), operations 1302-1314 are performed. The device receives an incoming message from the party to the user of the portable electronic device while the touch screen display is in a power saving state (1302). In the power save state, the touch screen is blank (nothing is displayed) and is not touch sensitive.

  In response to receiving the incoming message, the device powers on the touch screen display and displays at least a portion of the incoming message from the party to the user (eg, message portion 2212 of FIG. 9B) on the touch screen display. (1304). The device detects (1306) one or more predefined finger gestures (eg, gestures 9100-1, 9100-2, and 9100-3 in FIG. 9C) on the touch screen display corresponding to the pictogram character. Does the device detect a default user action to initiate transmission of a response to an incoming message (eg, detect activation of home button 204 or activation of a “send” button (not shown)) Or the swipe gesture 2213 of FIG. 9C is detected) (1308). In response to detecting a predefined user action to initiate transmission of a response to an incoming message, the device transmits a response to the incoming message (1310). The response includes an emoji character corresponding to one or more default finger gestures detected on the touch screen display. For example, the default finger gestures 9100-1, 9100-2, and 9100-3 in FIG. 9C correspond to smiley pictogram characters. One method for associating an emoji with one or more predefined finger gestures is described below with reference to FIGS. 10A-10E.

  In some embodiments, in response to detecting a default user action to initiate transmission of a response to an incoming message, the device returns the touch screen display to a power save state (1312). In some embodiments, the device returns (1314) the touch screen display to a power save state within a predetermined time (eg, 5 to 15 seconds) after a response to the incoming message is sent.

  FIG. 14 is a flowchart illustrating another method for responding to a message with a pictogram while the portable electronic device 100 equipped with the touch-sensitive display 112 according to an embodiment is in a user interface lock mode. This method 1400 provides a simple and intuitive way for a user to quickly respond to incoming messages while the device is locked, thereby conserving power and at the time of battery charging and battery charging. It extends the period between time.

  While the portable electronic device 100 is in the user interface lock mode (1401), operations 1402-1414 are performed. The device receives an incoming message from the party to the user of the portable electronic device while the touch screen display is in a power saving state (1402). In the power save state, the touch screen is blank (nothing is displayed) and is not touch sensitive.

  In response to receiving the incoming message, the device powers on the touch screen display and displays at least a portion of the incoming message from the party to the user (eg, message portion 2212 of FIG. 9D) on the touch screen display. And a plurality of predefined responses to the incoming message (eg, response 9110 of FIG. 9D or FIG. 9G) are displayed (1404). The predetermined response 9110 includes a plurality of pictogram characters (for example, the pictogram character 9111 in FIG. 9D or the pictogram character in the response 9110 in FIG. 9G).

  The device detects a default user action for selecting a certain default response among a plurality of default responses (eg, for a finger tap gesture 9120 for response 9110-1 in FIG. 9E, or for response 9110-4 in FIG. 9G). Finger tap gesture 9210) (1406). The device detects a default user action to start sending a response to the incoming message (1408). In some embodiments, upon detection of selection of default response 9110, transmission of the default response is initiated without detecting further action by the user. In some embodiments, after selection of a default response 9110 is detected, further default user actions to initiate transmission of the response are detected, for example, activation of the home button 204 is detected, and a “Send” button (Not shown) activation is detected, or a sliding finger gesture 2213 that apparently moves the unlocked image 302 across the channel 306 is detected (in this case, the message “slide to see” in the channel 306 is shown in FIG. Message "Slide to send response" or similar message).

  In response to detecting a default user action to initiate transmission of a response to an incoming message, the device transmits a response to the incoming message (1410). The selected default response 9110 includes a pictographic character (eg, 9111) among a plurality of pictographic characters (1410).

  In an embodiment, in response to detecting a default user action to initiate transmission of a selected default response to an incoming message, the device returns the touch screen display to a power save state (1412). . In some embodiments, the device returns (1414) the touch screen display to a power save state within a predetermined time (eg, 5 to 15 seconds) after the selected default response to the incoming message has been sent.

  FIG. 15 is a flowchart illustrating another method for responding to a message with a pictogram while the portable electronic device 100 equipped with the touch-sensitive display 112 is in a user interface lock mode according to an embodiment. This method 1500 provides a simple and intuitive way for a user to respond quickly to incoming messages while the device is in a locked state, thereby conserving power and at the time of battery charging and battery charging. It extends the period between time.

  While portable electronic device 100 is in the user interface lock mode (1501), operations 1502-1518 are performed. The device receives an incoming message from the party to the user of the portable electronic device while the touch screen display is in a power saving state (1502). In the power save state, the touch screen is blank (nothing is displayed) and is not touch sensitive.

  In response to receiving the incoming message, the device powers on the touch screen display and displays at least a portion of the incoming message from the party to the user (eg, message portion 2212 of FIG. 9B) on the touch screen display. (1504).

  The device detects a default user action (e.g., a finger tap on a response icon (not shown) or a double tap gesture on the touch screen 112) to initiate display of a plurality of default responses 9110 for incoming messages. (1506). The prescribed response includes a plurality of pictogram characters (for example, the pictogram character 9111 in FIG. 9D and the pictogram character in the response 9110 in FIG. 9G). In response to detecting a default user action to initiate display of a plurality of default responses to an incoming message, the device displays a plurality of default responses (eg, response 9110 of FIG. 9D or FIG. 9G). It is displayed (1508).

  The apparatus may use a predefined user action (eg, finger tap gesture 9120 for response 9110-1 in FIG. 9E or finger tap gesture 9210 for response 9110-4 in FIG. 9G) to select a predetermined response among a plurality of default responses. ) Is detected (1510).

  The device detects a default user action to initiate transmission of the selected default response to the incoming message (1512). In some embodiments, upon detection of selection of default response 9110, transmission of the default response is initiated without detecting further action by the user. In some embodiments, after selection of a default response 9110 is detected, further default user actions to initiate transmission of the response are detected, for example, activation of the home button 204 is detected, and a “Send” button (Not shown) activation is detected, or a sliding finger gesture 2213 that apparently moves the unlocked image 302 across the channel 306 is detected (in this case, the message “slide to see” in the channel 306 is shown in FIG. Message "Slide to send response" or similar message).

  In response to detecting a default user action to initiate transmission of the selected default response to the incoming message, the device transmits the selected default response to the incoming message (1514). The selected default response includes a pictographic character in the plurality of pictographic characters.

  In an embodiment, in response to detecting a default user action to initiate transmission of a selected default response to an incoming message, the device returns the touch screen display to a power save state (1516). . In some embodiments, the device returns 1518 the touch screen display to a power save state within a predetermined time (eg, 5-15 seconds) after a selected default response to an incoming message has been sent.

  FIG. 16 is a flowchart illustrating a method for specifying user-defined gestures for pictograms in portable electronic device 100 equipped with touch-sensitive display 112 according to some embodiments. This method 1600 provides a simple and intuitive way for a user to associate an emoji character with a user-defined gesture for later recall so that the user does not have to use an emoji keyboard. A character (for example, a pictograph character frequently used by a user) can be input to the apparatus.

  The portable electronic device detects a user-defined gesture (eg, gesture 1030 of FIG. 10B) on the touch screen display while it is in the user-defined gesture recording mode (1602). The device generates a representation of the user-defined gesture (1604). This representation includes the absolute or relative position of the finger contact in the gesture and the relative or absolute movement of the finger contact in the gesture. The device associates the representation of the user-defined gesture with the pictogram character (as described above with reference to FIGS. 10C-10E) (1606). The apparatus stores the expression of the user-defined gesture and the association between the expression of the user-defined gesture and the pictogram character (1608).

  In some embodiments, after storing the user-defined gesture representation and the association between the user-defined gesture representation and the emoji character, the device detects a gesture on the touchscreen display, and the detected gesture is a user-defined gesture. Determine if it corresponds to the stored representation of. If the detected gesture corresponds to a stored representation of a user-defined gesture, the device inserts an emoji character into an electronic message or document, such as an instant message (eg, text message) or an email message ( 1610).

  FIG. 17 is a flowchart illustrating a method for using a pictograph as a position indicator on a map via an electronic message according to an embodiment. This method 1700 is a quick and easy way to give location data with pictograms on the map and increase the information in the corresponding electronic message.

  The device (e.g., portable device 100 or desktop computing device) includes instructions that can initiate or operate to start the geographic map application 154, location data that represents a location on the geographic map, and glyph data that represents a glyph character. An electronic message is received (1702). The pictographic data is associated with the position data (for example, the pictographic character corresponding to the pictographic data is displayed on the map at the position corresponding to the position data). In one embodiment, the electronic message is a text message with a link to open a geographic map application (1704).

  The device displays (1706) an electronic message (eg, message 2216 of FIG. 11A) on the touch screen display. The device detects a default user action on the displayed electronic message to initiate the geographic map application (eg, tap gesture 1105 on link 2220 to Lake Tahoe in message 2216 of FIG. 11A). (1708). In response to detecting a default user action to initiate a geographic map application, the device displays a geographic map (eg, map 1115 in FIG. 11B) in geographic map application 154 and corresponds to location data. A pictogram character (eg, pictogram character 1120 in FIG. 11B) is displayed at the position of the geographic map (1710).

  The present invention has been illustrated above with reference to specific embodiments. However, the above description is not intended to be exhaustive or to limit the invention to the precise form disclosed herein. Many changes and modifications are possible in light of the above teaching. The above embodiments have been selected and described in order to best explain the principles of the invention and its particular application, and accordingly, those skilled in the art will recognize the invention and various embodiments. It may be best utilized with various modifications to suit the particular application.

100: Portable multifunction device 102: Memory 103: Communication bus or signal line 106: I / O subsystem 108: RF circuit 110: Audio circuit 111: Speaker 112: Touch-sensitive display system 113: Microphone 116: Other input control device 118: Peripheral interface 120: Processor 122: Controller 124: External port 126: Operating system 128: Communication module 130: Contact / motion module 132: Graphic module 134: Text input module 135: GPS module 136: Application 137: Contact module 138 : Telephone module 139: Video conference module 140: E-mail client module 141: Instant Messaging module 142: Training support module 143: Camera module 144: Video management module 145: Video player module 146: Music player module 147: Browsing module 148: Calendar module 149: Widget module 149-1: Weather widget 149-2: Stock widget 149-3: Calculator widget 149-4: Alarm clock widget 149-5: Dictionary widget 149-6: User generated widget 150: Widget generation module 151: Search module 152: Video and music player module 153: Note module 154: Map Module 155: Online video module 156: Display controller 158: optical sensor controller 160: other input controller 162: power supply system 164: Optical sensor 166: proximity sensor 168: accelerometer

Claims (20)

While in a portable electronic device with a touch screen display, the portable electronic device is in user interface lock mode,
Receiving an incoming message from a partner to a user of the portable electronic device while the touch screen display is in a power saving state;
In response to receiving the incoming message,
Power on the touch screen display;
Displaying at least a portion of the message arriving from the other party to the user on the touch screen display;
Steps to do that,
Detecting one or more predefined finger gestures on the touchscreen display corresponding to an emoji character without using an emoji keyboard;
Detecting a default user action to initiate transmission of a response to the incoming message;
Transmitting a response to the incoming message in response to detecting the default user action to initiate transmission of the response to the incoming message, the response being on the touch screen display Including the emoji character corresponding to the one or more predetermined finger gestures detected;
A computer-implemented method comprising:   The computer-implemented method of claim 1, further comprising returning the touch screen display to a power saving state in response to detecting a predetermined user action to initiate transmission of a response to the incoming message.   The computer-implemented method of claim 1, further comprising returning the touch screen display to a power saving state within a predetermined time after a response to the incoming message is transmitted. Touch screen display,
One or more processors;
Memory,
One or more programs,
And wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs are user interface locked by a portable electronic device. While in mode
Receiving a message arriving from the other party to the user of the portable electronic device while the touch screen display is in a power saving state;
In response to receiving the incoming message,
Power on the touch screen display;
Displaying at least a portion of the message arriving from the other party to the user on the touch screen display;
Doing
Detecting one or more default finger gestures on the touchscreen display corresponding to the emoji character without using the emoji keyboard;
Detecting a default user action to start sending a response to the incoming message;
Transmitting a response to the incoming message in response to detecting the default user action to initiate transmission of the response to the incoming message;
Portable electronic device, wherein the response includes an emoji character corresponding to one or more predetermined finger gestures detected on the touch screen display.   5. The portable electronic device of claim 4, further comprising instructions for returning the touch screen display to a power save state in response to detecting a predetermined user action to initiate transmission of a response to the incoming message. . While executed by a portable electronic device with a touch screen display, the portable electronic device is in user interface lock mode,
Receiving a message arriving from the other party to the user of the portable electronic device while the touch screen display is in a power saving state;
In response to receiving the incoming message,
Power on the touch screen display;
Displaying at least a portion of the message arriving from the other party to the user on the touch screen display;
Doing
Detecting one or more default finger gestures on the touchscreen display corresponding to the emoji character without using the emoji keyboard;
Detecting a default user action to start sending a response to the incoming message;
Storing instructions to cause the response to the incoming message to be transmitted in response to detecting the default user action to initiate transmission of the response to the incoming message, A computer readable storage medium comprising pictogram characters corresponding to the one or more predetermined finger gestures detected on a touch screen display.   The computer-readable storage medium of claim 6, further comprising instructions that cause the portable electronic device to return the touch screen display to a power saving state within a predetermined time after a response to the incoming message is transmitted. . In a portable electronic device with a touch screen display, while the portable electronic device is in a user interface lock mode,
Receiving an incoming message from a partner to a user of the portable electronic device while the touch screen display is in a power saving state;
In response to receiving the incoming message,
Power on the touch screen display;
Displaying at least a portion of the message arriving from the other party to the user on the touch screen display;
Displaying a plurality of default responses to the incoming message, the default responses including a plurality of emoji characters;
Steps to do that,
Detecting a default user action for selecting a default response among the plurality of default responses;
Detecting a default user action for initiating transmission of the selected default response to the incoming message , including a finger gesture on the touch screen display;
In transmitting in response to detecting the predetermined user action to start sending default response said selected relative to the incoming message, the default response said selected relative to the incoming message The selected default response includes a pictogram character among a plurality of pictogram characters;
A computer-implemented method comprising:   9. The method of claim 8, further comprising returning the touch screen display to a power saving state in response to detecting a default user action to initiate transmission of the selected default response to the incoming message. A computer-implemented method as described.   9. The computer-implemented method of claim 8, further comprising returning the touch screen display to a power saving state within a predetermined time after the selected default response to the incoming message is transmitted. Touch screen display,
One or more processors;
Memory,
One or more programs,
And wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs are user interface locked by a portable electronic device. While in mode
Receiving a message arriving from the other party to the user of the portable electronic device while the touch screen display is in a power saving state;
In response to receiving the incoming message,
Power on the touch screen display;
Displaying at least a portion of the message arriving from the other party to the user on the touch screen display;
Displaying a plurality of default responses to the incoming message, including a plurality of pictogram characters;
Doing
Detecting a default user action for selecting a default response among the plurality of default responses;
Detecting a default user action for initiating transmission of the selected default response to the incoming message , including a finger gesture on the touch screen display;
Transmitting the selected default response to the incoming message in response to detecting the default user action to initiate transmission of the selected default response to the incoming message;
A portable electronic device, wherein the selected default response includes an emoji character among a plurality of emoji characters.   Further comprising instructions for returning the touch screen display to a power save state in response to detecting the default user action to initiate transmission of the selected default response to the incoming message. The portable electronic device according to claim 11. While executed by a portable electronic device with a touch screen display, the portable electronic device is in user interface lock mode,
Receiving a message arriving from the other party to the user of the portable electronic device while the touch screen display is in a power saving state;
In response to receiving the incoming message,
Power on the touch screen display;
Displaying at least a portion of the message arriving from the other party to the user on the touch screen display;
Displaying a plurality of default responses to the incoming message, the default responses including a plurality of emoji characters;
Doing
Detecting a default user action for selecting a default response among the plurality of default responses;
Detecting a default user action for initiating transmission of the selected default response to the incoming message , including a finger gesture on the touch screen display;
Transmitting the selected default response to the incoming message in response to detecting the default user action to initiate transmission of the selected default response to the incoming message;
A computer-readable storage medium storing instructions to cause the selected default response to include a pictographic character among a plurality of pictographic characters.   14. The instruction of claim 13, further comprising causing the portable electronic device to return the touch screen display to a power save state within a predetermined time after the selected default response to the incoming message is transmitted. A computer-readable storage medium. In a portable electronic device with a touch screen display, while the portable electronic device is in a user interface lock mode,
Receiving an incoming message from a partner to a user of the portable electronic device while the touch screen display is in a power saving state;
In response to receiving the incoming message,
Power on the touch screen display;
Displaying at least a portion of the message arriving from the other party to the user on the touch screen display;
Steps to do that,
Detecting a default user action for initiating display of a plurality of default responses to the incoming message, wherein the default response includes a plurality of pictogram characters, wherein the at least one default response is text And a step including the emoji character at the same time;
Displaying the plurality of default responses in response to detecting the default user action to initiate display of a plurality of default responses to the incoming message;
Detecting a default user action for selecting a default response among the plurality of default responses;
Detecting a default user action for initiating transmission of the selected default response to the incoming message , including a finger gesture on the touch screen display;
In response to detecting the default user action to initiate transmission of the selected default response to the incoming message, transmitting the selected default response to the incoming message. The selected default response includes a pictogram character among the plurality of pictogram characters;
A computer-implemented method comprising:   16. The method of claim 15, further comprising returning the touch screen display to a power save state in response to detecting a default user action to initiate transmission of the selected default response to the incoming message. A computer-implemented method as described.   The computer-implemented method of claim 15, further comprising returning the touch screen display to a power saving state within a predetermined time after the selected default response to the incoming message is transmitted. Touch screen display,
One or more processors;
Memory,
One or more programs,
And wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs are user interface locked by a portable electronic device. While in mode
Receiving a message arriving from the other party to the user of the portable electronic device while the touch screen display is in a power saving state;
In response to receiving the incoming message,
Power on the touch screen display;
Displaying at least a portion of the message arriving from the other party to the user on the touch screen display;
Doing
Detecting a default user action for initiating display of a plurality of default responses to the incoming message, wherein the default response includes a plurality of emoji characters, and at least one default response includes text and emoji characters; Including at the same time,
In response to detecting the default user action to initiate display of the plurality of default responses to the incoming message, displaying the plurality of default responses;
Detecting a default user action for selecting a default response among the plurality of default responses;
Detecting a default user action for initiating transmission of the selected default response to the incoming message , including a finger gesture on the touch screen display;
Transmitting the selected default response to the incoming message in response to detecting the default user action to initiate transmission of the selected default response to the incoming message;
A portable electronic device, wherein the selected default response includes a pictographic character of the plurality of pictographic characters. In response to detecting the predetermined user action to start sending default response said selected relative to the incoming message, further comprising instructions for returning the touch screen display to the power saving state, The portable electronic device according to claim 18. While executed by a portable electronic device with a touch screen display, while the portable electronic device is in user interface lock mode,
Receiving a message arriving from the other party to the user of the portable electronic device while the touch screen display is in a power saving state;
In response to receiving the incoming message,
Power on the touch screen display;
Displaying at least a portion of the message arriving from the other party to the user on the touch screen display;
Doing
Detecting a default user action for initiating display of a plurality of default responses to the incoming message, wherein the default response includes a plurality of emoji characters, and at least one default response includes text and emoji characters; Including at the same time,
In response to detecting the default user action to initiate display of the plurality of default responses to the incoming message, displaying the plurality of default responses;
Detecting a default user action for selecting a default response among the plurality of default responses;
Detecting a default user action for initiating transmission of the selected default response to the incoming message , including a finger gesture on the touch screen display;
Transmitting the selected default response to the incoming message in response to detecting the default user action to initiate transmission of the selected default response to the incoming message;
A computer-readable storage medium storing instructions to cause the selected default response to include a pictographic character in the plurality of pictographic characters.

Images