CN110457104A - Multi-player real time communication user interface - Google Patents

Multi-player real time communication user interface Download PDF

Info

Publication number
CN110457104A
CN110457104A CN201910704856.XA CN201910704856A CN110457104A CN 110457104 A CN110457104 A CN 110457104A CN 201910704856 A CN201910704856 A CN 201910704856A CN 110457104 A CN110457104 A CN 110457104A
Authority
CN
China
Prior art keywords
participant
real time
communication session
time communication
expression
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201910704856.XA
Other languages
Chinese (zh)
Inventor
M·范欧斯
L·布劳顿
P·卡拉威
P·F·卡洛
J·W·常
A·德瑞尔
A·戴
C·费德瑞吉
R·加西亚
N·V·金
S·O·勒梅
J·劳驰
H·珐姆
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Apple Inc
Original Assignee
Apple Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from DKPA201870362A external-priority patent/DK180130B1/en
Application filed by Apple Inc filed Critical Apple Inc
Publication of CN110457104A publication Critical patent/CN110457104A/en
Pending legal-status Critical Current

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L51/00—User-to-user messaging in packet-switching networks, transmitted according to store-and-forward or real-time protocols, e.g. e-mail
    • H04L51/04—Real-time or near real-time messaging, e.g. instant messaging [IM]
    • H04L51/046—Interoperability with other network applications or services
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0481—Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
    • G06F3/04817—Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance using icons
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0484—Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
    • G06F3/04845—Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range for image manipulation, e.g. dragging, rotation, expansion or change of colour
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0487—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
    • G06F3/0488—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0487—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
    • G06F3/0488—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures
    • G06F3/04886—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures by partitioning the display area of the touch-screen or the surface of the digitising tablet into independently controllable areas, e.g. virtual keyboards or menus
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44—Arrangements for executing specific programs
    • G06F9/451—Execution arrangements for user interfaces
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q10/00—Administration; Management
    • G06Q10/10—Office automation; Time management
    • G06Q10/101—Collaborative creation, e.g. joint development of products or services
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q10/00—Administration; Management
    • G06Q10/10—Office automation; Time management
    • G06Q10/107—Computer-aided management of electronic mailing [e-mailing]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q50/00—Systems or methods specially adapted for specific business sectors, e.g. utilities or tourism
    • G06Q50/01—Social networking
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L12/00—Data switching networks
    • H04L12/02—Details
    • H04L12/16—Arrangements for providing special services to substations
    • H04L12/18—Arrangements for providing special services to substations for broadcast or conference, e.g. multicast
    • H04L12/1813—Arrangements for providing special services to substations for broadcast or conference, e.g. multicast for computer conferences, e.g. chat rooms
    • H04L12/1818—Conference organisation arrangements, e.g. handling schedules, setting up parameters needed by nodes to attend a conference, booking network resources, notifying involved parties
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L51/00—User-to-user messaging in packet-switching networks, transmitted according to store-and-forward or real-time protocols, e.g. e-mail
    • H04L51/48—Message addressing, e.g. address format or anonymous messages, aliases
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04M—TELEPHONIC COMMUNICATION
    • H04M1/00—Substation equipment, e.g. for use by subscribers
    • H04M1/72—Mobile telephones; Cordless telephones, i.e. devices for establishing wireless links to base stations without route selection
    • H04M1/724—User interfaces specially adapted for cordless or mobile telephones
    • H04M1/72403—User interfaces specially adapted for cordless or mobile telephones with means for local support of applications that increase the functionality
    • H04M1/7243—User interfaces specially adapted for cordless or mobile telephones with means for local support of applications that increase the functionality with interactive means for internal management of messages
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00—Television systems
    • H04N7/14—Systems for two-way working
    • H04N7/141—Systems for two-way working between two video terminals, e.g. videophone
    • H04N7/147—Communication arrangements, e.g. identifying the communication as a video-communication, intermediate storage of the signals
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00—Television systems
    • H04N7/14—Systems for two-way working
    • H04N7/15—Conference systems
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L51/00—User-to-user messaging in packet-switching networks, transmitted according to store-and-forward or real-time protocols, e.g. e-mail
    • H04L51/04—Real-time or near real-time messaging, e.g. instant messaging [IM]
    • H04L51/043—Real-time or near real-time messaging, e.g. instant messaging [IM] using or handling presence information
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04M—TELEPHONIC COMMUNICATION
    • H04M3/00—Automatic or semi-automatic exchanges
    • H04M3/42—Systems providing special services or facilities to subscribers
    • H04M3/56—Arrangements for connecting several subscribers to a common circuit, i.e. affording conference facilities

Abstract

Entitled " multi-player real time communication user interface " of the invention.The present invention provides a kind of methods, the described method includes: the instant message user interface of display instant message application program, the instant message user interface includes message region, and the message region includes multiple message in the message conversation between three or more participants.Receive the first data for indicating that the available real time communication session of three or more participants is in active state.In response to receiving first data, display indicates the notice that the real time communication session is in active state.When showing the notice, the second data for indicating that the real time communication session between three or more described participants is no longer on active state are received.In response to receiving second data, the notice that real time communication session described in stopping display or update instruction is in active state is to indicate that the real time communication session is no longer on active state.

Description

Multi-player real time communication user interface
The application be the applying date be September in 2018 28, application No. is 201811136766.7, entitled " multi-players The divisional application of the application for a patent for invention of real time communication user interface ".
Cross reference to related applications
Present patent application is related to entitled " the MULTI-PARTICIPANT LIVE submitted on May 7th, 2018 The U.S. Provisional Patent Application Serial No. 62/668,229 of COMMUNICATION USER INTERFACE ", the patent application Content is incorporated by reference is incorporated to accordingly.
Technical field
The present disclosure generally relates to computer user interfaces, more particularly, are related to for including the real-time of multiple participants The technology of communication.
Background technique
Since personal and business reason, user use electronic equipment to carry out virtual communication with other people more and more.Mostly Number electronic equipment can provide real time communication between user and multiple participants.As an example, some electronic equipments provide use In the interactive interface for showing and controlling real time communication.
Summary of the invention
However, for include using the real time communication of multiple participants of electronic equipment some technologies usually it is pretty troublesome and Inefficiency.For example, some prior arts are using complicated and time-consuming user interface, the user interface may include multiple key or Keystroke.The prior art was needed than required more times, this leads to the time and the plant capacity that waste user.This later regard exists It is especially important in battery-driven equipment.
Therefore, this technology for electronic equipment provide faster, more effective way and interface, for including multiple participants Real time communication.Such method and interface optionally supplement or replace its other party for the real time communication including multiple participants Method.Such method and interface are reduced to cognitive load caused by user, and generate more effective man-machine interface.For battery Power is saved at the calculating equipment of driving, such method and interface, and increases the time interval between battery charging.
In some embodiments, a kind of method is executed at the electronic equipment with display.This method can include: aobvious Show the instant message user interface of instant message application program, which includes message region, the message area Domain includes multiple message in the message conversation between three or more participants；It receives and indicates three or more participants The first data that available real time communication session is in active state；In response to receive indicate three or more participants it Between the first data for being in active state of real time communication session, display instruction real time communication session is in active state logical Know；While the notice that display instruction real time communication session is in active state, receives and indicate three or more participants Between real time communication session be no longer on the second data of active state；And three or more are indicated in response to receiving Real time communication session between participant is no longer on the second data of active state, stops display notice or update notification to refer to Show that real time communication session is no longer on active state.
In some embodiments, non-transient computer readable storage medium stores one or more programs, one Or multiple programs are configured as being executed by the one or more processors of the electronic equipment with display.It is one or more of Program includes the instruction for following operation: the instant message user interface of display instant message application program, the instant message User interface includes message region, which includes that multiple in the message conversation between three or more participants disappear Breath；Receive the first data that the instruction available real time communication session of three or more participants is in active state；In response to Receive the first data for indicating that the real time communication session between three or more participants is in active state, display instruction The notice that real time communication session is in active state；In the same of the notice that display instruction real time communication session is in active state When, receive the second data for indicating that the real time communication session between three or more participants is no longer on active state；And And indicate that the real time communication session between three or more participants is no longer on the second of active state in response to receiving Data stop display notice or update notification to indicate that real time communication session is no longer on active state.
In some embodiments, transitory computer readable storage medium stores one or more programs, it is one or Multiple programs are configured as being executed by the one or more processors of the electronic equipment with display.One or more of journeys Sequence includes the instruction for following operation: the instant message user interface of display instant message application program, the instant message are used Family interface includes message region, which includes that multiple in the message conversation between three or more participants disappear Breath；Receive the first data that the instruction available real time communication session of three or more participants is in active state.In response to Receive the first data for indicating that the real time communication session between three or more participants is in active state, display instruction The notice that real time communication session is in active state；In the same of the notice that display instruction real time communication session is in active state When, receive the second data for indicating that the real time communication session between three or more participants is no longer on active state；And And indicate that the real time communication session between three or more participants is no longer on the second of active state in response to receiving Data stop display notice or update notification to indicate that real time communication session is no longer on active state.
In some embodiments, electronic equipment includes display, one or more processors and memory.Memory Storage is configured as the one or more programs executed by one or more of processors.One or more of programs include Instruction for following operation: the instant message user interface of display instant message application program, the instant message user interface Including message region, which includes multiple message in the message conversation between three or more participants；It receives Indicate the first data that the available real time communication session of three or more participants is in active state.Refer in response to receiving Show the first data that the real time communication session between three or more participants is in active state, display instruction real time communication The notice that session is in active state；While the notice that display instruction real time communication session is in active state, reception refers to Show that the real time communication session between three or more participants is no longer on the second data of active state；And in response to connecing The second data for indicating that the real time communication session between three or more participants is no longer on active state are received, are stopped aobvious Show notice or update notification to indicate that real time communication session is no longer on active state.
In some embodiments, electronic equipment includes: display；For showing disappearing immediately for instant message application program The device of user interface is ceased, which includes message region, which includes three or more ginsengs Multiple message in message conversation between person；The available real time communication meeting of three or more participants is indicated for receiving Talk about the device for the first data being in active state；It is real-time logical between three or more participants of instruction in response to receiving The first data for being in active state of letter session and the device for being used for following operation: display instruction real time communication session is in activity The notice of state；The device while notice that display instruction real time communication session is in active state for following operation: Receive the second data for indicating that the real time communication session between three or more participants is no longer on active state；And it uses In the device for executing following operation: in response to receiving the real time communication session between three or more participants of instruction no longer The second data being in active state stop display notice or update notification to indicate that real time communication session is no longer on moving type State.
In some embodiments, a kind of method is executed at the electronic equipment with display.This method can include: connect Receive request of the display for the user interface of the real time communication session between two or more participants；And in response to receiving Request to display for the user interface of the real time communication session between two or more participants, display real time communication are used Family interface, multiple participants including being shown in real time communication session simultaneously indicate, wherein more in display real time communication session A participant indicates to include: according to the quantity of the participant in determining real time communication session to include the participant's number for being greater than threshold value Amount, while showing that more than first in addition to the user of electronic equipment are joined to be greater than one or more sizes of respective threshold size With the expression of person, and the expression of a participant more than second is shown in an overflow region, wherein a participant more than second includes not wrapping Other the one or more ginsengs for including one or more participants in participant a more than first, and showing in an overflow region Expression with person is shown with the size for being less than respective threshold size；And according to the participant's quantity determined in real time communication session Participant's quantity including being less than or equal to threshold value is shown with being greater than one or more sizes of respective threshold size except electronics is set The expression of participant except standby user shows without one or more sizes to be less than or equal to respective threshold size and removes The expression of participant except the user of electronic equipment.
In some embodiments, non-transient computer readable storage medium stores one or more programs, one Or multiple programs are configured as being executed by the one or more processors of the electronic equipment with display.It is one or more of Program includes the instruction for following operation: receiving display for the real time communication session between two or more participants The request of user interface；And in response to receiving display for the real time communication session between two or more participants The request of user interface shows real time communication user interface, including showing multiple participant's tables in real time communication session simultaneously Show, wherein multiple participants in display real time communication session indicate to include: the participant according to determining in real time communication session Quantity include participant's quantity greater than threshold value, while show with being greater than one or more sizes of respective threshold size except electric The expression of more than first a participants except the user of sub- equipment, and the table of a participant more than second is shown in an overflow region Show, wherein a participant more than second includes the one or more participants being not included in more than first in a participant, and is overflowing The expression of other the one or more participants shown in region out is shown with the size for being less than respective threshold size；And according to Determine that participant's quantity in real time communication session includes participant's quantity less than or equal to threshold value, to be greater than respective threshold ruler Very little one or more sizes show the expression of the participant in addition to the user of electronic equipment, without corresponding to be less than or equal to One or more sizes of threshold size show the expression of the participant in addition to the user of electronic equipment.
In some embodiments, transitory computer readable storage medium stores one or more programs, it is one or Multiple programs are configured as being executed by the one or more processors of the electronic equipment with display.One or more of journeys Sequence includes the instruction for following operation: receiving use of the display for the real time communication session between two or more participants The request at family interface；And in response to receiving use of the display for the real time communication session between two or more participants The request at family interface shows real time communication user interface, and multiple participants including being shown in real time communication session simultaneously indicate, Wherein showing multiple participants in real time communication session indicates to include: the number according to the participant in real time communication session is determined Amount includes participant's quantity greater than threshold value, while being shown with being greater than one or more sizes of respective threshold size except electronics is set The expression of more than first a participants except standby user, and the expression of a participant more than second is shown in an overflow region, In more than second a participants include the one or more participants being not included in more than first in a participant, and in overflow area The expression of other one or more participants of middle display is shown with the size for being less than respective threshold size；And it is real according to determining When communication session in participant's quantity include participant's quantity less than or equal to threshold value, to be greater than the one of respective threshold size A or multiple sizes show the expression of the participant in addition to the user of electronic equipment, without to be less than or equal to respective threshold ruler Very little one or more sizes show the expression of the participant in addition to the user of electronic equipment.
In some embodiments, electronic equipment includes display, one or more processors and memory.Memory Storage is configured as the one or more programs executed by one or more of processors.One or more of programs include Instruction for following operation: user interface of the display for the real time communication session between two or more participants is received Request；And in response to receiving user interface of the display for the real time communication session between two or more participants Request, real time communication user interface is shown, including showing that multiple participants in real time communication session indicate simultaneously, wherein showing Showing multiple participants in real time communication session indicates to include: to include according to the quantity of the participant in determining real time communication session The use except electronic equipment is shown greater than participant's quantity of threshold value, while with one or more sizes greater than respective threshold size The expression of more than first a participants except family, and the expression of a participant more than second is shown in an overflow region, wherein second Multiple participants include the one or more participants being not included in more than first in a participant, and are shown in an overflow region The expression of other one or more participants shown with the size for being less than respective threshold size；And according to determining real time communication Participant's quantity in session includes participant's quantity less than or equal to threshold value, to be greater than one or more of respective threshold size A size shows the expression of the participant in addition to the user of electronic equipment, without with one less than or equal to respective threshold size A or multiple sizes show the expression of the participant in addition to the user of electronic equipment.
In some embodiments, electronic equipment includes: display；For receiving display for two or more participations The device of the request of the user interface of real time communication session between person；And in response to receiving display for two or more The request of the user interface of real time communication session between a participant and the device for being used for following operation: display real time communication is used Family interface, multiple participants including being shown in real time communication session simultaneously indicate, wherein more in display real time communication session A participant indicates to include: according to the quantity of the participant in determining real time communication session to include being greater than participant's quantity of threshold value And it is used for the device of following operation: while the use except electronic equipment is shown with one or more sizes greater than respective threshold size The expression of more than first a participants except family, and the expression of a participant more than second is shown in an overflow region, wherein second Multiple participants include the one or more participants being not included in more than first in a participant, and are shown in an overflow region The expression of other one or more participants shown with the size for being less than respective threshold size；And the dress for following operation It sets: including participant's quantity less than or equal to threshold value according to the participant's quantity determined in real time communication session, to be greater than phase Answer one or more sizes of threshold size to show the expression of the participant in addition to the user of electronic equipment, without to be less than or Equal to the expression that one or more sizes of respective threshold size show the participant in addition to the user of electronic equipment.
In some embodiments, a kind of method is executed at the electronic equipment with display.This method can include: aobvious Show live video communication user interface comprising while showing the expression of the first participant in live video communication session, it is real When video communication sessions in second participant expression and the third participant in live video communication session expression； While showing live video communication user interface, the movable water of second participant in instruction live video communication session is received It is flat to have increased to the data higher than the activity level of first participant in live video communication session；And refer in response to receiving Show that the activity level of second participant in live video communication session has increased to higher than the first ginseng in live video communication session With the data of the activity level of person: changing the size of the expression of first participant and/or position in live video communication user interface It sets；Change the size of the expression of second participant and/or position in live video communication user interface；And change real-time video The size of the expression of third participant and/or position in communication user interface.
In some embodiments, non-transient computer readable storage medium stores one or more programs, one Or multiple programs are configured as being executed by the one or more processors of the electronic equipment with display.It is one or more of Program includes the instruction for following operation: display live video communication user interface comprising while showing that real-time video is logical Believe the expression of the first participant in session, the expression of the second participant in live video communication session and real-time video The expression of third participant in communication session；While showing live video communication user interface, instruction view in real time is received The activity level of second participant has increased to the work higher than first participant in live video communication session in frequency communication session Move horizontal data；And the activity level in response to receiving second participant in instruction live video communication session has increased To the data for being higher than the activity level of first participant in live video communication session: changing in live video communication user interface The size of the expression of first participant and/or position；Change the expression of second participant in live video communication user interface Size and/or position；And change the size of the expression of third participant and/or position in live video communication user interface.
In some embodiments, transitory computer readable storage medium stores one or more programs, it is one or Multiple programs are configured as being executed by the one or more processors of the electronic equipment with display.One or more of journeys Sequence includes the instruction for following operation: display live video communication user interface comprising while showing live video communication The expression of first participant in session, the expression of the second participant in live video communication session and real-time video are logical Believe the expression of the third participant in session；While showing live video communication user interface, instruction real-time video is received The activity level of second participant has increased to the activity higher than first participant in live video communication session in communication session Horizontal data；And the activity level in response to receiving second participant in instruction live video communication session has increased to Higher than the data of the activity level of first participant in live video communication session: changing the in live video communication user interface The size of the expression of one participant and/or position；Change the ruler of the expression of second participant in live video communication user interface Very little and/or position；And change the size of the expression of third participant and/or position in live video communication user interface.
In some embodiments, electronic equipment includes display, one or more processors and memory.Memory Storage is configured as the one or more programs executed by one or more of processors.One or more of programs include Instruction for following operation: display live video communication user interface comprising while showing in live video communication session First participant expression, the expression and live video communication session of the second participant in live video communication session In third participant expression；While showing live video communication user interface, instruction live video communication meeting is received The activity level of second participant has increased to higher than the activity level of first participant in live video communication session in words Data；And the activity level in response to receiving second participant in instruction live video communication session, which has increased to, to be higher than in fact When video communication sessions in first participant activity level data: change live video communication user interface in first participate in The size of the expression of person and/or position；Change live video communication user interface in second participant expression size and/or Position；And change the size of the expression of third participant and/or position in live video communication user interface.
In some embodiments, electronic equipment includes: display；For showing the dress of live video communication user interface It sets, display live video communication user interface includes showing the expression of the first participant in live video communication session simultaneously, The expression of second participant in live video communication session and the table of the third participant in live video communication session Show；Device for performing the following operations: while showing live video communication user interface, it is logical to receive instruction real-time video The activity level of second participant has increased to the movable water higher than first participant in live video communication session in letter session Flat data；And the activity level in response to receiving second participant in instruction live video communication session has increased to height The data of the activity level of first participant in live video communication session and the device for being used for following operation: change view in real time The size of the expression of first participant and/or position in frequency communication user interface；Change the in live video communication user interface The size of the expression of two participants and/or position；And change the expression of third participant in live video communication user interface Size and/or position.
In some embodiments, a kind of method is executed at the electronic equipment with display.This method can include: aobvious Show the instant message user interface of instant message application program, which includes: message region, the message area Domain includes in multiple message in the message conversation for including three or more participants between participant；And in reality When video communication application in start live video communication session separated the showing of message region can indicate；Detecting and show can table The corresponding input of the selection shown；And in response to detecting and showing the corresponding input of the selection that can indicate, in live video communication The process for starting live video communication session is initiated in application program, which includes message conversation Participant.
In some embodiments, non-transient computer readable storage medium stores one or more programs, one Or multiple programs are configured as being executed by the one or more processors of the electronic equipment with display.It is one or more of Program includes the instruction for following operation: the instant message user interface of display instant message application program, the instant message User interface includes: message region, which includes participating in the message conversation for including three or more participants Multiple message between person；And in live video communication application program start live video communication session message Separated the showing in region can indicate；Detect input corresponding with the selection that can indicate is shown；And it can be indicated in response to detecting and showing The corresponding input of selection, the process for starting live video communication session is initiated in live video communication application program, The live video communication session includes the participant of message conversation.
In some embodiments, transitory computer readable storage medium stores one or more programs, it is one or Multiple programs are configured as being executed by the one or more processors of the electronic equipment with display.One or more of journeys Sequence includes the instruction for following operation: the instant message user interface of display instant message application program, the instant message are used Family interface includes: message region, which includes the participant in the message conversation for including three or more participants Between multiple message；And in live video communication application program start live video communication session message area Separated the showing in domain can indicate；Detect input corresponding with the selection that can indicate is shown；And it can be indicated in response to detecting and showing Corresponding input is selected, the process for starting live video communication session is initiated in live video communication application program, it should Live video communication session includes the participant of message conversation.
In some embodiments, electronic equipment includes display, one or more processors and memory.Memory Storage is configured as the one or more programs executed by one or more of processors.One or more of programs include Instruction for following operation: the instant message user interface of display instant message application program, the instant message user interface It include: message region, which includes in the message conversation for including three or more participants between participant Multiple message；And it is separated with the message region for starting live video communication session in live video communication application program Show and can indicate；Detect input corresponding with the selection that can indicate is shown；And in response to detecting and showing the selection pair that can be indicated The process for starting live video communication session, the real-time view are initiated in the input answered in live video communication application program Frequency communication session includes the participant of message conversation.
In some embodiments, electronic equipment includes: display；For showing disappearing immediately for instant message application program The device of user interface is ceased, which includes: message region, which is included in including three or more Multiple message in the message conversation of multiple participants between participant；And in live video communication application program Separated the showing of message region for starting live video communication session can indicate；For detecting and showing that the selection that can be indicated is corresponding defeated The device entered；And the device of following operation to be used in response to detecting and showing the corresponding input of the selection that can be indicated: in reality When video communication application in initiate process for starting live video communication session, which includes The participant of message conversation.
In some embodiments, a kind of method is executed at the electronic equipment with display.This method can include: aobvious Show the live video communication user interface of live video communication application program, which includes: correspondence Show and can indicate in multiple participants of corresponding one or more participants, the multiple participant show can indicate include and three Or more the corresponding first participant of group of participant show and can indicate；And for starting new live video communication session New session, which is shown, to be indicated；Detection corresponds to the input of following request, that is, initiates the mistake for starting new live video communication session Journey；And in response to detecting the input: according to the corresponding input of the selection that can indicate is shown with first participant, initiating to start new Live video communication session process, which includes the group of three or more participants；And And according to the corresponding input of the selection that can be indicated is shown with new session, initiate for selecting two or more participants and opening The process for the live video communication session for beginning new, the new live video communication session include participant associated with the equipment And two or more selected participants.
In some embodiments, non-transient computer readable storage medium stores one or more programs, one Or multiple programs are configured as being executed by the one or more processors of the electronic equipment with display.It is one or more of Program includes the instruction for following operation: the live video communication user interface of display live video communication application program, should Live video communication user interface includes: to show and can indicate corresponding to multiple participants of corresponding one or more participants, institute It states multiple participants and shows that can indicate includes that first participant corresponding with the group of three or more participants shows and can indicate；And New session for starting new live video communication session is shown and can be indicated；Detection corresponds to the input of following request, that is, sends out Act the process for starting new live video communication session；And in response to detecting the input: showing energy according to first participant The process for starting new live video communication session, the new live video communication meeting are initiated in the corresponding input of the selection of expression Words include the group of three or more participants；And it according to the corresponding input of the selection that can be indicated is shown with new session, initiates For selecting two or more participants and starting the process of new live video communication session, the new real-time video is logical Believe that session includes participant associated with the equipment and participant selected by two or more.
In some embodiments, transitory computer readable storage medium stores one or more programs, it is one or Multiple programs are configured as being executed by the one or more processors of the electronic equipment with display.One or more of journeys Sequence includes the instruction for following operation: the live video communication user interface of display live video communication application program, the reality When video communications client interface include: to show and can indicate corresponding to multiple participants of corresponding one or more participants, it is described Multiple participants show that can indicate includes that first participant corresponding with the group of three or more participants shows and can indicate；And it uses Show and can indicate in the new session for starting new live video communication session；Detection corresponds to the input of following request, that is, initiates Start the process of new live video communication session；And in response to detecting the input: can table according to showing with first participant The process for starting new live video communication session, the new live video communication session are initiated in the corresponding input of the selection shown Group including three or more participants；And according to the corresponding input of the selection that can be indicated is shown with new session, initiate to use In selecting two or more participants and start the process of new live video communication session, the new live video communication Session includes participant associated with the equipment and participant selected by two or more.
In some embodiments, electronic equipment includes display, one or more processors and memory.Memory Storage is configured as the one or more programs executed by one or more of processors.One or more of programs include Instruction for following operation: the live video communication user interface of display live video communication application program, the real-time video Communication user interface includes: to show and can indicate corresponding to multiple participants of corresponding one or more participants, the multiple ginseng And person shows that can indicate includes that first participant corresponding with the group of three or more participants shows and can indicate；And for starting The new session of new live video communication session, which is shown, to be indicated；Detection corresponds to the input of following request, that is, initiates to start new Live video communication session process；And in response to detecting the input: showing the choosing that can be indicated according to first participant Corresponding input is selected, the process for starting new live video communication session is initiated, which includes three The group of a or more participant；And it according to the corresponding input of the selection that can be indicated is shown with new session, initiates for selecting Two or more participants and the process for starting new live video communication session, the new live video communication session packet Include participant associated with the equipment and participant selected by two or more.
In some embodiments, electronic equipment includes: display；For showing the reality of live video communication application program When video communications client interface device, which includes: to correspond to corresponding one or more ginseng Showing with multiple participants of person can indicate, the multiple participant shows that it includes right with the group of three or more participants for capable of indicating The first participant answered, which shows, to be indicated；And the new session for starting new live video communication session is shown and can be indicated；With Correspond to the input of following request in detection, that is, initiates the device for starting the process of new live video communication session；And it rings Ying Yu detects the input and is used for the device of following operation: according to showing that the selection that can be indicated is corresponding defeated with first participant Enter, initiates the process for starting new live video communication session, which includes three or more The group of participant；And it according to the corresponding input of the selection that can be indicated is shown with new session, initiates for selecting two or more A participant and the process for starting new live video communication session, the new live video communication session include and the equipment Associated participant and two or more selected participants.
Executable instruction for executing these functions, which is optionally included in, to be configured for being handled by one or more In the non-transient computer readable storage medium of device execution or other computer program products.For executing holding for these functions Row instruction is optionally included in the transitory computer readable storage medium for being configured for being performed by one or more processors Or in other computer program products.
Therefore, faster more efficient method and interface are provided with for the real time communication including multiple participants for equipment, Thus the validity, efficiency and user satisfaction of such equipment are improved.Such method and interface can supplement or replace for including The other methods of the real time communication of multiple participants.
Detailed description of the invention
The various embodiments in order to better understand should refer to following specific embodiment in conjunction with the following drawings, Wherein similar drawing reference numeral indicates corresponding part in all the appended drawings.
Figure 1A is to show the block diagram of the portable multifunction device with touch-sensitive display according to some embodiments.
Figure 1B is the block diagram for showing the example components for event handling according to some embodiments.
Fig. 2 shows the portable multifunction devices with touch screen according to some embodiments.
Fig. 3 is the block diagram according to the exemplary multifunctional equipment with display and touch sensitive surface of some embodiments.
Fig. 4 A, which is shown, to be shown according to some embodiments for the menu of application program on portable multifunction device Example property user interface.
Fig. 4 B, which is shown, to be set with the multi-functional of touch sensitive surface that display separates according to some embodiments for having Standby exemplary user interface.
Fig. 5 A shows the personal electronic equipments according to some embodiments.
Fig. 5 B is to show the block diagram of the personal electronic equipments according to some embodiments.
Fig. 5 C- Fig. 5 D, which is shown, according to some embodiments there is the personal electric of touch-sensitive display and intensity sensor to set Standby example components.
Fig. 5 E- Fig. 5 H shows the example components and user interface of the personal electronic equipments according to some embodiments.
Fig. 6 A to Fig. 6 AF shows the exemplary user interface according to some embodiments.
Fig. 7 A to Fig. 7 F shows the illustrative methods according to some embodiments.
Fig. 8 A to Fig. 8 BK shows the exemplary user interface according to some embodiments.
Fig. 9 A to Fig. 9 K shows the illustrative methods according to some embodiments.
Figure 10 A to Figure 10 S shows the exemplary user interface according to some embodiments.
Figure 11 A to Figure 11 F shows the illustrative methods according to some embodiments.
Figure 12 A to Figure 12 N shows the exemplary user interface according to some embodiments.
Figure 13 A to Figure 13 D shows the illustrative methods according to some embodiments.
Figure 14 A to Figure 14 I shows the exemplary user interface according to some embodiments.
Figure 15 A to Figure 15 C shows the illustrative methods according to some embodiments.
Specific embodiment
It is described below and elaborates illustrative methods, parameter etc..It should be appreciated, however, that such description is not intended to limit The scope of the present disclosure, but provided as the description to exemplary implementation scheme.
Electronic equipment needs to provide effective method and interface for the real time communication including multiple participants.Such skill Art can reduce the cognitive load for participating in the user of the real time communication including multiple participants, thus improve production efficiency.In addition, this Class technology can reduce the processor power being otherwise wasted in redundant subscribers input and the power of battery.
In the following, Figure 1A to Figure 1B, Fig. 2, Fig. 3, Fig. 4 A to Fig. 4 B and Fig. 5 A to Fig. 5 H are provided for executing for multiple The description of the example devices of the technology of real time communication session between participant.Fig. 6 A to Fig. 6 AF is shown for managing thing The exemplary user interface of part notice.Fig. 7 A to Fig. 7 F is to show between the multiple participants of progress according to some embodiments The flow chart of the method for real time communication session.User interface in Fig. 6 A to Fig. 6 AF is for showing including in Fig. 7 A to Fig. 7 F The process described below of process.
Fig. 8 A to Fig. 8 BK shows the exemplary user interface for the real time communication session between multiple participants.Figure 9A to Fig. 9 K is the process of the method for the real time communication session between the multiple participants of progress shown according to some embodiments Figure.User interface in Fig. 8 A to Fig. 8 BK is for showing the process described below including the process in Fig. 9 A to Fig. 9 K.
Figure 10 A to Figure 10 S shows the exemplary user interface for the real time communication session between multiple participants.Figure 11A to Figure 11 F is the stream of the method for the real time communication session between the multiple participants of progress shown according to some embodiments Cheng Tu.User interface in Figure 10 A to Figure 10 S is for showing the mistake described below including the process in Figure 11 A to Figure 11 F Journey.
Figure 12 A to Figure 12 N shows the exemplary user interface for the real time communication session between multiple participants.Figure 13A to Figure 13 D is the stream of the method for the real time communication session between the multiple participants of progress shown according to some embodiments Cheng Tu.User interface in Figure 12 A to Figure 12 N is for showing the mistake described below including the process in Figure 13 A to Figure 13 D Journey.
Figure 14 A to Figure 14 I shows the exemplary user interface for the real time communication session between multiple participants.Figure 15A to Figure 15 C is the stream of the method for the real time communication session between the multiple participants of progress shown according to some embodiments Cheng Tu.User interface in Figure 14 A to Figure 14 I is for showing the mistake described below including the process in Figure 15 A to Figure 15 C Journey.
Although being described below using term " first ", " second " etc. and describing various elements, these elements not Ying Shoushu The limitation of language.These terms are only intended to distinguish an element with another element.For example, first touches and can be named as the Two touches and similarly the second touch can be named as the first touch, without departing from the range of the various embodiments.The Both one touch and the second touch are to touch, but they are not same touches.
Term used in the description to the various embodiments is intended merely to description particular implementation side herein The purpose of case, and be not intended to be limiting.Such as in the description and the appended claims in the various embodiments Used "one" is intended to also include plural form with "the" singular like that, indicates unless the context clearly. It will be further understood that term "and/or" used herein refer to and cover in associated listed project one Any and all possible combinations of a or multiple projects.It will be further understood that term " includes " (" includes ", " including ", " comprises " and/or " comprising ") it specifies to exist when using in the present specification and be stated Feature, integer, step, operations, elements, and/or components, but it is not excluded that in the presence of or add other one or more features, whole Number, step, operation, component, assembly unit and/or its grouping.
Based on context, term " if " be optionally interpreted to mean " and when ... when ", " ... when " or " ring Should be in determination " or " in response to detecting ".Similarly, based on context, phrase " if it is determined that ... " or " if detected [condition or event stated] " is optionally interpreted to refer to " when in determination ... " or " in response to determination ... " or " In When detecting [condition or event stated] " or " in response to detecting [condition or event stated] ".
This document describes electronic equipments, the embodiment party of the user interface of such equipment and the correlated process using such equipment Case.In some embodiments, which is also portable comprising other function such as PDA and/or music player functionality Communication equipment, such as mobile phone.The exemplary implementation scheme of portable multifunction device includes but is not limited to come from Apple Inc. (Cupertino, California)Equipment, iPodEquipment andEquipment.Optionally Using other portable electronic devices, such as with the on knee of touch sensitive surface (for example, touch-screen display and/or touch tablet) Computer or tablet computer.It is to be further understood that in some embodiments, which is not portable communication device, but Desktop computer with touch sensitive surface (for example, touch-screen display and/or touch tablet).
In the following discussion, a kind of electronic equipment including display and touch sensitive surface is described.However, should manage Solution, the electronic equipment optionally include other one or more physical user-interface devices, such as physical keyboard, mouse and/or Control stick.
The equipment usually supports various application programs, such as one or more of following: drawing application program, presentation are answered Application program, disk editor application program, spreadsheet applications, game are created with program, word-processing application, website Application program, telephony application, videoconference application, email application, instant message application program, body-building Support application program, photo management application program, digital camera applications program, digital video camcorder application program, webpage clear Look at application program, digital music player application and/or video frequency player application program.
The physical user-interface device that the various application programs executed in equipment optionally use at least one general, it is all Such as touch sensitive surface.One or more functions of touch sensitive surface and the corresponding informance being displayed in equipment are optionally for difference Application program is adjusted and/or changes, and/or is adjusted and/or changes in corresponding application programs.In this way, equipment shares Physical structure (such as touch sensitive surface) supports various answer optionally with intuitive for a user and clear user interface Use program.
The embodiment that attention is drawn to the portable device with touch-sensitive display.Figure 1A is shown according to one The block diagram of the portable multifunction device 100 with touch-sensitive display system 112 of a little embodiments.Touch-sensitive display 112 has When be called for convenience " touch screen ", and sometimes referred to as or be called " touch-sensitive display system ".Equipment 100 includes depositing Reservoir 102 (it optionally includes one or more computer readable storage mediums), Memory Controller 122, one or more It is processing unit (CPU) 120, peripheral device interface 118, RF circuit 108, voicefrequency circuit 110, loudspeaker 111, microphone 113, defeated Enter/export (I/O) subsystem 106, other input control apparatus 116 and outside port 124.Equipment 100 optionally includes one Or multiple optical sensors 164.Equipment 100 is optionally included for detection device 100 (for example, touch sensitive surface, such as equipment 100 touch-sensitive display system 112) on contact intensity one or more contact strength sensors 165.Equipment 100 is appointed Selection of land includes for generating one or more tactile output generators 167 of tactile output on the appliance 100 (for example, touch-sensitive Tactile output is generated in the touch-sensitive display system 112 of surface such as equipment 100 or the Trackpad 355 of equipment 300).These portions Part is communicated optionally by one or more communication bus or signal wire 103.
As used in the present specification and claims, " intensity " of the contact on term touch sensitive surface refers to touching The power or pressure (power of per unit area) of contact (for example, finger contact) on sensitive surfaces, or refer to connecing on touch sensitive surface The power of touching or the substitute (surrogate) of pressure.The intensity of contact has value range, which includes at least four different Be worth and more typically include up to a hundred different values (for example, at least 256).The intensity of contact optionally uses various methods Combination with various sensors or sensor determines (or measurement).For example, below touch sensitive surface or adjacent to touch sensitive surface One or more force snesors optionally for measurement touch sensitive surface on difference at power.In some specific implementations, Power measured value from multiple force sensors is combined (for example, weighted average) to determine estimated contact force.Similarly, it touches Pressure of the pressure-sensitive top of pen optionally for determining stylus on touch sensitive surface.Alternatively, it is detected on touch sensitive surface The size of contact area and/or its variation, contact near touch sensitive surface capacitor and/or its variation and/or contact attached The power for the contact that the resistance of close touch sensitive surface and/or its variation are optionally used as on touch sensitive surface or the substitute of pressure. In some specific implementations, the substitute measurement of contact force or pressure, which is directly used in, determines whether to already exceed intensity threshold (example Such as, intensity threshold is described with corresponding to the unit of substitute measurement).In some specific implementations, contact force or pressure are replaced The power or pressure of estimation are converted into for object measurement, and the power or pressure estimated are used to determine whether to be more than intensity threshold (for example, intensity threshold is the pressure threshold measured with the unit of pressure).The intensity of contact is used to input as user Attribute, so that user be allowed to access original in the smaller equipment of user's limited area on the spot inaccessible additional set Standby function, the smaller equipment can indicate and/or receive user's input for showing (for example, on the touch sensitive display) (for example, via touch-sensitive display, touch sensitive surface or physical control/machinery control, such as knob or button).
As used in the specification and claims, term " tactile output ", which refers to, to utilize user's by user The equipment that sense of touch detects is opposite relative to physical displacement, the component (for example, touch sensitive surface) of equipment of the previous position of equipment In the displacement relative to the mass center of equipment of physical displacement or component of another component (for example, shell) of equipment.For example, In The component and user of equipment or equipment connect to sensitive surface (for example, other parts of finger, palm or user's hand) is touched In the case where touching, the tactile output generated by physical displacement will be construed to sense of touch by user, which corresponds to equipment or set The variation of the physical features of standby component perceived.For example, the movement of touch sensitive surface (for example, touch-sensitive display or Trackpad) " pressing click " or " unclamp and click " to physical actuation button is optionally construed to by user.In some cases, user will Feel sense of touch, such as " click is pressed " or " unclamp and click ", even if being physically pressed (example by the movement of user Such as, be shifted) physical actuation button associated with touch sensitive surface when not moving.For another example, though touch sensitive surface light When slippery is unchanged, the movement of touch sensitive surface also optionally can be explained by user or be sensed as touch sensitive surface " roughness ".Though Individuation sensory perception by user is limited such explanation of touch by right user, but is known many sense organs of touch Feel is that most users are shared.Therefore, when tactile output is described as the specific sensory perception corresponding to user (for example, " pressing Lower click ", " unclamp click ", " roughness ") when, unless otherwise stated, otherwise tactile output generated correspond to equipment or The physical displacement of its component, the physical displacement will generate the sensory perception of typical (or common) user.
It should be appreciated that equipment 100 is only an example of portable multifunction device, and equipment 100 optionally has There are components more more or fewer than shown component, optionally combines two or more components, or optionally there is this The different configurations of a little components or arrangement.Various parts shown in Figure 1A are with the combination of both hardware, software or hardware and software It realizes, including one or more signal processings and/or specific integrated circuit.
Memory 102 optionally includes high-speed random access memory, and also optionally includes nonvolatile memory, Such as one or more disk storage equipments, flash memory device or other non-volatile solid state memory equipment.Memory Controller 122 optionally controls the other component access memory 102 of equipment 100.
Peripheral device interface 118 can be used for the input peripheral of equipment and output peripheral equipment being couple to CPU 120 With memory 102.One or more of operations of processor 120 execute the various software programs stored in memory 102 And/or instruction set is to execute the various functions of equipment 100 and handle data.In some embodiments, peripheral device interface 118, CPU 120 and Memory Controller 122 are optionally implemented on one single chip such as chip 104.In some other realities It applies in scheme, they are optionally realized on independent chip.
RF (radio frequency) circuit 108 sends and receivees the RF signal for being also designated as electromagnetic signal.RF circuit 108 turns electric signal It is changed to electromagnetic signal/by electromagnetic signal and is converted to electric signal, and via electromagnetic signal and communication network and other communication equipments It is communicated.RF circuit 108 optionally includes the well known circuit for executing these functions, including but not limited to aerial system System, RF transceiver, one or more amplifiers, tuner, one or more oscillators, digital signal processor, encoding and decoding core Piece group, subscriber identity module (SIM) card, memory etc..RF circuit 108 optionally by wireless communication with network and other Equipment is communicated, the network be such as internet (also referred to as WWW (WWW)), Intranet and/or wireless network (such as Cellular phone network, WLAN (LAN) and/or Metropolitan Area Network (MAN) (MAN)).RF circuit 108 is optionally included for detecting near field The well known circuit for communicating the field (NFC), is such as detected by short-haul connections radio unit.Wireless communication optionally makes With any one of a variety of communication standards, agreement and technology, including but not limited to global system for mobile communications (GSM), enhanced Data GSM environment (EDGE), High Speed Uplink Packet access (HSUPA), is drilled high-speed downlink packet access (HSDPA) Into, clear data (EV-DO), HSPA, HSPA+, double unit HSPA (DC-HSPDA), long term evolution (LTE), near-field communication (NFC), It is wideband code division multiple access (W-CDMA), CDMA (CDMA), time division multiple acess (TDMA), bluetooth, Bluetooth Low Energy (BTLE), wireless Fidelity (Wi-Fi) is (for example, IEEE 802.11a, IEEE 802.11b, IEEE 802.11g, IEEE 802.11n and/or IEEE 802.11ac), voice over internet protocol (VoIP), Wi-MAX, email protocol are (for example, internet message access protocol (IMAP) and/or post office protocol (POP)), instant message (for example, scalable message processing and there are agreement (XMPP), for i.e. When message and there is Session initiation Protocol (SIMPLE), instant message and the presence service (IMPS) using extension), and/or it is short Messaging service (SMS), or include in this document submission date also it is untapped go out communication protocol any other is appropriate Communication protocol.
Voicefrequency circuit 110, loudspeaker 111 and microphone 113 provide the audio interface between user and equipment 100.Audio Circuit 110 receives audio data from peripheral device interface 118, audio data is converted to electric signal, and electric signal transmission is arrived Loudspeaker 111.Loudspeaker 111 converts electrical signals to the audible sound wave of human ear.Voicefrequency circuit 110 is also received by microphone 113 according to sound wave conversion come electric signal.Voicefrequency circuit 110 converts electrical signals to audio data, and audio data is transmitted To peripheral device interface 118 to be handled.Audio data is optionally retrieved from and/or is transmitted to by peripheral device interface 118 Memory 102 and/or RF circuit 108.In some embodiments, voicefrequency circuit 110 further includes earphone jack (for example, in Fig. 2 212).Earphone jack provides the interface between voicefrequency circuit 110 and removable audio input/output peripheral equipment, the periphery Equipment is the earphone such as only exported or has output (for example, single head-receiver or bi-telephone) and input (for example, Mike Wind) both headset.
I/O subsystem 106 is by such as touch screen 112 of the input/output peripheral equipment in equipment 100 and other input controls Equipment 116 is coupled to peripheral device interface 118.I/O subsystem 106 optionally includes display controller 156, optical sensor control Device 158 processed, depth camera controller 169, intensity sensor controller 159, tactile feedback controller 161, and it is used for other One or more input controllers 160 of input or control equipment.One or more of input controllers 160 are inputted from other Control equipment 116 receives electric signal/by electric signal and is sent to other described input control apparatus.Other input control apparatus 116 Optionally include physical button (for example, pushing button, rocker buttons etc.), dial, slide switch, control stick, click wheel etc.. Some optionally in embodiments, one or more input controllers 160 be optionally coupled to any one of the following terms (or It is not coupled to any one of the following terms): keyboard, infrared port, USB port and pointing device such as mouse.One Or multiple buttons (for example, 208 in Fig. 2) optionally include the volume control for loudspeaker 111 and/or microphone 113 Increase/reduction button.One or more buttons, which optionally include, pushes button (for example, 206 in Fig. 2).
Quickly pressing pushes button and optionally releases the locking to touch screen 112 or optionally begin to use on touch screen Gesture come the process that is unlocked to equipment, entitled " the Unlocking a such as submitted on December 23rd, 2005 The U.S. Patent application 11/322,549 of Device by Performing Gestures on an Unlock Image " is (i.e. United States Patent (USP) No.7,657,849) described in, which is incorporated by reference accordingly is incorporated to.Long-pressing pushing button (such as 206) equipment 100 is optionally made to be switched on or shut down.The function of one or more buttons is optionally customizable by a user.Touch screen 112 for realizing virtual push button or soft button and one or more soft keyboards.
Touch-sensitive display 112 provides the input interface and output interface between equipment and user.Display controller 156 is from touching It touches 112 reception electric signal of screen and/or sends the touch screen for electric signal.Touch screen 112 shows visual output to user. Visual output optionally includes figure, text, icon, video and any combination of them (being referred to as " figure ").Some In embodiment, the visual output of some visual outputs or whole is optionally corresponding with user interface object.
Touch screen 112 has the touch sensitive surface for receiving input from the user based on tactile and/or tactile contact, sensing Device or sensor group.Touch screen 112 and display controller 156 (with any associated module in memory 102 and/or refer to Enable collection together) contact (and any movement or interruption of the contact) on detection touch screen 112, and by detected contact The user interface object on touch screen 112 is converted to and is displayed on (for example, one or more soft keys, icon, webpage or figure Picture) interaction.In an exemplary embodiment, the contact point between touch screen 112 and user corresponds to the finger of user.
Touch screen 112 optionally uses LCD (liquid crystal display) technology, LPD (light emitting polymer displays) technology or LED (light emitting diode) technology, but other display technologies are used in other embodiments.Touch screen 112 and display controller 156 optionally using in the currently known or later a variety of touch-sensing technologies that will be developed any technology and other connect Nearly sensor array contacts and its appoints to detect for determining the other elements of the one or more points contacted with touch screen 112 What is mobile or interrupts, and a variety of touch-sensing technologies are including but not limited to capacitive, resistive, infrared ray and surface Technology of acoustic wave.In an exemplary embodiment, using projection-type mutual capacitance detection technology, such as Apple Inc. is being come from (Cupertino, California's)And iPodUsed in technology.
Touch-sensitive display in some embodiments of touch screen 112 is optionally similar to described in following United States Patent (USP) The touch-sensitive Trackpad of multiple spot: 6,323,846 (Westerman et al.), 6,570,557 (Westerman et al.) and/or 6,677, 932 (Westerman et al.) and/or U.S. Patent Publication 2002/0015024A1, each patent is accordingly in full with the side of reference Formula is incorporated to.However, touch screen 112 shows the visual output from equipment 100, and touch-sensitive Trackpad does not provide visual output.
In some embodiments, the touch-sensitive display of touch screen 112 is as the following request said: (1) on May 2nd, 2006 The U.S. Patent application No.11/381,313 of entitled " the Multipoint Touch Surface Controller " that submits； The U.S. Patent application of entitled " Multipoint Touchscreen " (multi-point touch panel) submitted on May 6th, (2) 2004 No.10/840,862；Entitled " the Gestures For Touch Sensitive Input submitted on July 30th, (3) 2004 The U.S. Patent application No.10/903,964 of Devices " (gesture for touch sensitive input devices)；(4) it is filed in 2005 1 The U.S. Patent application of entitled " the Gestures For Touch Sensitive Input Devices " on the moon 31 No.11/048,264；Entitled " the Mode-Based Graphical User submitted on January 18th, (5) 2005 Interfaces For Touch Sensitive Input Devices " (the pattern-based figure for touch sensitive input devices Shape user interface) U.S. Patent application No.11/038,590；On September 16th, (6) 2005 entitled " Virtual submitted Input Device Placement On A Touch Screen User Interface " (is placed on touch-screen user interface On virtual input device) U.S. Patent application No.11/228,758；It submits on September 16th, (7) 2005 entitled " Operation Of A Computer With A Touch Screen Interface " (calculating with touch screen interface The operation of machine) U.S. Patent application No.11/228,700；On September 16th, (8) 2005 entitled " Activating submitted Virtual Keys Of A Touch-Screen Virtual Keyboard " (virtual key of activation touch screen dummy keyboard) U.S. Patent application No.11/228,737；And entitled " the Multi-Functional that on March 3rd, (9) 2006 submits The U.S. Patent application No.11/367,749 of Hand-Held Device " (multi-functional hand-held device).All these application full text It is herein incorporated by reference.
Touch screen 112 is optionally with the video resolution for being more than 100dpi.In some embodiments, touch screen has The video resolution of about 160dpi.User is optionally come using any suitable object or additives such as stylus, finger etc. It is contacted with touch screen 112.It in some embodiments, is mainly to pass through contact and gesture based on finger by user-interface design Carry out work, since the contact area of finger on the touchscreen is larger, this may be accurate not as good as the input based on stylus.One In a little embodiments, equipment converts accurate pointer/cursor position or order for the rough input based on finger to be used to hold The desired movement of row user.
In some embodiments, in addition to a touch, equipment 100 optionally includes specific for activating or deactivating The Trackpad of function.In some embodiments, Trackpad is the touch sensitive regions of equipment, different from touch screen, the touch sensitive regions Do not show visual output.Touch tablet is optionally the touch sensitive surface separated with touch screen 112, or the touching formed by touch screen The extension of sensitive surfaces.
Equipment 100 further includes the electric system 162 for powering for various parts.Electric system 162 optionally includes electricity Power management system, one or more power supply (for example, battery, alternating current (AC)), recharging system, power failure detection circuit, Power converter or inverter, power supply status indicator (for example, light emitting diode (LED)) and with the electricity in portable device Power generates, managees, and distributes any other associated component.
Equipment 100 optionally further includes one or more optical sensors 164.Figure 1A, which is shown, is couple to I/O subsystem The optical sensor of optical sensor controller 158 in 106.Optical sensor 164 optionally includes charge-coupled device (CCD) or complementary metal oxide semiconductor (CMOS) phototransistor.Optical sensor 164 receive from environment by one or Multiple lens and the light projected, and convert light to indicate the data of image.In conjunction with (the also referred to as camera mould of image-forming module 143 Block), optical sensor 164 optionally captures still image or video.In some embodiments, optical sensor is located at equipment It is opposite facing with the touch-screen display 112 on equipment front on 100 rear portion, so that touch-screen display can be used as being used for The view finder of still image and/or video image acquisition.In some embodiments, optical sensor is located at the front of equipment On, so that the figure of the user is optionally obtained while checking other video conference participants on touch-screen display in user Picture, to be used for video conference.In some embodiments, the position of optical sensor 164 can be changed by user (for example, passing through Lens and sensor in slewing shell) so that single optical sensor 164 be used together with touch-screen display with In video conference and both still image and/or video image acquisition.
Equipment 100 optionally further includes one or more depth camera sensors 175.Figure 1A, which is shown, is couple to I/O The depth camera sensor of depth camera controller 169 in system 106.Depth camera sensor 175 receives data from environment To create the threedimensional model of the object in scene (for example, face) from viewpoint (for example, depth camera sensor).In some implementations In scheme, in conjunction with image-forming module 143 (also referred to as camera model), depth camera sensor 175 is optionally for determining by being imaged The depth map of the different piece of 143 captured image of module.In some embodiments, depth camera sensor is located at equipment 100 front, so that optionally obtaining tool while checking other video conference participants on touch-screen display in user There are the user images of depth information to be used for video conference, and captures the self-timer with depth map data.In some embodiment party In case, depth camera sensor 175 is located at the rear portion of equipment or the rear portion and front of equipment 100.In some embodiments In, the position of depth camera sensor 175 can be changed by user (for example, by lens and sensing in slewing shell Device) so that depth camera sensor 175 be used together with touch-screen display with for video conference and still image and/or Both video image acquisitions.
In some embodiments, depth map (for example, depth map image) include in scene to image distance viewpoint (example Such as, camera, optical sensor, depth camera sensor) the relevant information of distance (for example, value).In a reality of depth map It applies in scheme, each depth pixel defines the position where the corresponding two-dimensional pixel of its in the Z axis of viewpoint.In some embodiments In, depth map is made of pixel, wherein each pixel is defined by being worth (for example, 0 to 255).For example, " 0 " value indicates to be located at " three Pixel in dimension " scene apart from viewpoint (for example, camera, optical sensor, depth camera sensor) farthest, " 255 " value table Show the pixel being located in " three-dimensional " scene apart from viewpoint most nearby.In other embodiments, depth map indicates pair in scene As the distance between the plane with viewpoint.In some embodiments, depth map includes feeling emerging in the visual field about depth camera The information of the relative depth of the various features of interesting object is (for example, the eyes of user's face, nose, mouth, ear is relatively deep Degree).In some embodiments, depth map includes the information for enabling a device to determine the profile of object of interest in a z-direction.
Equipment 100 optionally further includes one or more contact strength sensors 165.Figure 1A, which is shown, is couple to I/O The contact strength sensor of intensity sensor controller 159 in system 106.Contact strength sensor 165 optionally includes one A or multiple piezoresistive strain instrument, capacitive force transducer, electrical force snesor, piezoelectric force transducer, optics force snesor, capacitor Formula touch sensitive surface or other intensity sensors are (for example, the sensing of the power (or pressure) for measuring the contact on touch sensitive surface Device).Contact strength sensor 165 receives contact strength information (for example, surrogate of pressure information or pressure information) from environment. In some embodiments, at least one contact strength sensor and touch sensitive surface (for example, touch-sensitive display system 112) juxtaposition Arrangement is neighbouring.In some embodiments, at least one contact strength sensor is located on the front of equipment 100 On the rear portion of the opposite facing equipment 100 of touch-screen display 112.
Equipment 100 optionally further includes one or more proximity sensors 166.Figure 1A, which is shown, to be couple to peripheral equipment and connects The proximity sensor 166 of mouth 118.Alternatively, proximity sensor 166 is optionally couple to the input control in I/O subsystem 106 Device 160.Proximity sensor 166 executes optionally as described in following U.S. Patent application: 11/241,839, it is entitled "Proximity Detector In Handheld Device"；No.11/240,788, entitled " Proximity Detector In Handheld Device " (proximity detector in handheld device)；No.11/620,702, it is entitled " Using Ambient Light Sensor To Augment Proximity Sensor Output " (use ambient light Sensor enhances proximity sensor output)；No.11/586,862, entitled " Automated Response To And Sensing Of User Activity In Portable Devices " (automated tos respond to and senses the use in portable device Family activity)；And No.11/638,251, entitled " Methods And Systems For Automatic Configuration Of Peripherals " (method and system automatically configured for peripheral equipment), these U.S. are special Benefit application is incorporated by reference accordingly to be incorporated herein.In some embodiments, when multifunctional equipment is placed in the ear of user When (for example, when user is carrying out call) near piece, proximity sensor closes and disables touch screen 112.
Equipment 100 optionally further includes one or more tactile output generators 167.Figure 1A, which is shown, is couple to I/O The tactile output generator of tactile feedback controller 161 in system 106.Tactile output generator 167 optionally includes one Or multiple electroacoustic equipments, such as loudspeaker or other acoustic components；And/or it converts the energy into the electromechanics of linear movement and sets Standby, such as motor, solenoid, electroactive polymerizer, piezoelectric actuator, electrostatic actuator or other tactiles export generating unit (for example, the component for converting the electrical signal to the tactile output in equipment).Contact strength sensor 165 is from haptic feedback module 133, which receive touch feedback, generates instruction, and it is defeated to generate the tactile that can be felt by the user of equipment 100 on the appliance 100 Out.In some embodiments, at least one tactile output generator and touch sensitive surface (for example, touch-sensitive display system 112) Alignment is neighbouring, and optionally by vertically (for example, to the surface inside/outside of equipment 100) or laterally (for example, In plane identical with the surface of equipment 100 rearwardly and a forwardly) the mobile touch sensitive surface exports to generate tactile.In some realities It applies in scheme, at least one tactile output generator sensor is located on the rear portion of equipment 100, with the front for being located at equipment 100 On touch-screen display 112 it is opposite facing.
Equipment 100 optionally further includes one or more accelerometers 168.Figure 1A, which is shown, is couple to peripheral device interface 118 accelerometer 168.Alternatively, accelerometer 168 is optionally couple to the input controller 160 in I/O subsystem 106. Accelerometer 168 executes optionally as described in following U.S. Patent Publication: U.S. Patent Publication 20050190059, Entitled " Acceleration-based Theft Detection System for Portable Electronic Devices " and U.S. Patent Publication 20060017692, entitled " Methods And Apparatuses For Operating A Portable Device Based On An Accelerometer ", the two U.S. Patent Publications are complete Text is herein incorporated by reference.In some embodiments, based on to from the received data of one or more accelerometers Analysis to show information on touch-screen display with longitudinal view or transverse views.Equipment 100 is optionally in addition to accelerometer It further include magnetometer and GPS (or GLONASS or other Global Navigation Systems) receiver except 168, for obtaining about setting Standby 100 position and the information of orientation (for example, vertical or horizontal).
In some embodiments, the software component being stored in memory 102 includes operating system 126, communication module (or instruction set) 128, contact/motion module (or instruction set) 130, figure module (or instruction set) 132, text input module (or instruction set) 134, global positioning system (GPS) module (or instruction set) 135 and application program (or instruction set) 136.This Outside, in some embodiments, memory 102 (Figure 1A) or 370 (Fig. 3) store equipment/overall situation internal state 157, such as Figure 1A With shown in Fig. 3.Equipment/overall situation internal state 157 includes one or more of the following terms: applications active state, Indicate which application program (if any) current active；Display state indicates what application program, view or other information Occupy each region of touch-screen display 112；Sensor states, including each sensor and input control apparatus from equipment 116 information obtained；And about the position of equipment and/or the location information of posture.
Operating system 126 is (for example, Darwin, RTXC, LINUX, UNIX, OS X, iOS, WINDOWS or embedded operation System such as VxWorks) include for control and manage general system task (for example, memory management, storage equipment control, Power management etc.) various software components and/or driver, and promote logical between various hardware componenies and software component Letter.
Communication module 128 is conducive to be communicated by one or more outside ports 124 with other equipment, and also Including for handling the various software components by RF circuit 108 and/or the received data of outside port 124.Outside port 124 (for example, universal serial bus (USB), firewire etc.) be suitable for be directly coupled to other equipment or indirectly by network (for example, Internet, Wireless LAN etc.) it is coupled.In some embodiments, outside port be with(the quotient of Apple Inc. Mark) in equipment used in 30 needle connectors are same or similar and/or spininess (for example, the 30 needles) connection that is compatible with Device.
Contact/motion module 130 optionally detect with touch screen 112 (in conjunction with display controller 156) and other touch-sensitive set The contact of standby (for example, Trackpad or physics click wheel).Contact/motion module 130 includes various software components for executing Various operations relevant to the detection of contact, such as to determine that whether being in contact (for example, detection finger down event), having determined Contact intensity (for example, contact power or pressure, or contact power or pressure substitute), determine whether there is contact It moves and tracks the movement on touch sensitive surface (for example, detecting one or more finger drag events) and determine that contact is It is no to have stopped (for example, detection digit up event or contact disconnect).Contact/motion module 130 is received from touch sensitive surface and is contacted Data.Determine that the movement of contact point optionally includes the rate (magnitude) of determining contact point, speed (magnitude and direction) and/or adds The movement of speed (change in magnitude and/or direction), the contact point is indicated by a series of contact data.These operations are optionally It is applied to single-contact (for example, single abutment) or multiple spot while contacting (for example, " multiple point touching "/multiple fingers connect Touching).In some embodiments, contact/motion module 130 detects the contact on Trackpad with display controller 156.
In some embodiments, contact/motion module 130 determines operation using one group of one or more intensity threshold Whether (for example, determining that whether user " clicks " icon) is executed by user.In some embodiments, according to software parameter To determine at least one subset of intensity threshold (for example, intensity threshold is not by the activation threshold of specific physical actuation device Lai really Fixed, and can be conditioned in the case where not changing the physical hardware of equipment 100).For example, not changing Trackpad or touch In the case where panel type display hardware, mouse " click " threshold value of Trackpad or touch screen can be configured to the big of predefined threshold value Any one threshold value in range.In addition, being provided to the user of equipment for adjusting one group of intensity threshold in some specific implementations In one or more intensity thresholds (for example, by adjusting each intensity threshold and/or by using being to " intensity " parameter Irrespective of size click comes the multiple intensity thresholds of Primary regulation) software setting.
Contact/motion module 130 optionally detects the gesture input of user.Different gestures on touch sensitive surface have difference Contact mode (for example, the different motion of detected contact, timing and/or intensity).Therefore, special optionally by detection Determine contact mode and carrys out detection gesture.For example, detection finger Flick gesture includes detection finger down event, then pressed with finger Detection finger is lifted and (is lifted away from) at (for example, at the position of icon) the identical position (or substantially the same position) of lower event Event.As another example, it includes detection finger down event that finger is detected on touch sensitive surface and gently sweeps gesture, is then detected One or more finger drag events, and then detection finger lifts and (is lifted away from) event.
Figure module 132 includes for the various known of figure to be presented and shown on touch screen 112 or other displays Software component, including the visual impact for changing shown figure (for example, brightness, transparency, saturation degree, contrast Or other perceptual properties) component.As used herein, term " figure " includes any object that can be displayed to user, non- It restrictively include text, webpage, icon (user interface object including soft key), digital picture, video, animation etc..
In some embodiments, figure module 132 stores the data for indicating figure ready for use.Each figure is optionally It is assigned corresponding code.Figure module 132 specifies one or more generations of figure to be shown from receptions such as application programs Code also receives coordinate data and other graphic attribute data together in the case of necessary, then generate screen image data with It is output to display controller 156.
Haptic feedback module 133 includes the various software components for generating instruction, and the instruction is by tactile output generator 167 use, and the one or more positions so as to the interaction in response to user and equipment 100 on the appliance 100 generate tactile Output.
The text input module 134 for being optionally the component of figure module 132 is provided in various application program (examples Such as, contact person 137, Email 140, IM 141, browser 147 and any other application program for needing text input) in Input the soft keyboard of text.
GPS module 135 determines the position of equipment, and provides the information for using in various application programs (for example, mentioning Phone 138 is supplied for using in location-based during dialing；It is supplied to camera 143 and is used as picture/video metadata；And it mentions Supply provides the application of location based service, such as weather desktop small routine, local Yellow Page desktop small routine and map/navigation Desktop small routine).
Application program 136 is optionally included with lower module (or instruction set) or its subset or superset:
Contact module 137 (sometimes referred to as address list or contacts list)；
Phone module 138；
Video conference module 139；
Email client module 140；
Instant message (IM) module 141；
Body-building support module 142；
For still image and/or the camera model 143 of video image；
Image management module 144；
Video player module；
Musical player module；
Browser module 147；
Calendaring module 148；
Desktop small routine module 149, optionally includes one or more of the following terms: weather desktop small routine 149-1, stock market desktop small routine 149-2, calculator desktop small routine 149-3, alarm clock desktop small routine 149-4, dictionary desktop The desktop small routine 149-6 of small routine 149-5 and other desktop small routines and user's creation for being obtained by user；
It is used to form the desktop small routine builder module 150 of the desktop small routine 149-6 of user's creation；
Search module 151；
Video and musical player module 152 merge video player module and musical player module；
Notepad module 153；
Mapping module 154；And/or
Online Video module 155.
The example for the other applications 136 being optionally stored in memory 102 includes other text processing application journeys Other picture editting's application programs, drawing application program, application program, the application program for supporting JAVA, encryption, number is presented in sequence Word rights management, speech recognition and speech reproduction.
In conjunction with touch screen 112, display controller 156, contact/motion module 130, figure module 132 and text input mould Block 134, contact module 137 is optionally for management address list or contacts list (for example, being stored in memory 102 or depositing In the application program internal state 192 of contact module 137 in reservoir 370), comprising: it is added to address list one or more Name；Name is deleted from address book；Telephone number, e-mail address, physical address or other information are associated with name；It will Image is associated with name；Name is sorted out and is classified；Telephone number or e-mail address are provided to initiate and/or promote Pass through phone 138, the communication of video conference module 139, Email 140 or instant message 141；Etc..
In conjunction with RF circuit 108, voicefrequency circuit 110, loudspeaker 111, microphone 113, touch screen 112, display controller 156, contact/motion module 130, figure module 132 and text input module 134, phone module 138 optionally for input with The electricity that the corresponding character string of telephone number, one or more telephone numbers in access contact module 137, modification have inputted Words number dials corresponding telephone number, conversates, and disconnecting or hang up when session is completed.As described above, channel radio Trust selection of land and uses any one of a variety of communication standards, agreement and technology.
In conjunction with RF circuit 108, voicefrequency circuit 110, loudspeaker 111, microphone 113, touch screen 112, display controller 156, optical sensor 164, optical sensor controller 158, contact/motion module 130, figure module 132, text input mould Block 134, contact module 137 and phone module 138, video conference module 139 include for according to user instructions come initiate, into Row and the executable instruction for terminating the video conference between user and other one or more participants.
In conjunction with RF circuit 108, touch screen 112, display controller 156, contact/motion module 130,132 and of figure module Text input module 134, email client module 140 include for creating, send in response to user instruction, receive and Manage the executable instruction of Email.In conjunction with image management module 144, email client module 140 to hold very much The easily Email of creation and transmission with the still image or video image shot by camera model 143.
In conjunction with RF circuit 108, touch screen 112, display controller 156, contact/motion module 130,132 and of figure module Text input module 134, instant message module 141 include the executable instruction for following operation: input corresponds to disappear immediately Character that the character string of breath, modification are previously entered, the corresponding instant message of transmission (for example, using short message service (SMS) or Multimedia messaging service (MMS) agreement with for the instant message based on phone, or using XMPP, SIMPLE or IMPS with In instant message Internet-based), receive instant message and check the received instant message of institute.In some embodiments In, the instant message transferred and/or received optionally include figure, photo, audio file, video file and/or MMS and/ Or other attachmentes supported in enhancing messaging service (EMS).As used herein, " instant message " refers to the message based on phone (for example, the message sent using SMS or MMS) and message Internet-based using XMPP, SIMPLE or IMPS (for example, sent out Both the message sent).
In conjunction with RF circuit 108, touch screen 112, display controller 156, contact/motion module 130, figure module 132, text This input module 134, GPS module 135, mapping module 154 and musical player module, body-building support module 142 include creation The executable instruction of body-building (for example, with time, distance and/or caloric burn target)；(movement is set with body-building sensor It is standby) it is communicated；Receive workout sensor data；Calibrate the sensor for monitoring body-building；Music is selected and played for body-building； And it shows, store and transmit workout data.
In conjunction with touch screen 112, display controller 156, one or more optical sensors 164, optical sensor controller 158, contact/motion module 130, figure module 132 and image management module 144, camera model 143 include being used for following operation Executable instruction: capture still image or video (including video flowing) and store them in memory 102, modify it is quiet The feature of state image or video deletes still image or video from memory 102.
In conjunction with touch screen 112, display controller 156, contact/motion module 130, figure module 132, text input module 134 and camera model 143, image management module 144 include the executable instruction for following operation: arrangement, modification (for example, Editor) or otherwise manipulate, tag, deleting, demonstrating (for example, in digital slide or photograph album) and storing static Image and/or video image.
In conjunction with RF circuit 108, touch screen 112, display controller 156, contact/motion module 130,132 and of figure module Text input module 134, browser module 147 include (including searching for, linking to browse internet for according to user instructions To, receive and display webpage or part thereof, and link to the attachment and alternative document of webpage) executable instruction.
In conjunction with RF circuit 108, touch screen 112, display controller 156, contact/motion module 130, figure module 132, text This input module 134, email client module 140 and browser module 147, calendaring module 148 include for according to Family instructs to create, show, modify and store calendar and data associated with calendar (for example, calendar, backlog Deng) executable instruction.
In conjunction with RF circuit 108, touch screen 112, display system controller 156, contact/motion module 130, figure module 132, text input module 134 and browser module 147, desktop small routine module 149 are optionally to be downloaded by user and used Miniature applications (for example, weather desktop small routine 149-1, stock market desktop small routine 149-2, calculator desktop small routine 149- 3, alarm clock desktop small routine 149-4 and dictionary desktop small routine 149-5) or by user creation miniature applications (for example, user create The desktop small routine 149-6 built).In some embodiments, desktop small routine include HTML (hypertext markup language) file, CSS (cascading style sheets) file and JavaScript file.In some embodiments, desktop small routine includes XML (expansible Markup language) file and JavaScript file be (for example, Yahoo！Desktop small routine).
In conjunction with RF circuit 108, touch screen 112, display controller 156, contact/motion module 130, figure module 132, text This input module 134 and browser module 147, desktop small routine builder module 150 are optionally used by a user in creation desktop Small routine (for example, user's specified portions of webpage are gone in desktop small routine).
In conjunction with touch screen 112, display controller 156, contact/motion module 130, figure module 132 and text input mould Block 134, search module 151 include for coming in searching storage 102 to match one or more search conditions according to user instructions The text of (for example, search term that one or more user specifies), music, sound, image, video and/or alternative document can It executes instruction.
In conjunction with touch screen 112, display controller 156, contact/motion module 130, figure module 132, voicefrequency circuit 110, Loudspeaker 111, RF circuit 108 and browser module 147, video and musical player module 152 include allow user download and Play back the music recorded stored with one or more file formats (such as MP3 or AAC file) and other audio files Executable instruction, and for showing, presenting or otherwise play back video (for example, on touch screen 112 or via outer Portion port 124 connect external display on) executable instruction.In some embodiments, equipment 100 optionally includes The function of MP3 player such as iPod (trade mark of Apple Inc.).
In conjunction with touch screen 112, display controller 156, contact/motion module 130, figure module 132 and text input mould Block 134, notepad module 153 create and manage the executable instruction of notepad, backlog etc. including according to user instructions.
In conjunction with RF circuit 108, touch screen 112, display controller 156, contact/motion module 130, figure module 132, text This input module 134, GPS module 135 and browser module 147, mapping module 154 optionally for connecing according to user instructions It receives, display, modify and store map and data associated with map (for example, at or near steering direction and specific position Shop and the related data of other points of interest and other location-based data).
In conjunction with touch screen 112, display controller 156, contact/motion module 130, figure module 132, voicefrequency circuit 110, Loudspeaker 111, RF circuit 108, text input module 134, email client module 140 and browser module 147, online Video module 155 includes the instruction for performing the following operation: allowing user to access, browsing, receives (for example, being passed by streaming It is defeated and/or downloading), playback (such as on the touchscreen or on the external display connected via outside port 124), send Email with the link to specific Online Video, and otherwise manage one or more file formats such as H.264 Online Video.In some embodiments, using instant message module 141 rather than email client module 140 are sent to the link of specific Online Video.Other descriptions of Online Video application program can be that on June 20th, 2007 mentions Entitled " Portable Multifunction Device, Method, the and Graphical User Interface handed over The U.S. Provisional Patent Application 60/936,562 of for Playing Online Videos " and on December 31st, 2007 submit Entitled " Portable Multifunction Device, Method, and Graphical User Interface It is found in the U.S. Patent application 11/968,067 of for Playing Online Videos ", the content of the two patent applications It is incorporated by reference and is incorporated herein accordingly.
Above-mentioned each module and application program correspond to for executing above-mentioned one or more functions and in this patent Shen Please described in method (for example, computer implemented method as described herein and other information processing method) executable instruction Collection.These modules (for example, instruction set) need not realize with independent software program, process, or module, and therefore these modules Each subset is optionally combined in various embodiments or is otherwise rearranged.For example, video player module is appointed Selection of land and musical player module are combined into individual module (for example, video and musical player module 152 in Figure 1A).One In a little embodiments, memory 102 optionally stores the subset of above-mentioned module and data structure.In addition, memory 102 is optionally Store the other module and data structure being not described above.
In some embodiments, equipment 100 is uniquely to be predefined thereon by touch screen and/or Trackpad to execute One group of function operation equipment.By using touch screen and/or Trackpad as the main input for operating equipment 100 Control equipment, the optionally quantity for being physically entered control equipment (for example, pushing button, driver plate etc.) in reduction equipment 100.
Predefined one group of function is uniquely performed by touch screen and/or Trackpad and is optionally included in user circle Navigation between face.In some embodiments, Trackpad when touched by user by equipment 100 from display on the appliance 100 Any user interface navigation to main menu, home menu or root menu.In such embodiment, realized using touch tablet " menu button ".In some other embodiments, menu button is that physics pushes button or other are physically entered control and set It is standby, rather than touch tablet.
Figure 1B is to show the block diagram of the example components for event handling according to some embodiments.In some realities It applies in scheme, memory 102 (Figure 1A) or memory 370 (Fig. 3) include event classifier 170 (for example, in operating system 126 In) and corresponding application program 136-1 (for example, any of aforementioned applications program 137 to 151,155,380 to 390 is answered With program).
Event classifier 170 receive the application program 136-1 that event information is delivered to by event information and determination and The application view 191 of application program 136-1.Event classifier 170 includes event monitor 171 and event dispatcher module 174.In some embodiments, application program 136-1 includes application program internal state 192, the application program internal state It is movable for indicating when application program or while being carrying out is displayed on one or more current applications on touch-sensitive display 112 Views.In some embodiments, equipment/overall situation internal state 157 is by event classifier 170 for which (which to be determined Application program is currently movable a bit), and application program internal state 192 is used for determination for thing by event classifier 170 The application view 191 that part information is delivered to.
In some embodiments, application program internal state 192 includes other information, and one in such as the following terms Person or more persons: when application program 136-1 restores to execute recoverys information to be used, indicate just shown by application program 136-1 The information shown is ready for the user interface state information of the information shown by the application program, for allowing users to Back to the state queue and the prior actions taken of user of the previous state or view of application program 136-1 repetition/remove Sell queue.
Event monitor 171 receives event information from peripheral device interface 118.Event information includes about subevent (example Such as, user on touch-sensitive display 112 touches, a part as multi-touch gesture) information.Peripheral device interface 118 It is transmitted (to pass through from I/O subsystem 106 or sensor such as proximity sensor 166, accelerometer 168 and/or microphone 113 Voicefrequency circuit 110) received information.Peripheral device interface 118 includes showing from touch-sensitive from the received information of I/O subsystem 106 Show the information of device 112 or touch sensitive surface.
In some embodiments, event monitor 171 sends the request to peripheral equipment at predetermined intervals and connects Mouth 118.In response, 118 transmitting event information of peripheral device interface.In other embodiments, peripheral device interface 118 is only When there are significant events (for example, receiving the input higher than predetermined noise threshold and/or receiving is more than to predefine Duration input) when ability transmitting event information.
In some embodiments, event classifier 170 further includes hit view determination module 172 and/or life event Identifier determining module 173.
When touch-sensitive display 112 shows more than one view, hit view determination module 172 is provided for determining sub- thing The part software process where occurred in one or more views.View can be seen over the display by user Control and other elements are constituted.
The another aspect of user interface associated with application program is one group of view, otherwise referred to as applies journey herein Sequence view or user interface windows are wherein showing information and the gesture based on touch occur.Wherein detecting touch (corresponding application programs) application view optionally corresponds in the sequencing or view hierarchies structure of application program Sequencing is horizontal.For example, being optionally referred to as hit view in the floor level view for wherein detecting touch, and it is identified Event set to correctly enter is based in part on the hit view of initial touch optionally at least to determine, the initial touch is opened Primordium is in the gesture of touch.
It hits view determination module 172 and receives information relevant to the subevent of the gesture based on touch.Work as application program When with multiple views with hierarchical structure tissue, hit view determination module 172 will hit that view is identified as should be to sub- thing Minimum view in the hierarchical structure that part is handled.In most cases, hit view is to initiate subevent (for example, shape At the first subevent in the subevent sequence of event or potential event) in the floor level view wherein occurred.Once hit View be hit view determination module 172 identification, hit view just usually receive with its be identified as hit view it is targeted Same touch or the relevant all subevents of input source.
It is specific that life event identifier determining module 173 determines which or which view in view hierarchies structure should receive Subevent sequence.In some embodiments, life event identifier determining module 173 determines that only hit view should receive specific Subevent sequence.In other embodiments, life event identifier determining module 173 determines the physical location including subevent All views be active participation view, and it is thus determined that all views actively participated in should all receive specific subevent Sequence.In other embodiments, even if touch subevent is confined to region associated with a particular figure completely, point Higher view in level structure will still maintain view for active participation.
Event information is assigned to event recognizer (for example, event recognizer 180) by event dispatcher module 174.It is wrapping In the embodiment for including life event identifier determining module 173, event information is delivered to by living by event dispatcher module 174 Dynamic 173 definite event identifier of event recognizer determining module.In some embodiments, event dispatcher module 174 exists Event information is stored in event queue, which is retrieved by corresponding event receiver 182.
In some embodiments, operating system 126 includes event classifier 170.Alternatively, application program 136-1 packet Include event classifier 170.In other embodiments, event classifier 170 is standalone module, or to be stored in memory A part of another module (such as contact/motion module 130) in 102.
In some embodiments, application program 136-1 includes multiple button.onreleases 190 and one or more application Views 191, wherein each application view includes for handling the corresponding view occurred in the user interface of application program The instruction of touch event in figure.Each application view 191 of application program 136-1 includes one or more event recognitions Device 180.In general, corresponding application programs view 191 includes multiple event recognizers 180.In other embodiments, event recognition One or more event recognizers in device 180 are a part of standalone module, which is such as user interface tool The object of packet or application program the 136-1 therefrom higher level of inheritance method and other attributes.In some embodiments, accordingly Button.onrelease 190 includes one or more of the following terms: data renovator 176, object renovator 177, GUI update Device 178 and/or from the received event data 179 of event classifier 170.Button.onrelease 190 optionally with or call number It updates according to renovator 176, object renovator 177 or GUI renovator 178 using internal state 192.Alternatively, application program One or more application views in view 191 include one or more corresponding event processing routines 190.In addition, one In a little embodiments, one or more of data renovator 176, object renovator 177 and GUI renovator 178 are included in In corresponding application programs view 191.
Corresponding event recognizer 180 receives event information (for example, event data 179) from event classifier 170, and From event information identification events.Event recognizer 180 includes Event receiver 182 and event comparator 184.In some embodiment party In case, event recognizer 180 further include metadata 183 and event transmitting instruction 188 (its optionally include subevent delivering refers to Enable) at least one subset.
Event receiver 182 receives the event information from event classifier 170.Event information includes about subevent example As touched or touching mobile information.According to subevent, event information further includes additional information, the position of such as subevent.When When subevent is related to the movement touched, event information optionally further includes rate and the direction of subevent.In some embodiments In, event include equipment from an orientation rotate to another orientation (for example, rotate to horizontal orientation from machine-direction oriented, or vice versa ), and event information includes the corresponding informance of the current orientation (also referred to as equipment posture) about equipment.
Event information and predefined event or subevent definition are compared by event comparator 184, and being based on should Compare to determine event or subevent, or the state of determining or update event or subevent.In some embodiments, event Comparator 184 includes that event defines 186.Event defines 186 definition (for example, predefined subevent sequence) comprising event, Such as event 1 (187-1), event 2 (187-2) and other events.In some embodiments, the sub- thing in event (187) Part includes for example touching start, touch and terminate, touch mobile, touch cancellation and multiple point touching.In one example, event 1 The definition of (187-1) is the double-click on shown object.For example, double-clicking includes the of predetermined duration on shown object The primary first time for touching (touch starts), predefining duration is lifted away from (touch terminates), is shown predefining on object Touch (touch starts) and predetermined duration is lifted away from (touch terminates) second of duration for the second time.In another example In, the definition of event 2 (187-2) is the dragging on shown object.For example, dragging includes predefining on shown object The touch (or contact) of duration, touch movement on touch-sensitive display 112 and touch be lifted away from (touch terminates).Some In embodiment, event further includes the information for one or more associated button.onreleases 190.
In some embodiments, it includes the definition for the event of respective user interfaces object that event, which defines 187,.One In a little embodiments, event comparator 184 executes hit test to determine which user interface object is associated with subevent.Example Such as, in the application view that three user interface objects are shown on touch-sensitive display 112, when on touch-sensitive display 112 When detecting touch, event comparator 184 executes hit test to determine which of these three user interface objects user Interface object is associated with touch (subevent).If each shown object and corresponding 190 phase of button.onrelease Association, then event comparator using the hit test as a result, to determine which button.onrelease 190 should be activated.Example Such as, the selection of event comparator 184 button.onrelease associated with the object of subevent and triggering hit test.
In some embodiments, the definition of corresponding event (187) further includes delay voltage, which postpones event The delivering of information, until having determined that whether subevent sequence exactly corresponds to or do not correspond to the event type of event recognizer.
It, should when the determining subevent sequence of corresponding event identifier 180 does not define any event in 186 with event to be matched 180 entry event of corresponding event identifier is impossible, event fails or event terminates state, ignores after this based on touch The subsequent subevent of gesture.In this case, for hit view keep other movable event recognizers (if there is Words) continue to track and handle the subevent of the gesture based on touch of lasting progress.
In some embodiments, corresponding event identifier 180 includes metadata 183, and the metadata has instruction thing Part delivery system how should execute the configurable attribute to the delivering of the subevent of the event recognizer of active participation, label and/ Or list.In some embodiments, metadata 183 include instruction event recognizer how to interact each other or how Configurable attribute, mark and/or the list interacted.In some embodiments, metadata 183 includes that instruction subevent is No configurable attribute, mark and/or the list for being delivered to view or the different levels in sequencing hierarchical structure.
In some embodiments, when one or more specific subevents of identification events, corresponding event identifier 180 Activate button.onrelease 190 associated with event.In some embodiments, corresponding event identifier 180 will be with event Associated event information is delivered to button.onrelease 190.Activation button.onrelease 190 is different from sending subevent (and delaying to send) hits view to corresponding.In some embodiments, event recognizer 180 is dished out related to institute's identification events The mark of connection, and button.onrelease 190 associated with the mark obtains the mark and executes predefined process.
In some embodiments, event delivery instruction 188 includes delivering the event information about subevent without activating The subevent delivery instructions of button.onrelease.On the contrary, event information is delivered to and subevent sequence by subevent delivery instructions Associated button.onrelease or the view for being delivered to active participation.With subevent sequence or with the view phase of active participation Associated button.onrelease receives event information and executes predetermined process.
In some embodiments, data renovator 176 creates and updates the data used in application program 136-1. For example, data renovator 176 is updated telephone number used in contact module 137, or to video player Video file used in module is stored.In some embodiments, object renovator 177 is created and is updated and applying Object used in program 136-1.For example, object renovator 177 creates new user interface object or more new user interface pair The position of elephant.GUI renovator 178 updates GUI.For example, GUI renovator 178 prepares display information and sends display information To figure module 132, for showing on the touch sensitive display.
In some embodiments, button.onrelease 190 includes data renovator 176, object renovator 177 and GUI Renovator 178, or with the access authority to the data renovator, the object renovator and the GUI renovator.In some implementations In scheme, data renovator 176, object renovator 177 and GUI renovator 178 are included in corresponding application programs 136-1 or answer In individual module with Views 191.In other embodiments, they are included in two or more software modules In.
It should be appreciated that the above-mentioned discussion of the event handling about user's touch on touch-sensitive display is applied also for using defeated Enter user's input that equipment carrys out the other forms of operating multifunction equipment 100, not all user's input is all in touch screen Upper initiation.For example, optionally pressing or pinning the mouse movement to cooperate and mouse button down with single or multiple keyboards；Touching The contact controlled on plate is mobile, tap, dragging, rolling etc.；Stylus input；The movement of equipment；Spoken command；The eye detected Eyeball is mobile；Biological characteristic input；And/or any combination of them is optionally used as corresponding to the son for limiting the event to be identified The input of event.
Fig. 2 shows the portable multifunction devices 100 with touch screen 112 according to some embodiments.Touch screen One or more figures are optionally shown in user interface (UI) 200.In the present embodiment and be described below its In his embodiment, user can by, for example, one or more fingers 202 (being not necessarily to scale in the accompanying drawings) or Person makes gesture on figure using one or more stylus 203 (being not necessarily to scale in the accompanying drawings) to select these figures In one or more figures.In some embodiments, when user is interrupted with the contact of one or more figures, will occur Selection to one or more figures.In some embodiments, gesture optionally includes one or many taps, primary or more The secondary rolling for gently sweeping the finger being in contact (from left to right, from right to left, up and/or down) and/or with equipment 100 Dynamic (from right to left, from left to right, up and/or down).In some specific implementations or in some cases, inadvertently with Pattern contact will not select figure.For example, being swept above application icon when gesture corresponding with selection is tap Gently corresponding application program will not optionally be selected by sweeping gesture.
Equipment 100 optionally further includes one or more physical buttons, such as " home " or menu button 204.Such as preceding institute It states, menu button 204 is optionally for any application navigate in one group of application being optionally performed on the appliance 100 136.Alternatively, in some embodiments, menu button is implemented as the soft key in the GUI being shown on touch screen 112.
In some embodiments, equipment 100 includes touch screen 112, menu button 204, for making device power-up/shutdown With pushing button 206, the one or more volume knobs 208, subscriber identity module (SIM) card slot for locking device 210, earphone jack 212 and docking/charging external port 124.Push button 206 optionally for by depress the button and The button, which is maintained at depressed state, continues predefined time interval to carry out machine open/close to equipment；By depressing the button And the button is discharged before in the past in the predefined time interval and carrys out locking device；And/or equipment is unlocked or is initiated Unlocking process.In alternative embodiment, equipment 100 is also received by microphone 113 for activating or deactivating The speech of function inputs.Equipment 100 also optionally includes the one or more of the intensity for detecting the contact on touch screen 112 Contact strength sensor 165, and/or export and send out for one or more tactiles that the user for equipment 100 generates tactile output Raw device 167.
Fig. 3 is the block diagram according to the exemplary multifunctional equipment with display and touch sensitive surface of some embodiments. Equipment 300 is not necessarily portable.In some embodiments, equipment 300 is laptop computer, desktop computer, plate Computer, multimedia player device, navigation equipment, educational facilities (such as children for learning toy), game system or control equipment (for example, household controller or industrial controller).Equipment 300 generally includes one or more processing units (CPU) 310, one A or multiple networks or other communication interfaces 360, memory 370 and for make the one or more of these component connections communicate it is total Line 320.Communication bus 320, which optionally includes, the circuit of the communication between system unit interconnection and control system component ( When referred to as chipset).Equipment 300 includes input/output (I/O) interface 330 with display 340, and the display is usual It is touch-screen display.I/O interface 330 also optionally includes keyboard and/or mouse (or other sensing equipments) 350 and Trackpad 355, for generating the tactile output generator 357 of tactile output in equipment 300 (for example, being similar to above with reference to Figure 1A institute The tactile output generator 167 stated), sensor 359 is (for example, optical sensor, acceleration transducer, proximity sensor, touch-sensitive Sensor and/or contact strength sensor (similar to the contact strength sensor 165 above with reference to described in Figure 1A)).Memory 370 include high-speed random access memory, such as DRAM, SRAM, DDR RAM or other random access solid state memory devices； And nonvolatile memory is optionally included, such as one or more disk storage equipments, optical disc memory apparatus, flash memory storage Device equipment or other non-volatile solid-state memory devices.Memory 370 optionally includes one or more positioned far from CPU 310 A storage equipment.In some embodiments, the memory of memory 370 storage and portable multifunction device 100 (Figure 1A) Similar program, module and the data structure or their subset of program, module and the data structure stored in 102.In addition, storage Device 370 is optionally stored in appendage, module and the data being not present in the memory 102 of portable multifunction device 100 Structure.For example, the memory 370 of equipment 300 optionally stores graphics module 380, module 382, word processing module is presented 384, website creation module 386, disk editor module 388 and/or spreadsheet module 390, and portable multifunction device 100 The memory 102 of (Figure 1A) does not store these modules optionally.
Each element in above-mentioned element in Fig. 3 is optionally stored in one or more of previously mentioned memory devices In a memory devices.Each module in above-mentioned module corresponds to the instruction set for executing above-mentioned function.Above-mentioned module or Program (for example, instruction set) need not be implemented as individual software program, process, or module, and therefore these modules is various Subset is optionally combined in various embodiments or otherwise rearranges.In some embodiments, memory 370 optionally store the subgroup of above-mentioned module and data structure.In addition, memory 370 optionally store it is not described above another Outer module and data structure.
It attention is drawn to the implementation for the user interface optionally realized on such as portable multifunction device 100 Scheme.
Fig. 4 A shows the menu for the application program on portable multifunction device 100 according to some embodiments Exemplary user interface.Similar user interface is optionally realized in equipment 300.In some embodiments, Yong Hujie Face 400 includes following elements or its subset or superset:
One or more signal strengths instruction of one or more wireless communication (such as cellular signal and Wi-Fi signal) Symbol 402；
Time 404；
Bluetooth indicator 405；
Battery Status Indicator 406；
The pallet 408 of icon with common application program commonly uses application icon such as:
The icon 416 for being marked as " phone " of zero phone module 138, the icon optionally include missed call or voice The indicator 414 of the quantity of message；
The icon 418 for being marked as " mail " of zero email client module 140, which, which optionally includes, does not read The indicator 410 of the quantity of Email；
The icon 420 for being marked as " browser " of zero browser module 147；And
Zero video and musical player module 152 (also referred to as iPod (trade mark of Apple Inc.) module 152) are marked It is denoted as the icon 422 of " iPod "；And
The icon of other application, such as:
The icon 424 for being marked as " message " of zero IM module 141；
The icon 426 for being marked as " calendar " of zero calendaring module 148；
The icon 428 for being marked as " photo " of zero image management module 144；
The icon 430 for being marked as " camera " of zero camera model 143；
The icon 432 for being marked as " Online Video " of zero Online Video module 155；
The icon 434 for being marked as " stock market " of zero stock market desktop small routine 149-2；
The icon 436 for being marked as " map " of zero mapping module 154；
The icon 438 for being marked as " weather " of zero weather desktop small routine 149-1；
The icon 440 for being marked as " clock " of zero alarm clock desktop small routine 149-4；
The icon 442 for being marked as " body-building support " of zero body-building support module 142；
The icon 444 for being marked as " notepad " of zero notepad module 153；And
Zero for being arranged the icon 446 for being marked as " being arranged " of application program or module, which is provided to equipment 100 And its access of the setting of various application programs 136.
It should be pointed out that icon label shown in Fig. 4 A is merely exemplary.For example, video and music player mould The icon 422 of block 152 is labeled " music " or " music player ".Other marks are optionally used for various application icons Label.In some embodiments, the label of corresponding application programs icon includes application corresponding with the corresponding application programs icon The title of program.In some embodiments, the label of application-specific icon is different from and the application-specific icon The title of corresponding application program.
Fig. 4 B is shown with the 451 (example of touch sensitive surface separated with display 450 (for example, touch-screen display 112) Such as, the tablet computer or Trackpad 355 of Fig. 3) equipment (for example, equipment 300 of Fig. 3) on exemplary user interface.Equipment 300 also optionally include one or more contact strength sensor (examples of the intensity for detecting the contact on touch sensitive surface 451 Such as, one or more sensors in sensor 359) and/or for for equipment 300 user generate tactile output one or Multiple tactile output generators 357.
Although following by being provided with reference to the input on touch-screen display 112 (being wherein combined with touch sensitive surface and display) Some examples in example, but in some embodiments, the input on touch sensitive surface that equipment detection is separated with display, As shown in Figure 4 B.In some embodiments, touch sensitive surface (for example, 451 in Fig. 4 B) have with display (for example, 450) the corresponding main shaft (for example, 452 in Fig. 4 B) of main shaft (for example, 453 in Fig. 4 B) on.According to these embodiments, Equipment detection is in the position corresponding to the corresponding position on display (for example, in figure 4b, 460 correspond to 468 and 462 pairs It should be in the contact with touch sensitive surface 451 of 470) place (for example, 460 in Fig. 4 B and 462).In this way, in touch sensitive surface (for example, figure 451 in 4B) it when being separated with the display of multifunctional equipment (450 in Fig. 4 B), is detected on touch sensitive surface by equipment User's input (for example, contact 460 and 462 and their movement) is used to manipulate the user interface on display by the equipment. It should be appreciated that similar method is optionally for other users interface as described herein.
In addition, though mostly in reference to finger input (for example, finger contact, singly refer to that Flick gesture, finger gently sweep gesture) To provide following example it should be appreciated that in some embodiments, one or more of these fingers input Finger input is by input (for example, input or stylus based on mouse input) substitution from another input equipment.For example, gently sweeping Gesture optionally clicks (for example, rather than contact) by mouse, be cursor later along the moving of the path gently swept (for example, and It is not the movement of contact) substitution.For another example, Flick gesture optionally by above the position that cursor is located at Flick gesture when mouse Click (for example, being off detection contact later instead of the detection to contact) substitution.Similarly, when being detected simultaneously by multiple use Family input when, it should be appreciated that multiple computer mouses be optionally used simultaneously or mouse and finger contact optionally by It uses simultaneously.
Fig. 5 A shows exemplary personal electronic equipments 500.Equipment 500 includes main body 502.In some embodiments, Equipment 500 may include relative to some or all of feature described in equipment 100 and 300 (for example, Figure 1A to Fig. 4 B) feature. In some embodiments, equipment 500 has the touch-sensitive display panel 504 of hereinafter referred to as touch screen 504.Alternatively or conduct The supplement of touch screen 504, equipment 500 also have display and touch sensitive surface.As the case where equipment 100 and 300, some In embodiment, touch screen 504 (or touch sensitive surface) is optionally included for detecting applied contact (for example, touch) intensity One or more intensity sensors.One or more intensity sensors of touch screen 504 (or touch sensitive surface) can provide expression The output data of the intensity of touch.The user interface of equipment 500 can make a response touch based on its intensity, it means that no Touch with intensity can call the different user interface operation in equipment 500.
Example technique for detecting and handling touch intensity is seen in for example following related application: 2013 5 Entitled " Device, Method, and Graphical User Interface for Displaying that the moon is submitted on the 8th The international patent application serial number of User Interface Objects Corresponding to an Application " PCT/US2013/040061 is issued as WIPO patent publication No. WO/2013/169849；And on November 11st, 2013 submits Entitled " Device, Method, and Graphical User Interface for Transitioning Between The international patent application serial number PCT/US2013/ of Touch Input to Display Output Relationships " 069483, it is issued as WIPO patent publication No. WO/2014/105276, which is incorporated by reference simultaneously accordingly Enter.
In some embodiments, equipment 500 has one or more input mechanisms 506 and 508.506 He of input mechanism 508 can be physical form (if including).The example for being physically entered mechanism includes pushing button and Rotatable mechanism. In some embodiments, equipment 500 has one or more attachment mechanisms.Such attachment mechanism (if including) can permit Perhaps by equipment 500 and such as cap, glasses, earrings, necklace, shirt, jacket, bracelet, watchband, bangle, trousers, belt, shoes, The attachment such as wallet, knapsack.These attachment mechanisms allow user's wearable device 500.
Fig. 5 B shows exemplary personal electronic equipments 500.In some embodiments, equipment 500 may include relative to Some or all of the component of Figure 1A, Figure 1B and Fig. 3 description component.Equipment 500 has bus 512, and the bus is by the part I/O 514 operatively couple with one or more computer processors 516 and memory 518.The part I/O 514 may be connected to aobvious Show device 504, which there can be touch sensing element 522 and optionally have intensity sensor 524 (for example, contact strength passes Sensor).In addition, the part I/O 514 can be connect with communication unit 530, for using Wi-Fi, bluetooth, near-field communication (NFC), bee Nest and/or other wireless communication techniques receive application program and operating system data.Equipment 500 may include input mechanism 506 And/or 508.For example, input mechanism 506 is optionally rotatable input equipment or pressable input equipment and rotatable defeated Enter equipment.In some instances, input mechanism 508 is optionally button.
In some instances, input mechanism 508 is optionally microphone.Personal electronic equipments 500 optionally include various Sensor, such as GPS sensor 532, accelerometer 534, orientation sensor 540 (for example, compass), gyroscope 536, movement pass Sensor 538 and/or combination thereof, all these equipment are operatively connected to the part I/O 514.
The memory 518 of personal electronic equipments 500 may include the one or more for storing computer executable instructions Non-transient computer readable storage medium, the executable instruction when by one or more computer processors 516 execute when for example Computer processor may make to execute following technologies, including process 700,900,1100,1300 and 1500 (Fig. 7 A to Fig. 7 F, figure 9A to Fig. 9 K, Figure 11 A to Figure 11 F, Figure 13 A to Figure 13 D and Figure 15 A to Figure 15 C).Computer readable storage medium can be Computer executable instructions can be physically include or stored so that instruction execution system, device and equipment use or in connection Any medium.In some instances, storage medium is transitory computer readable storage medium.In some instances, storage medium It is non-transient computer readable storage medium.Non-transient computer readable storage medium may include but be not limited to magnetic memory apparatus, Optical storage, and/or semiconductor storage.The example of such storage device includes disk, based on CD, DVD or blue light CD and persistence solid-state memory flash memory, solid state drive of technology etc..Personal electronic equipments 500 are not limited to figure The component of 5B and configuration, but may include other component or additional component in various configurations.
As used herein, term " showing can indicate " refers to optionally in equipment 100,300 and/or 500 (Figure 1A, Fig. 3 and figure 5A to Fig. 5 B) display screen display user's interactive graphical user interface object.For example, image (for example, icon), button It is optionally respectively formed with text (for example, hyperlink) and shows and can indicate.
As used herein, term " focus selector " refers to the user interface for being used to indicate that user is just interacting therewith The input element of current portions.In some specific implementations for including cursor or other positions label, cursor serves as " focus selection Device ", so that when cursor is above particular user interface element (for example, button, window, sliding block or other users interface element) Detect input (for example, pressing on touch sensitive surface (for example, touch sensitive surface 451 in touch tablet 355 or Fig. 4 B in Fig. 3) Input) in the case where, which is conditioned according to detected input.It is including that can be realized and touch The directly interactive touch-screen display of the user interface element on panel type display is touched (for example, the touch-sensitive display system in Figure 1A Unite 112 or Fig. 4 A in touch screen 112) some specific implementations in, the detected contact on touch screen serve as " focus choosing Select device " so that working as on touch-screen display in particular user interface element (for example, button, window, sliding block or other users Interface element) position at detect input (for example, by contact carry out pressing input) when, the particular user interface element root It is conditioned according to detected input.In some specific implementations, focus is moved to user from a region of user interface Another region at interface, the movement of the contact in correspondence movement or touch-screen display without cursor is (for example, by making Focus is moved to another button from a button with Tab key or arrow key)；In these specific implementations, focus selector It is moved according to movement of the focus between the different zones of user interface.The specific shape that focus selector is taken is not considered Formula, focus selector be usually interacted with expected from the user of user interface from user's control to deliver (for example, pass through to The user of equipment indicative user interface it is expected the element interacted) user interface element (or on touch-screen display Contact).For example, when detecting pressing input on touch sensitive surface (for example, touch tablet or touch screen), focus selector (example Such as, cursor, contact or choice box) position above the corresponding button will indicate that user it is expected to activate the corresponding button (rather than to set The other users interface element shown on standby display).
As used in the specification and in the claims, " characteristic strength " of contact this term refers to one based on contact The feature of the contact of a or multiple intensity.In some embodiments, characteristic strength is based on multiple intensity samples.Characteristic strength is appointed Selection of land is based on (for example, after detecting contact, before detecting that contact is lifted away from, detecting relative to predefined event Before or after contact start is mobile, before detecting that contact terminates, before or after detecting that the intensity of contact increases And/or detect contact intensity reduce before or after) in the predetermined period (for example, 0.05 second, 0.1 Second, 0.2 second, 0.5 second, 1 second, 2 seconds, 5 seconds, 10 seconds) during acquire predefined quantity intensity sample or one group of intensity sample. The characteristic strength of contact is optionally based on one or more of the following terms: maximum value, the intensity of contact of the intensity of contact Mean value, the average value of the intensity of contact, contact intensity preceding 10% at value, half maximum value of the intensity of contact, contact 90% maximum value of intensity etc..In some embodiments, the duration (example of contact is used when determining characteristic strength Such as, when characteristic strength is the average value of the intensity of contact in time).In some embodiments, by characteristic strength and one The one or more intensity thresholds of group are compared, to determine whether user has executed operation.For example, the group one or more intensity Threshold value optionally includes the first intensity threshold and the second intensity threshold.In this example, characteristic strength is less than first threshold Contact leads to the first operation, and characteristic strength is more than the first intensity threshold but is less than the contact of the second intensity threshold and leads to the second behaviour Make, and the contact that characteristic strength is more than second threshold causes third to operate.In some embodiments, using characteristic strength and one Comparison between a or multiple threshold values come determine whether to execute one or more operations (for example, be execute corresponding operating or Abandon executing corresponding operating) rather than the first operation or the second operation are executed for determining.
Fig. 5 C, which is shown, utilizes multiple contacts on multiple intensity sensor 524A-524D detection touch-sensitive display panel 504 552A-552E.Fig. 5 C further includes intensity map, shows the current strong of intensity sensor 524A-524D relative to volume unit Spend measured value.In this example, the intensity measurements of intensity sensor 524A and 524D are 9 volume units, and intensity The intensity measurements of sensor 524B and 524C are 7 volume units.In some specific implementations, integrated intensity is multiple strong The sum of the intensity measurements for spending sensor 524A-524D, are 32 volume units in this example.In some embodiments, Respective strengths, i.e. a part of integrated intensity are assigned in each contact.Fig. 5 D show based on it with 554 center of power away from Integrated intensity is distributed from and to contact 552A-552E.In this example, each contact in 552A, 552B and 552E is contacted Be assigned to the intensity of the contact of 8 volume units of integrated intensity, and contact in 552C and 552D it is each contact by Distribution is with the intensity of the contact of 4 volume units of integrated intensity.More generally say, in some specific implementations, each contact j Respective strengths Ij is all assigned according to predefined mathematical function Ij=A (Dj/ Σ Di), is a part of integrated intensity A, Wherein Dj is corresponding contact distance of the j away from power center, and Σ Di is all corresponding contacts (for example, i=1 is to the end) away from power The sum of the distance of the heart.The electronic equipment that can use similar or identical to equipment 100,300 or 500 is executed with reference to Fig. 5 C- figure Operation described in 5D.In some embodiments, one or more intensity of the characteristic strength of contact based on contact.In some realities It applies in scheme, intensity sensor is for determining single feature intensity (for example, the single feature intensity individually contacted).It should refer to Out, intensity map is not the part of shown user interface, and is included in Fig. 5 C- Fig. 5 D to help reader.
In some embodiments, identify a part of gesture for determining characteristic strength.For example, touch sensitive surface is optional Ground reception continuously gently sweeps contact, this is continuously gently swept contact from initial position transition and reaches end position, in the stop bits Place is set, contact strength increases.In this example, characteristic strength of the contact at end position is optionally based only upon continuously gently to sweep and connect A part of touching, rather than entirely gently sweep contact (for example, the only part for gently sweeping contact at end position).In some embodiment party In case, algorithm is smoothed to the intensity application for gently sweeping contact in the Optional for determining the characteristic strength of contact.For example, smoothing Algorithm optionally includes one of the following terms or a variety of: do not weight sliding average smoothing algorithm, triangle smoothing algorithm, Median filter smooths algorithm and/or exponential smoothing algorithm.In some cases, these smoothing algorithms, which eliminate, gently sweeps Narrow spike or recess in the intensity of contact, to realize the purpose for determining characteristic strength.
Detection intensity threshold value, light press intensity threshold, depth are optionally such as contacted relative to one or more intensity thresholds Intensity threshold and/or other one or more intensity thresholds are pressed to characterize the contact strength on touch sensitive surface.In some implementations In scheme, light press intensity threshold correspond to such intensity: under the intensity equipment will execute usually with click physics mouse Or the associated operation of button of touch tablet.In some embodiments, deep pressing intensity threshold corresponds to such intensity: In Equipment will execute the operation different from operation usually associated with the button for clicking physics mouse or Trackpad under the intensity.In In some embodiments, when detecting characteristic strength lower than light press intensity threshold (for example, and to be higher than Nominal contact detection strong Spend threshold value, the contact lower than Nominal contact detection intensity threshold value be no longer detected) contact when, equipment will touched according to contact Movement on sensitive surfaces carrys out moving focal point selector, associated with light press intensity threshold or deep pressing intensity threshold without executing Operation.In general, unless otherwise stated, otherwise these intensity thresholds are consistent between different groups of user interface attached drawing 's.
Contact characteristic intensity from the intensity lower than light press intensity threshold increase between light press intensity threshold and it is deep by Intensity between Compressive Strength threshold value is sometimes referred to as " light press " input.Contact characteristic intensity is from lower than deep pressing intensity threshold The intensity that intensity increases to above deep pressing intensity threshold is sometimes referred to as " deep pressing " input.Contact characteristic intensity is from lower than connecing The intensity of touching detection intensity threshold value increases to the intensity between contact detection intensity threshold value and light press intensity threshold sometimes Referred to as detect the contact on touch-surface.Contact characteristic intensity is reduced to low from the intensity for being higher than contact detection intensity threshold value Sometimes referred to as detect that contact is lifted away from from touch-surface in the intensity of contact detection intensity threshold value.In some embodiments, Contacting detection intensity threshold value is zero.In some embodiments, contact detection intensity threshold value is greater than zero.
Herein in some embodiments, in response to detecting gesture or response including corresponding pressing input One or more operations are executed in detecting using the corresponding pressing input of corresponding contact (or multiple contacts) execution, wherein extremely It is at least partly based on and detects that the intensity of the contact (or multiple contacts) increases to above pressing input intensity threshold value and detects Corresponding pressing input.In some embodiments, in response to detecting that it is strong that the intensity accordingly contacted increases to above pressing input Spend threshold value (for example, " downward stroke " of corresponding pressing input) Lai Zhihang corresponding operating.In some embodiments, pressing input Intensity including accordingly contacting increase to above pressing input intensity threshold value and the contact intensity be decreased subsequently to lower than by Press input intensity threshold value, and in response to detect the intensity accordingly contacted be decreased subsequently to lower than pressing input threshold value (for example, " up stroke " of corresponding pressing input) Lai Zhihang corresponding operating.
Fig. 5 E- Fig. 5 H shows the detection to gesture, and gesture includes and contacts 562 intensity from lower than the flicking in Fig. 5 E Compressive Strength threshold value is (for example, " ITL") intensity increase to above the deep pressing intensity threshold in Fig. 5 H (for example, " ITD") intensity Corresponding pressing input.Shown including the user interface of the application icon 572A-572D shown in predefined region 574 On 570, while with display highlighting 576 above 2 corresponding application icon 572B of application, benefit is detected on touch sensitive surface 560 The gesture executed with contact 562.In some embodiments, the detection gesture on touch-sensitive display 504.Intensity sensor detection The intensity of contact on touch sensitive surface 560.The equipment determines, contact 562 intensity pressing intensity threshold deeply (for example, “ITD") more than reach peak value.562 are kept in contact on touch sensitive surface 560.In response to detecting gesture, and according in gesture Period intensity is increased to deep pressing intensity threshold (for example, " ITD") more than contact 562, display recently for using 2 open text The expression 578A-578C (for example, thumbnail) that the scale of shelves reduces, as shown in Fig. 5 F- Fig. 5 H.In some embodiments, with One or more intensity thresholds are compared, and intensity is the characteristic strength of contact.It should be pointed out that not being for the intensity map of contact 562 The part of shown user interface, and be included in Fig. 5 E- Fig. 5 H to help reader.
In some embodiments, the display for indicating 578A-578C includes animation.For example, indicating that 578A is shown at the beginning Near application icon 572B, as illustrated in figure 5f.With the progress of animation, indicate that 578A is moved up, and in application icon 572B, which is nearby shown, indicates 578B, as shown in figure 5g.Then, indicate that 578A is moved up, 578B is upward towards expression 578A It is mobile, and expression 578C is shown near application icon 572B, as shown in Fig. 5 H.Indicate that 578A-578C forms icon Array above 572B.In some embodiments, animation is according to the strength development for contacting 562, as shown in Fig. 5 F- Fig. 5 G, Wherein indicate 578A-578C occur and with the intensity of contact 562 to deep pressing intensity threshold (for example, " ITD") increase and to Upper movement.In some embodiments, intensity based on animation progress is the characteristic strength of contact.Can use be similar to or The electronic equipment for being equal to equipment 100,300 or 500 executes the operation with reference to described in Fig. 5 E- Fig. 5 H.
In some embodiments, equipment is lagged using intensity to avoid the accident input sometimes referred to as " shaken ", Middle equipment limits or selection has the lag intensity threshold of predefined relationship (for example, lag intensity with pressing input intensity threshold value Threshold value than the low X volume unit of pressing input intensity threshold value, or lag intensity threshold be pressing input intensity threshold value 75%, 90% or some rational proportion).Therefore, in some embodiments, pressing input includes that the intensity accordingly contacted increases to height It is decreased subsequently in the intensity of pressing input intensity threshold value and the contact lower than the lag for corresponding to pressing input intensity threshold value Intensity threshold, and in response to detecting that the intensity accordingly contacted is decreased subsequently to lower than lag intensity threshold (for example, accordingly pressing Press " up stroke " of input) Lai Zhihang corresponding operating.Similarly, in some embodiments, contact only is detected in equipment Intensity from be equal to or less than lag intensity threshold intensity increase to be equal to or higher than pressing input intensity threshold value intensity and Optionally contact strength is decreased subsequently to be equal to or less than just to detect that pressing inputs when the intensity of lag intensity, and in response to Detect pressing input (for example, according to environment, contact strength increases or contact strength reduces) Lai Zhihang corresponding operating.
In order to be easy to explain, optionally, trigger in response to detecting any situation in following various situations to sound Operation Ying Yuyu pressing input intensity threshold value associated pressing input or executed in response to including the gesture of pressing input Description: contact strength increase to above pressing input intensity threshold value, contact strength from lower than lag intensity threshold intensity increase It is big to be decreased below pressing input intensity threshold value to the intensity, the contact strength that are higher than pressing input intensity threshold value, and/or contact strong Degree is decreased below lag intensity threshold corresponding with pressing input intensity threshold value.In addition, describing the operations as in response to inspection The intensity for measuring contact is decreased below pressing input intensity threshold value in the example that executes, is optionally in response to detect contact Intensity be decreased below and correspond to and be less than the lag intensity threshold of pressing input intensity threshold value and execute operation.
As used herein, " mounted application program " refer to have been downloaded into electronic equipment (for example, equipment 100,300 and/ Or 500) on and be ready in equipment start (for example, becoming opening) software application.In some embodiments, under The application program of load becomes mounted application program, software package of the mounted application program from downloading using installation procedure Extraction procedure part simultaneously integrates the part of extraction and the operating system of computer system.
As used herein, term " application program of opening " or " application program in execution " refer to believes with hold mode Cease the software application journey of (for example, a part as equipment/overall situation internal state 157 and/or application program internal state 192) Sequence.Application program opening or in execution is any one of optionally following kind of application program:
Applications active is currently displayed on the display screen for the equipment that the application program is used thereon；
Background application (or background process), do not show currently but one or more processes of the application program just It is handled by one or more processors；With
Pause or suspend mode application program, do not run but have be stored in memory (have respectively volatibility and It is non-volatile) in and the execution that can be used for restoring the application program status information.
As used herein, term " application program of closing " refers to the software application without hold mode information (for example, the status information for the application program closed is not stored in the memory of equipment).Therefore, closing application program includes Stop and/or remove the application program process of application program and removes the status information of application program from the memory of equipment. In general, it when in the first application program, opens the second application program and is not turned off the first application program.In display second When application program and shown the first application program stopping, the first application program becomes background application.
It attention is drawn in electronic equipment (such as portable multifunction device 100, equipment 300 or equipment 500) The embodiment of the user interface (" UI ") and associated process of upper realization.
Fig. 6 A to Fig. 6 AF show according to some embodiments for initiate and participate in real time communication session (for example, come From instant message application program) exemplary user interface.User interface in these attached drawings be used to show including Fig. 7 A extremely The process described below of process in Fig. 7 F.
Fig. 6 A shows the equipment 600 with touch-sensitive display 602.In some embodiments, equipment 600 includes equipment 100,300 and 500 one or more features.In some embodiments, equipment 600 further includes camera 603, and the camera is at least The imaging sensor of the data of a part (for example, visible light, infrared light or ultraviolet light) of spectrum is indicated including that can capture.In In some embodiments, camera 603 includes multiple images sensor and/or other kinds of sensor.In addition to capture indicates sense It surveys except the data of light, in some embodiments, camera 603 can capture other kinds of data such as depth data.Example Such as, in some embodiments, camera 603 also uses based on the technology of spot, flight time, parallax or focus and captures depth Data.Equipment 600 uses a part that 603 captured image data of camera include corresponding to the spectrum of the scene in viewing field of camera Data.In addition, in some embodiments, captured image data further include the depth data of light data.Some other In embodiment, captured image data include the number for being enough the data determination for the part of spectrum or generating depth data According to.
In some instances, electronic equipment 600 includes depth camera (for example, part as camera 603), such as infrared Camera, thermal imaging camera or combinations thereof.In some instances, which further includes light-emitting device (for example, light projector), all Such as IR floodlight, structured light projector or combinations thereof.Optionally, light-emitting device is used to pass through Visible Light Camera and depth Object is illuminated during camera (for example, IR camera) capture image, and the information from depth camera and Visible Light Camera is used for Determine the depth map of the different piece of the object captured by Visible Light Camera.In some embodiments, depth map is (for example, deep Spend figure image) it include in scene to information relevant at a distance from image distance viewpoint (for example, camera) (for example, value).In depth In one embodiment of figure, each depth pixel defines the position where the corresponding two-dimensional pixel of its in the Z axis of viewpoint.One In a little examples, depth map is made of pixel, wherein each pixel is defined by being worth (for example, 0 to 255).For example, " 0 " value indicates position Apart from the pixel of viewpoint (for example, camera) farthest in " three-dimensional " scene, " 255 " value indicates to be located at distance in " three-dimensional " scene The pixel of viewpoint most nearby.In other examples, depth map indicates the distance between object and plane of viewpoint in scene.In In some embodiments, depth map includes the relative depth of the various features of object of interest in the visual field about depth camera Information (for example, relative depth of the eyes of user's face, nose, mouth, ear).In some embodiments, depth map packet Include the information for enabling a device to determine the profile of object of interest in a z-direction.In some embodiments, using from two The parallax information of camera (for example, two Visible Light Cameras) show illuminating effect as described herein with after being used for image, and To image (for example, certainly before being used for using the depth information from depth camera in conjunction with the image data from Visible Light Camera Clap image).In some embodiments, true when determining depth information using two Visible Light Cameras and using depth camera When determining depth information, using identical user interface, to even if generate illuminating effect determining using completely different technology When use information when, also provide consistent experience for user.In some embodiments, the illumination effect is applied in display When the camera user interface of one of fruit, equipment detection camera switching shows the selection that can be indicated and from forward direction camera (for example, depth Camera and Visible Light Camera) be switched to rear Xiang Xiangji (for example, two Visible Light Cameras being separated from each other) (or vice versa), Keep simultaneously the display of user interface controls with application illuminating effect and by it is preceding show to the visual field of camera replace with after to phase The visual field of machine (vice versa).
As shown in FIG, equipment 600 shows the instant message user interface 604 of instant message application program.Immediately disappear Ceasing user interface 604 includes message region 606, which includes in the participant group including three or more participants The message 608A to 608D in message conversation between (" Dream Team ").
When showing instant message user interface 604, equipment 600 receives what instruction real time communication session was in active state Data (for example, invitation that real time communication session is added), the participant that wherein real time communication session can be used in " Dream Team " group. In some embodiments, real time communication session is voice communication session or audio/video communication session.
The data being in active state in response to receiving instruction real time communication session, the display instruction of equipment 600 are logical in real time The notice that letter session is in active state.As shown in Figure 6B, which includes as the participant in message region 606 The message of Pablo notifies 614, and instruction Pablo has initiated real time communication session.Notify 614 instructions have text " video " and The type of the real time communication session of video camera icon.614 are notified to also indicate currently in real time communication session in moving type The quantity (" 1 people is in active state ") of the participant of state.In some embodiments, notify 614 instructions currently real-time The name for the one or more participants being in active state in communication session.Notify that 614 further include that energy is shown in real time communication addition It indicates 614A, initiates equipment 600 for the process by notifying the real time communication session of 614 expressions to be added. It has been described in more detail below the technology for real time communication session to be added.
Optionally, the data being in active state in response to receiving instruction real time communication session, equipment 600 generate instruction The audio output that real time communication session is in active state.In some embodiments, audio output is different from and instant message The associated audio output of application program (for example, notice of received message) and audio associated with telephony application are defeated (for example, the tinkle of bells for incoming call) out.In some embodiments, audio output be audio associated with message informing and with The mixing of the associated audio of call-in reporting.In some embodiments, the sound of audio output and call and message informing Acoustic characteristic (for example, common tone, note, rhythm) is shared in both frequency output.
It similarly responds to receive the data that instruction real time communication session is in active state, equipment 600 shows logical in real time Letter, which shows, can indicate 610.In some embodiments, real time communication show can indicate 610 before real time communication session is initiated (example Such as, under the graying state) it is shown, and when real time communication session is initiated by visual emphasis (for example, being highlighted aobvious Show) or modified (for example, becoming non-graying state).In some embodiments, real time communication show can indicate 610 or option show It can indicate that the process for real time communication session to be added is initiated in 612 selection.In some embodiments, display notify 614 and/ Or real time communication show can indicate 610 without audio and/or tactile output (for example, telephone bell output) (for example, because it just It is shown in the alive messages dialogue that user may have paid attention to).
As shown in Figure 6 C, it as additional message is added to the message conversation in message region 606, notifies 614 to moving up Move the different location in message region 606.In some embodiments, equipment 600 will notify 614 be moved to message region it Outer position (for example, the banner for notifying 614 602 tops of display being transformed into except message region 606 from message).
In some embodiments, equipment 600 is mobile in response to satisfaction notice reorientation standard notifies 614.Some In embodiment, notice reorientation standard includes one or more standards selected from the following terms: the predetermined time goes over (for example, the time after display notifies 614 goes over), instant message user interface 604 stop display, instant message application journey Sequence is closed, and/or notify 614 whole or notify 614 a part be hidden in message region 606 (for example, by Message region 606 is filled in new incoming message).In some embodiments, notify 614 be moved to display 602 bottom, Right side or left side.In some embodiments, notify that 614 be to correspond directly to receive the instruction available number of real time communication session According to and display 612 near top show banner notice.
In some embodiments, equipment 600 receives the prompting data (example that the prompting of real time communication session is added in instruction Such as, the prompting sent in dialogue or after the predetermined period by participant).As shown in Figure 6 D, in response to connecing The data that the prompting of real time communication session is added in instruction are received, equipment 600, which is shown, full frame to be notified 616 and export audio output 618. In some embodiments, audio output 618 is different from audio output associated with instant message application program (for example, institute Receive the notice of message) and audio output associated with telephony application (for example, the tinkle of bells for incoming call).In some realities It applies in scheme, audio output 618 is the mixing of audio associated with message informing and audio associated with call-in reporting.In In some embodiments, audio output 618 is than audio output associated with telephony application (for example, the bell for incoming call Sound) it is short and longer than audio output associated with instant message application program (for example, notice of received message).One In a little embodiments, the tinkle of bells for incoming call includes the multiplicating of respective audio output, and audio output 618 is corresponding sound The single of frequency output repeats.In some embodiments, the audio output two of audio output 618 and call and message informing Person shares acoustic characteristic (for example, common tone, note, rhythm).In some embodiments, it is defeated to export tactile for equipment 600 Out.As shown in Figure 6 D, full frame to notify 616 titles (" Dream Team ") including group, currently in activity in real time communication session The name (" PABLO " and " STEPHEN ") of the participant of state, and initially initiate real time communication session or initiate the ginseng reminded With the name (" PABLO ") of person.In some embodiments, it is full frame notify 616 include be invited into real time communication session Participant name (for example, order that the group is added to according to them).It is full frame notify 616 further include for be added in real time The reply of communication session, which is shown, can indicate 620 (for example, having for real time communication session or display to be added immediately for being added in real time The menu of the option of communication session), for release it is full frame notify 616 " now not all right " shows can indicate 622, and for returning It is organized to instant message user interface 604 or to " Dream Team " and sends the message of message and show and can indicate 624.Full frame notice further includes figure Picture (for example, video of the user of the equipment 600 from camera 603) (for example, the preview of the image as user, once it is added, Other participants being utilized by the real time communication in session are seen).
As shown in Figure 6 D, the reception of equipment 600 (for example, detection) is shown with " now not all right " can indicate that 622 selection is corresponding User inputs 650A (for example, tap).As illustrated in fig. 6e, 650A is inputted in response to user, equipment 600 stops showing full frame notice 616 and return to instant message user interface 604.
As illustrated in fig. 6e, equipment 600, which receives (for example, detection) and shows with option, can indicate that the 612 corresponding user of selection is defeated Enter 650B (for example, tap).650B is inputted in response to receiving user, equipment 600 extends the mark of instant message user interface 604 Head is to include that audio shows that can indicate that 626A, video are added shows that can indicate that 626B and group details show can indicate 626C.
As fig 6 f illustrates, equipment 600 receives (for example, detection) and is added with video and shows and can indicate the corresponding use of the selection of 626B Family inputs 650C (for example, tap).In some embodiments, equipment 600 is direct in response to receiving user input 650B It is added real time communication session (for example, not needing additional input).In some embodiments, real time communication is added in equipment 600 Session and simultaneous transmission video and audio.In some embodiments, equipment 600 is added real time communication session and only transmits audio (for example, not transmitting video).In some embodiments, the selection that can indicate 626A is shown in response to audio, equipment 600 is with only sound Frequency is added real time communication session or initiates individual Conference calling to the participant of the group.
As shown in Figure 6 G, 650C is inputted in response to receiving user, equipment 600 shows the real-time of real time communication application program Communication interface 628.Real time communication interface 628 includes group (" Dream Team "), image (for example, the camera from 600 front of equipment The video of the user of the equipment 600 of (for example, camera 603)) instruction, be currently in active state in real time communication session The instruction of participant (" PABLO " and " STEPHEN ") initiates the instruction of the participant of real time communication session (" from PABLO's Video calling ") and Call- Control1 menu 630.Call- Control1 menu 630 shows including processing can indicate that 631, effect shows energy table Show 632, calling show can indicate 634 and menu show and can indicate 636.
As shown in Figure 6 G, equipment 600 shows that can indicate to receive (for example, detection) user on 636 inputs 650D in menu.Such as figure Shown in 6H, 650D is inputted in response to user, it is related to real time communication session to show that equipment 600 extends Call- Control1 menu 630 Additional information and control.In some embodiments, equipment 600 in response to processing show can indicate to start near 631 to On gently sweep and extend Call- Control1 menu 630.When expanded, Call- Control1 menu 630 further includes that audio ON/OFF is shown and can be indicated 638, audio-source menu shows that can indicate that 640, video ON/OFF is shown can indicate (the finger with participant's quantity in group of 642, group name 644 Show), group message is shown can indicate 646, includes participant corresponding to the group shows the list 648 that can indicate 648A to 648E, with And addition participant shows and can indicate 652.In some embodiments, show group message the choosing that can indicate 646 in response to receiving It selects, the starting of equipment 600 and/or display instant message user interface 604 (for example, Fig. 6 F), for being sent to the participant of the group Message.
As shown in figure 6h, showing for the participant in list 648 can indicate to include corresponding participant about real time communication session Communications status instruction.Indicate that 648A instruction participant Pablo is connected to real time communication session and is providing video counts According to (e.g., including the real-time media stream of video and audio data).Indicate that 648B instruction participant Stephen has connected and provides Audio data (for example, without video).Indicate that 648C and 648D indicates respectively that participant Marcel and Allison are not attached to reality When communication session (for example, video and audio data are not provided).Not connected participant or never addition real time communication session, It has been added before and has been then departed from real time communication session.Exemplary participant's communications status type is including only audio (for example, ginseng Audio is used only with person to communicate)；Video (for example, participant is just talked using video and audio)；Video pause (example Such as, the video pause of participant)；Video does not decode (for example, the problem of due to delay issue, video stream format etc., can not be handled Participant's video flowing)；Leave (for example, participant has left real time communication session)；And it waits to be added (for example, participant's quilt It invites participation real time communication session but real time communication session is not yet added).
In some embodiments, the participant of live video communication includes the ginseng for being currently connected to communication session of the group With the participant for being not currently connected to live video communication session of person and the group.That is, regardless of connection status, All participants of the group are referred to as the participant of real time communication session.When real time communication session is in active state, ginseng (disconnection) real time communication session can be added and (connect) or leave with person.
As shown in fig. 6i, equipment 600 receive (for example, detection) show with video ON/OFF can indicate 642 the corresponding use of selection Family inputs 650E (for example, tap).As shown in Fig. 6 J, in response to receive user input 650E, equipment 600 modify video open/ Pass, which shows, can indicate 642 to indicate video closed state, and by changing into the expression of the video camera in Fig. 6 I in Fig. 6 J The expression of phone, which is shown modifying calling, can indicate 634 to indicate only audio status.In only audio status, equipment 600 is with only sound Real time communication session (for example, equipment 600 does not provide video data) is added in frequency.
As shown in Fig. 6 J, equipment 600 receives (for example, detection) and shows and can indicate that the 634 corresponding user of selection is defeated with calling Enter 650F (for example, tap).As shown in fig. 6k, 650F is inputted in response to receiving user, equipment 600 is initiated the connection with only audio Real time communication session is added in mode.As shown in fig. 6k, equipment 600 stops the Call- Control1 menu 630 of display extension (for example, setting Standby 600 reduction Call- Control1 menu 630) and show connection screen 654.In some embodiments, equipment 600 is in response to video It is added and shows that the user that can be indicated on 626B inputs 650C, initiate the connection so that real time communication session is added and shows connection screen 654. In some embodiments, notify that 614 include showing and capable of indicating (for example, notifying " addition " icon on 614 or entirely notifying 614 It itself is selectable).In some embodiments, equipment 600 is initiated the connection in response to the selection to notifying 614 to be added Real time communication session simultaneously shows connection screen 654.
As shown in Fig. 6 L, after real time communication session is added in equipment 600, equipment 600 shows real time communication user interface 656, which includes the expression 658 of the user of equipment 600, participant Pablo (for example, initiating logical in real time Believe session participant) expression 670A and participant Stephen expression 670B.According to the communication shape indicated in list 648 The expression 670A of state, participant Pablo includes live video stream.Since participant Stephen only provides audio data, ginseng Expression 670B with person Pablo includes head portrait.In some embodiments, the cloth of the expression in real time communication user interface 656 Quantity (for example, user of two participant and equipment 600) of the office based on the participant for being currently connected to real time communication session.Figure Layout shown in 6L is referred to as 2 pair of 1 layout, and wherein first position corresponds to the position for indicating 670A, and the second position is corresponding In the position for indicating 670B.
In some embodiments, when real time communication session is added in equipment 600, equipment 600 determines the quantity of participant Whether ((movable and be invited to) for example, movable or total) is more than predetermined threshold number.It is super according to determining participant's quantity Threshold value is crossed, the display of equipment 600 has the overflow area overflowed and indicate (for example, thumbnail), as described in more detail below.Optionally Ground is less than threshold value according to determining participant's quantity, and equipment 600 is not shown with the overflow area for overflowing expression.
When real time communication session has been added in equipment 600, real-time media is streamed to other participant's (examples by equipment 600 Such as, the participant of real time communication session has been added).When real time communication session has been added simultaneously in participant associated with another equipment And equipment 600, when being also added, equipment 600 receives the real-time media stream for corresponding to the participant.
As shown in Fig. 6 L, when showing real time communication user interface 656, equipment 600 received on display 602 (for example, Detection) user's input 650G (for example, tap).As shown in Fig. 6 M, 650G is inputted in response to receiving user, equipment 600 is shown Call- Control1 menu 630.It indicates that the size of 670A and 670B reduces, and indicates the Xiang Shangyi on display 602 of 670B and 658 It is dynamic, so that Call- Control1 menu 630 is not Chong Die with expression.With expression 670A and 670B vertical alignment shown in Fig. 6 L and not The configuration of overlapping indicates that 670A and 670B is offset from one another and again on the contrary, when showing Call- Control1 menu 630 in Fig. 6 M It is folded.
As shown in fig. 6n, menu of the equipment 600 in Call- Control1 menu 630, which shows, can indicate to be received on 636 (for example, inspection Survey) user's input 650H (for example, tap).As shown in Fig. 6 O, 650H is inputted in response to receiving user, the extension of equipment 600 is exhaled Cry control menu 630 (for example, as discussed about Fig. 6 H).In some embodiments, equipment 600 is in Call- Control1 menu Display alarm shows and can indicate at the expression of participant for being not attached to real time communication session in 630.As shown in Fig. 6 O, list 648 instruction participant Marcel and Allison are not attached to real time communication session.Due to participant's Marcel and Allison (example Such as, equipment associated with participant Marcel and Allison) it is not attached to real time communication session, therefore the calling control extended Menu 630 processed includes that the tinkle of bells in list 648 respectively at the expression 648C and 648D of participant Marcel and Allison shows energy Indicate 662 and 664.Show in response to the tinkle of bells can indicate 662 or 664 selection, equipment 600 make at equipment provide and correspond to Selected the tinkle of bells shows the associated notice of the participant that can be indicated (for example, audio output, such as calling or the tinkle of bells).In some implementations In scheme, notice is the new notice different from the previous notification for being sent to participant.In some embodiments, new notice is than first Preceding notice more invasion property (for example, compared with the banner notice that no audio or tactile export, audio the tinkle of bells, tactile output or Full frame notice).In this way, user can remind participant's real time communication session that real time communication session is not yet added to be in and live It dynamic state and is added for participant (for example, being similar to the user of equipment 600 by full frame reminder announced as shown in Figure 6 D Mode).
As shown in Fig. 6 O, equipment 600 receive (for example, detection) show with video ON/OFF can indicate 642 the corresponding use of selection Family inputs 650I (for example, tap).650I is inputted in response to receiving user, equipment 600 provides real-time to real time communication session Video flowing (for example, camera in equipment 600) and modify video ON/OFF show can indicate 642 with indicate video open, such as scheme Shown in 6P.In Fig. 6 Q, equipment 600 receives (for example, detection) user on display 602 and inputs 650J (for example, light downwards Sweep) to reduce Call- Control1 menu 630.Use as shown in Fig. 6 R, when Call- Control1 menu 630 reduces, corresponding to equipment 600 The expression 658 at family includes the video of the camera 603 from equipment 600.
As shown in Fig. 6 S, equipment 600 receives (for example, detection) and shows and can indicate that the 634 corresponding user of selection is defeated with calling Enter 650K (for example, tap).650K is inputted in response to user, equipment 600 is disconnected with real time communication session.Fig. 6 T is shown The instant message user interface of the instant message application program shown after equipment 600 and real time communication session disconnect 604 embodiment.Although equipment 600 is disconnected from real time communication session, real time communication session holding is in active state. In some embodiments, as long as at least one of connection participant, even if initiating the participation of live video communication session Person leaves session, and real time communication session also keeps being in active state.As shown in Fig. 6 T, equipment 600 maintenance notify 614 and in real time It communicates and shows the display that can indicate 610, instruction real time communication session is in active state.Update notification 614 is to indicate real time communication The duration (for example, three minutes and 42 seconds, corresponding to the time quantum since initiating real time communication session) of session.
Display instruction real time communication session be in active state when notifying 614, equipment 600 receive instruction real time communication Session is no longer on the data of active state (for example, being over).In some embodiments, when the person of being not engaged in is connected to When real time communication session, real time communication conversation end is (for example, initiated or be added all participants of real time communication session not It reconnects).The data for being no longer on active state in response to receiving instruction real time communication session, it is logical that equipment 600 stops display Know 614 and/or update notification 614 to indicate that real time communication session is no longer on active state.Fig. 6 U is shown real-time The embodiment of the instant message user interface 604 for the instant message application program that communication session is shown after terminating is (for example, ring Ying Yu receives the data that instruction real time communication session is no longer on active state).As shown in Fig. 6 U, update notification 614 is to wrap Including real time communication session and being no longer on active state and remove real time communication shows the instruction that can indicate 610 (for example, text " is exhaled Cry end ").In some embodiments, real time communication, which is shown, can indicate 610 and/or notify that 614 is graying to indicate real time communication meeting Words are no longer on active state.In some embodiments, notify that 614 be no longer optional after real time communication conversation end 's.In some embodiments, after real time communication conversation end, 614 are notified to remain selectable and can be selected To initiate the new real time communication session with the group.In some embodiments, if real time communication meeting is not yet added in equipment 600 Words, then 600 update notification 614 of equipment is to indicate that user misses real time communication session (for example, " missed call ").
As shown in Fig. 6 U, equipment 600 receives (for example, detection) to be shown with return can indicate that the 611 corresponding user of selection is defeated Enter 650L (for example, tap).As shown in Fig. 6 V, 650L is inputted in response to receiving user, equipment 600 shows instant message application The user interface 671 of program comprising can indicate the list 673 of 673A to 673G for accessing showing for corresponding message dialogue.Show It can indicate the message pair that 673D corresponds between three participants of a group (user of Matthew, Danielle and equipment 600) Words.Showing can indicate that 673D includes real time communication indicator 675, and indicating the group, there are movable real time communication sessions.On the contrary, by It is not at active state in the real time communication session of " Dream Team ", therefore can indicate that 673A does not include corresponding to showing for " Dream Team " Real time communication indicator 675.
Turning now to Fig. 6 W, equipment 600 is illustrated as showing the interface in addition to instant message user interface 604.In Fig. 6 W In, equipment 600 shows lock-screen 668, and indicating equipment 600 is in the lock state.Lock-screen 668 include notice 677A and 677B.Notice 677A receives the message from user John Appleseed before indicating five minutes.Notice 677B expression no longer may be used Real time communication session (for example, the video calling missed) for addition.The video calling notice missed includes the person of being invited to participate in The state (" missing ") and real time communication session setup of (John, Amy and Rodrigo), real time communication session or end when Between (" before 1 hour ") instruction.In the locked state, equipment 600 does not show applications active.
When in a locked condition, equipment 600 receives the data that instruction real time communication session can be used for being added.In some realities It applies in scheme, 600 pairs of the equipment responses for receiving the data that instruction real time communication session can be used for being added depend on the upper of equipment 600 Hereafter, available real time communication session or other factors.In some embodiments, according to available reality between two participants When communication session (for example, One-to-one communication session), equipment 600 show the first kind notice.As shown in Fig. 6 X, the first kind is logical Know and notifies 672 and audio output 674 including full frame.In some embodiments, audio output 674 is different from logical with call Know associated audio output and audio output associated with message informing.In some embodiments, audio output 674 with Both audio output of call and message informing share acoustic characteristic (for example, common tone, note, rhythm).Optionally Ground, according to real time communication session available between three or more participants (for example, group real time communication session), equipment The notice of 600 display Second Types.As shown in Fig. 6 Y, the notice of Second Type includes that banner notifies 676 (for example, not being full frame) With tactile output 678 (for example, but there is no audio output).In some embodiments, according in three or more participants Between available real time communication session (for example, group real time communication session), equipment 600 is initially displayed full frame notice (for example, similar It is full frame shown in Fig. 6 E to notify 616), then show that (for example, being transformed into) banner notifies 676.In some embodiments, Notify that 676 include that can indicate (for example, as above the showing for process initiated when selected for corresponding real time communication session to be added Text is with reference to described in Fig. 6 F to Fig. 6 L).
With the notice 677B corresponding to the video calling missed on the contrary, notifying 676 to correspond to movable real time communication session. 676 are notified to include group (" Dream Team "), the participant (Pablo) of the movement for the displaying for executing trigger notice and real time communication meeting The instruction of the initiation time (" present ") of words.In some embodiments, 676 updates are notified accurately to reflect and initiate to lead in real time Believe the time of session and the state of real time communication session.In some embodiments, when the real time communication session for notifying 676 expressions At the end of, notify 676 text to be updated to " it is logical to miss the video from PABLO from " this is added the video calling from PABLO " Words ".
In some embodiments, according to determination do not show applications active (as shown in Fig. 6 W), equipment 600 in response to It receives the data that instruction real time communication session can be used for being added and shows full frame notice.In some embodiments, according to true Fixed to show applications active (for example, instant message application program), equipment 600 shows non-full frame notice (for example, banner Notice).
In some embodiments, it is sounded an alarm according to the user of true orientation equipment 600, equipment 600 exports alarm (example Such as, notice or the output of audio or tactile)；And it is not sounded an alarm to the user of equipment 600 according to determination, equipment 600 avoids defeated Alarm out.In some embodiments, the determination for issuing the user with alarm is based on interference standard (for example, equipment is in locking shape State or unlocked state).In some embodiments, (for example, as shown in Fig. 6 Y) is in the lock state according to equipment 600, if Standby output alarm (for example, tactile output 678)；And it is (for example, as shown in Fig. 6 Z) in the unlocked state according to equipment 600, if Standby 600 display notifies (for example, banner notifies 676) and avoids providing alarm (for example, exporting without tactile).
In some embodiments, whether interference standard instruction user is just seeing to equipment 600 (for example, based on from equipment The data of the sensor of 600 fronts).Fig. 6 AA shows the user 684 of gripping device 600.Watching attentively for user 680 is not directed to set Standby 600.Equipment 600 determines whether user 684 does not see to equipment 600, and is not seen according to determining user 684 to equipment 600, equipment 600, which provides, non-vision notifies 682 (for example, tactile or audio output).As shown in Fig. 6 AB, user's watches 680 fingers attentively To equipment 600.It is just being seen according to determining user 684 to equipment 600, equipment 600 avoids providing non-vision notice (for example, equipment 600 Show that banner notifies in the case where no tactile or audio output).In some embodiments, 600 use of equipment is configured Imaging sensor for the face of identification user determines whether user is just seeing to equipment 600.
Fig. 6 AC to Fig. 6 AF is gone to, equipment 600A includes display 602A (for example, touch-sensitive display), rotatable input machine Structure 601A and mechanical button 601B.In some embodiments, equipment 600A includes the feature of equipment 100,300,500 or 600. In some embodiments, equipment 600A communicates (for example, via two-way wireless communication) with equipment 600.In some embodiments In, equipment 600A and equipment 600 are in pairing relationship.
In some embodiments, equipment 600A (for example, from equipment 600), which is received, indicates that three or more participants can The data that real time communication session is in active state.As shown in Fig. 6 AC, in response to receiving at instruction real time communication session In the data of active state, equipment 600A shows user interface 604A, which includes that 605, real time communication is notified to be added Show can indicate 607 and releasing show and can indicate 609.In some embodiments, notify 605 include it is above-mentioned notify 614 feature (example Such as, participant's name, participant's quantity, trigger notice participant's name).In some embodiments, equipment 600A is responded Audio and/or tactile output are generated in receiving the data that are in active state of instruction real time communication session.In some implementations In scheme, the data that equipment 600A is in active state in response to receiving instruction real time communication session are generated raw with equipment 600 At audio and/or tactile export different audio and/or tactile output.In some embodiments, refer in response to receiving Show the primary data that real time communication session is in active state, is provided at equipment 600 (for example, not at equipment 600A) real-time The notice (for example, display, audio and/or tactile output) that communication session is in active state, and in response to receiving instruction The prompting data for the prompting that real time communication session is in active state all provide notice at equipment 600 and equipment 600A.One In a little embodiments, the primary data being in active state in response to receiving instruction real time communication session in equipment 600 and is set The notice (for example, display, audio and/or tactile output) that real time communication session is in active state all is provided at standby 600A, and And the prompting data in response to receiving the prompting that instruction real time communication session is in active state, at equipment 600 (for example, Not at equipment 600A) notice is provided.
In some embodiments, equipment 600A, which is received, shows the corresponding use of selection that can indicate 607 with real time communication addition Family inputs (for example, showing the tap that can be indicated on 617 in real time communication).Show and can indicate in response to receiving real time communication addition 607 selection, equipment 600A makes equipment 600 and/or equipment 600A is added real time communication session or initiates that real time communication meeting is added The process of words.In some embodiments, show the selection that can indicate 607, equipment 600A in response to receiving real time communication and being added Equipment 600 is set to show real time communication user interface 628 (Fig. 6 G), 654 (Fig. 6 K) or 656 (Fig. 6 L).
Fig. 6 AD shows the user interface 604A shown on equipment 600A after real time communication session is added.One In a little embodiments, equipment 600A includes providing the Mike of audio input to real time communication session via equipment 600A for user Wind.In Fig. 6 AD, user interface 604A include volume control 611, participant instruction 613 (" PABLO " and " 2 other People ") and real time communication session duration (" 9:45 "), terminate calling show that can indicate that 615, audio ON/OFF is shown can indicate 617 and menu show and can indicate 619.
In Fig. 6 AE, equipment 600A receives (for example, detection) user and inputs 650M (for example, 619 can be indicated by showing in menu At or near start upward gently sweep gesture).As shown in Fig. 6 AF, 650M, equipment 600A modification are inputted in response to receiving user Instruction 613 (" participate in numbers 4 ") simultaneously shows that keyboard shows that can indicate that 621, audio-source is shown can indicate 623 (for example, using equipment 600 Participate in real time communication session) and real time communication session participant list 625 (for example, be added and/or invite participation Person).In some embodiments, equipment 604A inputs 650M in response to receiving user in the user interface 604 in Fig. 6 AC (for example, before real time communication session is added) and show list 625.In some embodiments, list 625 includes participant Communications status instruction.In some embodiments, list 625 includes reminding to show and can indicate, can be selected as to not yet The participant that real time communication session is added sends alert notification.
Fig. 7 A to Fig. 7 F is the flow chart for showing the method according to some embodiments.In the equipment (example with display Such as, 100,300,500,600) place executes method 700.Some operations in method 700 are optionally combined, time of some operations Sequence is optionally changed, and some operations are optionally omitted.
As described below, method 700, which is especially provided, provides interactive mode for the real time communication session between multiple participants Notify the intuitive manner of (for example, 614,672,676).The method reduce users to use innovative technology between multiple participants Real time communication session provide notice cognitive load, to create more effective man-machine interface.For battery powered calculating Equipment, allow users to and notify quick interaction so that the real time communication session between multiple participants faster and more effectively, because This saves electric power and increases the time of battery charging room.
At frame 702, equipment shows the instant message user interface (for example, 604) of instant message application program.Immediately disappear Ceasing user interface (for example, 604) includes message region (for example, 606).The message region is included in three or more participants Between message conversation in multiple message (for example, 608A to 608D).
At frame 704, equipment receives the instruction available real time communication session of three or more participants (for example, audio Communication session or audio/video communication session) the first data for being in active state.In some embodiments, real time communication The participant of session has " being currently joined into " state (for example, providing real-time media stream to other participants；Also referred to as " have connected " Or " activity ") or " not being added currently " (for example, session is never added or has been added the participation that session is then departed from session before Person；Also referred to as " not connected " or " invitation ").In some embodiments, the participant of real time communication session include it is current The participant of real time communication session is added and the participant of real time communication session is not currently added (for example, 648A to 648D).When When real time communication session is in active state, participant is added (connection) or leaves (disconnection) real time communication session.When with it is another When real time communication session has been added in the associated participant of equipment, equipment receives the real-time media stream for corresponding to the participant.When When real time communication session has been added in the user of equipment, real-time media is streamed to other participants (for example, reality has been added by equipment When communication session participant).In some embodiments, as long as at least one of connection participant, real time communication meeting Words keep being in active state (for example, even if the participant for initiating real time communication session leaves session).
At frame 706, indicate that the real time communication session between three or more participants is in work in response to receiving First data of dynamic state, the notice (for example, 614) that equipment display instruction real time communication session is in active state.Some In embodiment, the notice that is in active state of instruction real time communication session include real time communication addition show and can indicate (for example, 614A).In some embodiments, notice includes the instruction of the quantity of the participant in real time communication session (for example, in session In the quantity of participant that is in active state；It is invited to the quantity of the participant of participant session).In some embodiments, Notice includes the instruction (for example, name, initials, photo or head portrait) of participant, executes triggering and shows over the display The movement of notice is (for example, initiate the participant of real time communication session, or selection " jingle bell again " button etc. is with alert device The participant of user's addition real time communication session).In some embodiments, in response to receiving three or more ginsengs of instruction The first data that real time communication session between person is in active state, equipment via equipment one or more audio output Equipment generates the audio output (for example, 618) that instruction real time communication session is in active state, wherein instruction real time communication session The audio output being in active state is different from and the associated audio output of phone call notification and associated with message informing Audio output.In some embodiments, audio output and call that instruction real time communication session is in active state It is shared acoustic characteristic (for example, common tone, note, rhythm) with both audio output of message informing.Display instruction is real-time The notice (for example, 614) and/or generation instruction real time communication session that communication session is in active state are in active state Audio output (for example, 618) provide a user the feedback of the current state about real time communication session, and provide a user view Feel feedback, if visual feedback instruction user's activation notification (for example, when notice is shown including addition to be indicated), will execute Certain operations associated with notice and/or real time communication session.Improved visual feedback, which is provided, for user enhances equipment Operability, and keep user-equipment interface more effective (for example, suitably inputting by helping user to provide and reducing operation User's mistake when equipment/interacted with equipment), this is further through allowing users to faster and more effectively subtract using equipment Lack electricity usage and extends the battery life of equipment.
Optionally, at frame 704, equipment shows that instruction real time communication session is at the first position in message region The notice (for example, 606) of active state.In some embodiments, the notice that instruction real time communication session is in active state (for example, 614) include that real time communication addition is shown and can be indicated (for example, 614A).
At frame 710, while notice (for example, 614) that display instruction real time communication session is in active state, if It is standby to receive the second data for indicating that the real time communication session between three or more participants is no longer on active state.
Optionally, at frame 712, equipment will indicate the notice that is in active state of real time communication session from first position (for example, bottom of message region 606) is moved to the second position (for example, the centre or top of message region 606, or display Top side, bottom side, right side and/or the left side of device (602)).In some embodiments, reorientation standard is notified in response to meeting, Mobile notice (for example, 614) occurs.In some embodiments, notice reorientation standard includes one or more standards, such as The time of predetermined volume go over (for example, display notice after time go over), instant message user interface (for example, 604) stop display, instant message application program is closed, due to new incoming message of filling message region (for example, 606) etc. And makes entirely to notify (for example, 614) or a part of (for example, 614) is notified to be hidden in message region (for example, 606).It moves The notice that dynamic instruction real time communication session is in active state provides a user the anti-of the current state about real time communication session Feedback, provides a user visual feedback, and visual feedback instruction will execute associated certain with notice and/or real time communication session Operation (if selection), and reduce and execute the required input quantity of operation.Improved visual feedback is provided for user to increase The strong operability of equipment, and keep user-equipment interface more effective (for example, by helping user to provide suitable input And user's mistake when reducing operation equipment/interact with equipment), this is further through allowing users to faster and more effectively make Reduce electricity usage with equipment and extends the battery life of equipment.The required input number of operation is executed in addition, reducing Amount also enhances the operability of equipment, and keeps user-equipment interface more efficient (for example, more available by providing for user Input and reduce the quantity of input), this reduces electric power further through allowing users to more rapidly and efficiently using equipment Using and improve the battery life of equipment.
Optionally, at frame 714 and 716, equipment receive correspond to real time communication addition show and can indicate (for example, 614A or First user of selection 626B) inputs (for example, 650C).Show and can indicate in response to receiving to correspond to real time communication and be added Selection the first user input, equipment be added real time communication session (for example, be added real time communication session and from electronic equipment simultaneously Transmit video and audio and/or addition real time communication session and only from electronic equipment audio (not transmitting video)).It rings Ying Yu is received to be shown the selection that can be indicated and real time communication is added reduces to execute and add including real time communication addition in the notification The quantity of input needed for entering the operation of real time communication session.That reduces that input quantity needed for executing operation also enhances equipment can Operability, and keep user-equipment interface more efficient (for example, by providing the available input of more users and reducing addition in fact When communication session required input quantity), this reduces electricity further through allowing users to more rapidly and efficiently using equipment Power uses and improves the battery life of equipment.
Optionally, in frame 718 and 720, equipment receives the prompting data (example that the prompting of real time communication session is added in instruction Such as, Fig. 6 D) (for example, the prompting sent in dialogue or after the predetermined period by participant).In response to connecing Receive instruction be added real time communication prompting prompting data (for example, remind by dialogue in participant send), equipment via One or more audio output apparatus of equipment generate audio output that instruction real time communication session is in active state (for example, 618) (for example, telephone bell output).In some embodiments, in conjunction with show again notice (for example, 616) generate audio it is defeated (for example, 618) out.In some embodiments, when receiving prompting data, the display of notice is being carried out.In some realities It applies in scheme, the audio output that instruction real time communication session is in active state is different from sound associated with phone call notification Frequency output and audio output associated with message informing.In some embodiments, instruction real time communication session is in activity The shared acoustic characteristic of both audio output of the audio output of state and call and message informing (for example, common tone, Note, rhythm).In some embodiments, initial audio output and instruction that instruction real time communication session is in active state The prompting audio output that real time communication session is in active state is (for example, show that the participant that can be indicated is raw based on selection " the tinkle of bells " At audio output) different (for example, shorter, quieter or more low-intensity).In some embodiments, real time communication meeting is indicated The single that the initial audio output being in active state is the repetition audio output for call is talked about to repeat and indicate in fact When the prompting audio output that is in active state of communication session be the repetition audio output for call multiplicating, or Person is identical as the audio output of call.In some embodiments, real time communication is added in response to receiving instruction Prompting prompting data, equipment display initiate remind real time communication session in participant instruction (for example, name, surname Name abbreviation, photo or head portrait).Generating the prompting including audio output when a given condition is fulfilled allows user to miss view at them Quickly identify that real time communication session is available when feeling notice (for example, 614).It has been satisfied in one group of condition without further User's input when execute optimized operation and enhance the operability of equipment, and make the more effective (example of user-equipment interface Such as, by help user provide suitably input and reduce operation equipment/with equipment interact when user's mistake), this is again Electricity usage faster and more effectively can be reduced using equipment by using family and extend the battery life of equipment.
It optionally, include that menu access shows the embodiment that can indicate (for example, 626B) in notice in frame 722 and 724 In, equipment, which receives, corresponds to the second user input (for example, 650C) that menu access shows the selection that can be indicated.The equipment in response to Receive correspond to menu access show the selection that can be indicated second user input, display including video addition option (for example, 634) (for example, show can indicate, drop-down menu, check box) and only audio option (for example, 642) (for example, show can indicate, pull down, Check box) menu (for example, 630).Show the selection that can be indicated in response to receiving menu access and show including for being added The menu of the different options of real time communication session provides additional control option without since the control of additional display keeps UI mixed and disorderly. Additional control option is provided without making UI just enhance the operability of equipment in a jumble due to adding the control shown, and is used Family-equipment interface it is more efficient (for example, by help user provide it is appropriate input and reduce operation equipment/when being interacted with equipment User's mistake), this reduces electricity usage and extends further through allowing users to more rapidly and efficiently using equipment The battery life of equipment.
Optionally, at frame 726 and 728, equipment receives third user input while showing menu (for example, 630). (for example, 650F) is inputted according to the third user for corresponding to the selection that option (for example, 634) are added in video, equipment use is from electricity Real time communication session is added in the audio and video of sub- equipment transmission.In some embodiments, device prompts user checks them Whether want with only audio and/or only video be added (for example, " you want with video addition ").When the defined condition (example of satisfaction Such as, option is added in response to video to be selected) when, real time communication session is added using from the audio and video of electronic equipment Allow user using audio and video be added real time communication session, without user select or modify equipment additional input and Output (for example, open camera and/or open microphone).It has been satisfied in one group of condition without further using Optimized operation is executed when family inputs and enhances the operability of equipment, and keeps user-equipment interface more effective (for example, logical Cross the user's mistake for helping user to provide when suitably inputting and reduce operation equipment/interact with equipment), this is further through making User faster and more effectively can be reduced electricity usage using equipment and extend the battery life of equipment.
Optionally, defeated according to third user corresponding with the addition selection of option (for example, 642) of only audio in frame 730 Enter, equipment is used to be added from the audio (for example, the audio recorded by one or more microphone of equipment) of electronic equipment Enter real time communication session, without from electronic equipment video data.In some embodiments, equipment shows user using only The indicator (for example, 658 in Fig. 6 L) of audio addition real time communication.This makes user feel comfortable, because video is not total to It enjoys).When meeting defined condition, allow user using only using real time communication session is added from the audio of electronic equipment Real time communication session is added in audio, and the additional of equipment is selected or modified without user and is output and input (for example, closing camera And/or open microphone).It has been satisfied in one group of condition without executing optimization when further user input Operation enhances the operability of equipment, and keeps user-equipment interface more effective (for example, suitable by helping user to provide Input and user's mistake when reducing operation equipment/interact with equipment), this is further through allowing users to more rapidly and have Effect ground is reduced electricity usage using equipment and extends the battery life of equipment.
In frame 732, indicate that the real time communication session between three or more participants is no longer in response to receiving Second data of active state, equipment stop display notice (for example, 614) or update notification to indicate real time communication session no longer It is in active state.In some embodiments, when real time communication session is in active state, update notification is to indicate in real time The current duration of communication session.In some embodiments, when real time communication session is no longer on active state, notice The instruction (for example, " end of calling ") that the duration of stopping display session, simultaneously display session terminated.Refer in response to receiving Show that real time communication session is no longer on the data of active state and stop display notice, provide a user visual feedback, instruction with Notice and/or the associated certain operations of real time communication session are no longer available.Improved visual feedback is provided for user to enhance The operability of equipment, and keep user-equipment interface more effective (for example, suitably inputting and subtracting by helping user to provide User's mistake when equipment/interact with equipment is operated less), this is further through allowing users to faster and more effectively using setting It is standby and reduce electricity usage and extend the battery life of equipment.In addition, in response to receiving instruction real time communication session It is no longer on the data of active state and stops display notice, meet one group of condition in the case where not further user input Shi Zidong executes operation.It has been satisfied in one group of condition and has increased without executing optimized operation when further user input The strong operability of equipment, and keep user-equipment interface more effective (for example, unwanted in user interface by reducing Element and user's mistake when helping user to provide suitable input and reducing operation equipment/interact with equipment), this Further through allowing users to faster and more effectively reduce electricity usage using equipment and extend the battery longevity of equipment Life.
Optionally, in frame 736,738 and 740, equipment receives the second real time communication session of instruction in two or more participations Available third data between person.It can be used between two participants according to the second real time communication session, equipment display instruction the The notice (for example, 672) (for example, full frame notice) for the first kind that two real time communications are in active state.In some embodiment party In case, the notice (for example, 672) of the first kind includes menu (for example, 630), has and is used for video (for example, 634), only sound Frequently the option of (for example, 643) and/or refusal (for example, 622).According to the second real time communication session in three or more participations It can be used between person, equipment shows the Second Type for notifying different the second real time communications of instruction to be in active state from the first kind It notifies (for example, 676).In some embodiments, the notice of Second Type is banner notice (for example, 676), is not full frame Or be not persistently full frame.Optionally, at frame 742, equipment is initially with first state (for example, full frame state 672) display The notice of Second Type.Optionally, in step 744, the notice (for example, 672) of Second Type is converted to the second state by equipment (for example, 676) (for example, lasting non-full frame state (for example, the persistence banner for being located at display top edge)).Based on what When meet rated condition and show the certain types of notice that is in active state of real time communication session, allow user to be based on defined Condition quickly identifies that real time communication is available while providing the interference of more and/or less reduction (for example, when two participations for user There are showing full frame notice when available real time communication session between person, and when between two participants there are it is available in real time Banner is shown when communication session).It has been satisfied in one group of condition without executing optimization when further user input Operation enhances the operability of equipment, and keeps user-equipment interface more effective (for example, suitable by helping user to provide Input and user's mistake when reducing operation equipment/interact with equipment), this is further through allowing users to more rapidly and have Effect ground is reduced electricity usage using equipment and extends the battery life of equipment.
Optionally, at frame 746,748,750 and 752, equipment receives instruction third real time communication session the available 4th Data.In response to receiving instruction available 4th data of third real time communication session, according to determining applications active (example Such as, application program (rather than for example, lock-screen (for example, 668) or main screen (for example, 1400) of equipment), such as immediately Message application or the third party application provided by the application developer in addition to the developer of operating system), The equipment display instruction available third type of third real time communication session is (for example, non-full frame state is (for example, be located at display Top edge banner)) notice (for example, 614 or 676).It is available in response to receiving instruction third real time communication session 4th data do not show applications active (for example, Fig. 6 W) that equipment provides instruction third real time communication session can according to determination The notice (for example, 678 or full frame notices (for example, 616)) of 4th type.It is shown in real time based on when meeting rated condition The specific type of communication session notifies, allows user to be based on defined condition and quickly identifies that real time communication is available while mentioning for user Interference for more and/or less reduction (for example, showing full frame notice when equipment shows application program, and is not shown in equipment Banner is shown when showing application program).It has been satisfied in one group of condition optimal without being executed when further user input Change operation and enhance the operability of equipment, and keeps user-equipment interface more effective (for example, by helping user to provide conjunction Suitable input and user's mistake when reducing operation equipment/interact with equipment), this further through allow users to more rapidly and Equipment is efficiently used and reduces electricity usage and extends the battery life of equipment.
Optionally, at frame 754,756,758 and 760, equipment receives the 4th real time communication session the available 5th of instruction Data.In response to receive instruction available 5th data of the 4th real time communication session, according to determine electronic equipment it is current on Hereafter (for example, the current mode of equipment is (for example, whether equipment is in interruption-free (DND) mode；Certain application programs or Whether hardware is operationally effective；Equipment is locked out or unlocks；Whether the one or more sensors of equipment detect user Just seeing to display) meet alarm sending standard, equipment shows the available notice (for example, 676) of the 4th real time communication session, together When the available non-vision notice (for example, 678) of the 4th real time communication session (for example, audio or tactile export) is provided.Some In embodiment, output or alarm correspond to notice (for example, 614).It can in response to receiving the 4th real time communication session of instruction 5th data are unsatisfactory for alarm according to the current context of determining electronic equipment and issue standard, and equipment shows that the 4th is real-time The available notice (for example, 614) of communication session, without provide the 4th real time communication session it is available it is non-vision notice (for example, 678).In some embodiments, when not issuing the alarm of the first kind, the alarm of Second Type is issued (for example, vision is alert Report, such as banner notice (for example, 676)).In some embodiments, the related context of equipment is that equipment is in locking shape State or unlocked state.In some embodiments, when the device is locked, issue the first kind alarm (for example, audio or Tactile output)；When equipment is unlocked (for example, when user's active operation equipment/interacted with equipment), do not issue (for example, suppression System) alarm.In some embodiments, the related context of equipment is whether equipment detects (for example, via one or more Sensor (for example, camera, infrared sensor)) user be see to display again without.In some embodiments, work as equipment When locked, the alarm (for example, audio or tactile output) of the first kind is issued.In some embodiments, when equipment detects When do not seen to user to display (for example, Fig. 6 AA), the alarm (for example, 682) of the first kind is generated.In some embodiment party In case, when user's active operation equipment/interact with equipment (for example, Fig. 6 AB), (for example, inhibition) first kind is not generated Alarm.
In some embodiments, it includes that equipment is locked to meet wanting for alarm sending standard that alarm, which issues standard, It asks.In some embodiments, it includes that equipment does not detect user to the attention of equipment to meet alarm that alarm, which issues standard, The requirement of sending standard.(for example, the user based on the input equipment with electronic equipment detected inputs, based on via note (for example, 680) are watched attentively depending on the user that tracking, feature tracking etc. detect).Based on when meeting rated condition display notice (example Such as, 614) and determine whether to provide non-vision notice, allow user to be based on rated condition and quickly identify that real time communication is available simultaneously More and/or less interference is provided for user.When one group of condition has been satisfied without further user input It executes optimized operation and enhances the operability of equipment, and keep user-equipment interface more effective (for example, by helping to use Family provides user's mistake when suitably inputting and reduce operation equipment/interact with equipment), this is further through allowing users to Faster and more effectively reduce electricity usage using equipment and extends the battery life of equipment.
It may be noted that the details above in relation to process described in method 700 (for example, Fig. 7 A to Fig. 7 F) can also similar side Formula is suitable for methods described below.For example, method 700 is optionally included below with reference to method 900,1100,1300 and 1500 The one or more features of the various methods.For example, method 1300 and 1500 can be used to start real time communication for method 700 Session, and start and run real time communication session to may include method 900 and 1100.For brevity, these details are under It is not repeated in text.
Fig. 8 A to Fig. 8 BK shows the exemplary user interface for real time communication session according to some embodiments. User interface in these attached drawings be used to show the process described below including the process in Fig. 9 A to Fig. 9 K.
Fig. 8 A shows equipment of the display for the instant message user interface 604 of the message conversation between group " Dream Team " 600.Real time communication session between the participant of the group is not at active state and (shows and can indicate for example, such as lacking real time communication Indicated by 610).As shown in Figure 8 A, the header of instant message user interface 604 is expanded to display video addition and shows and can indicate 626B (as described above).As shown in Figure 8 A, equipment 600 receives real-time between the participant of display participant group " Dream Team " The corresponding user of the request of the user interface of communication session inputs 850A (for example, showing in video addition can indicate light on 626B It hits).As shown in Figure 8 A, which includes starting the request of new live video communication session.In some embodiments, this is asked Ask the request (for example, as above for described in Fig. 6 F) including existing real time communication session is added.
650A is inputted in response to user, equipment 600 shows message 800 in message region 606 and initiating in real time The instruction 802 of communication session.In some embodiments, message 800 include it is above-mentioned notify 614 some features.Once initiating Real time communication session, equipment 600 just show real time communication user interface 804, simultaneously include the participation in real time communication session Multiple expressions of person.
It include participant's quantity greater than threshold value (for example, being more than according to the participant's quantity determined in real time communication session Five participants, the user including equipment 600), equipment 600 be greater than respective threshold size one or more sizes (for example, Three different sizes) more than first a participants of the display in addition to the user of equipment 600 simultaneously expression, and in an overflow region The expression of a participant more than second is shown, wherein a participant more than second includes one be not included in more than first in a participant Or multiple participants, and the expression of other the one or more participants shown in an overflow region is to be less than respective threshold ruler Very little size is shown.
In the embodiment shown in Fig. 8 A to Fig. 8 BK, the number of thresholds of participant is five participants (including equipment 600 user).Since group " Dream Team " includes 15 participants, equipment 600 determines the participation in real time communication session Person's quantity includes participant's quantity greater than threshold value.As shown in Figure 8 C, according to the determination, real time communication user interface 804 includes Four of four corresponding participants are main other than the user of the equipment 600 shown in canvas area 811 indicates 810A extremely 4 pair of 1 layout of 810D (being referred to as 810), the user of the camera view of equipment 600 indicate the 806 (use corresponding to equipment 600 Family), and with the participant corresponding to the real time communication session in addition to the user of equipment 600 and in canvas area 811 Its son that the spilling of the participant of expression indicates the overflow area 820 of 822A to 822J (being referred to as 822), shows at any time Collection is (for example, 822A to 822D).In some embodiments, layout is one in multiple predefined layouts, wherein participant Expression be anchored at the different location in user interface 804.Fig. 8 AW shows another embodiment of user interface 804. In the embodiment shown in Fig. 8 AW, overflow area 820 includes the real time communication session other than the user of equipment 600 The spilling of all participants indicates, so that the participant indicated in canvas area 811 also indicates in overflow area 820.
If participant's quantity in the group is less than or equal to five, according to the participant's number determined in real time communication session Amount is less than or equal to five, and equipment 600 shows the expression of the main expression of participant and the user of equipment 600, and in overflow area In do not show spilling indicate.
It is main to indicate that each of 810 predetermined indicate the display of one of size (small, medium and big) with three.It is main Indicate that 810A is shown with small size, it is main to indicate that 810B is shown with large scale, and mainly indicate 810C and mainly indicate 810D is shown with medium size.It is main to indicate that each of 810 is Chong Die at least one other main expression in Fig. 8 C. Overflowing indicates that 822 are all shown with identical size, and being less than indicates 810 associated small expression sizes with main.In illustration In embodiment, respective threshold size above-mentioned is greater than the size for overflowing expression 822 and is less than the small of main expression 810 Indicate size.User indicates that 806 are of the same size with expression 822 is overflowed, and is aligned or is shown in overflow area 820 In the overflow area.
As shown in Figure 8 C, once expression and a part are overflowed in display three completely at real time communication session subscriber interface 804 Overflowing indicates.The expression of remaining spilling is hiding, and can be shown by rolling overflow area 820 (for example, in response to The event that family inputs or detects, such as participant are actively engaged in by talk or movement).In Fig. 8 C, overflowing indicates 820D By partly shadowed overflow area 820 is indicated and/or can rolled to indicate to exist more spillings.In some embodiment party In case, the spilling in overflow area 820 indicates that 822 order depends on the order that real time communication session is added in participant.One In a little embodiments, the spilling in overflow area 820 indicates that 822 order changes over time (for example, due to for example overflowing expression User's selection or the participant that indicates in an overflow region be actively engaged in, when participant is added or leaves real time communication meeting When words or when participant is into and out canvas area 811, as described in greater detail below).In some embodiments In, the order that the spilling in overflow area 820 indicates is fixed (do not change at any time for example, the order indicated is overflowed in display Become).In some embodiments, the order of expression is overflowed based on the order of list associated with group (for example, in creation group Participant is added to the order of group).In some embodiments, the display (for example, real-time video feeding) of main expression 810 The higher rate of display to indicate 822 than spilling updates.
Fig. 8 C show after real time communication session setup soon and any participant (in addition to 600 phase of equipment Except associated participant) the real time communication session interface 804 before real time communication session is added.As shown in Figure 8 C, main table Showing 810 and overflowing indicates that 822 include the not placeholder with the participant of real time communication session connection.As shown in Figure 8 C, placeholder Including the expression with participant's name (for example, name or first and the last one initial) and the instruction of colored object. In some embodiments, the color of each colored object and/or pattern are different or selected from one group of predefined colors And/or pattern.Optionally, placeholder includes indicating that participant is waiting to connect to the status indicator of real time communication session, And (" the tinkle of bells ") can be indicated for showing to the prompting of the corresponding participant transmission available prompting of real time communication session.Some In embodiment, prompting show can indicate it is initial do not include on placeholder, and according to determination the predetermined period it Participant is corresponded to afterwards real time communication session is not added to show.
As shown in Figure 8 C, equipment 600 receives (for example, detection) user corresponding with the main expression selection of 810C and inputs 850A (for example, tap).As in fig. 8d, input 850A in response to receiving user, equipment 600 modify it is main indicate 810C with Including additional identification information (for example, the name of participant, Marcel) and for sending real time communication meeting to corresponding participant The prompting for talking about available prompting, which is shown, can indicate 808A.
Fig. 8 D, which is also shown, corresponds to the main participant's addition real time communication session for indicating 810B.In some embodiments In, equipment 600 receives instruction and corresponds to the data that real time communication session has been added in the main participant for indicating 822B.Such as Fig. 8 D institute Show, real time communication session is added in response to participant corresponding with main expression 822B, equipment 600 is with from corresponding participant's The main placeholder for indicating 822B of real time video data replacement.(for example, Fig. 8 AX) in some embodiments, equipment 600 is used to The spilling for corresponding to participant from the real time video data replacement of participant indicates 822B.As used herein, phrase is " from participation The real time video data of person " (or its equivalent) includes by equipment associated with participant (for example, smart phone, on knee Computer, desktop computer or tablet computer) or by with participant's interlock account (for example, email account or cloud storage account Family) transmission of associated equipment real time video data.
As in fig. 8d, equipment 600, which receives (for example, detection) and shows with prompting, can indicate that the 808 corresponding user of selection is defeated Enter 850B (for example, tap).In response to inputting 850B, equipment 600 sends instruction to cause newly to notify (for example, calling, the tinkle of bells Deng), instruction real time communication session can be used at the associated equipment of corresponding participant, wherein new notice is different from previously hair Give the notice of participant.In some embodiments, new notice invasion property more than previous notification is (for example, with no audio Or the banner notice of tactile output is compared, audio the tinkle of bells, tactile output or full frame notice).In Fig. 8 D, equipment 600 is also in master 808B can be indicated by indicating on 810D that display alarm shows, being corresponded to instruction mainly indicates that the participant of 810D is currently just being alerted Real time communication session it is available (for example, due to the prompting that another participant for being currently connected to real time communication session sends, such as by It is main to indicate the participant that 810B is indicated or the prompting being previously sent by the user of equipment 600).Equipment 600 also modifies prompting 808A can be indicated to indicate notifying corresponding participant (for example, by using " jingle bell " replacement " the tinkle of bells ", such as Fig. 8 E institute by showing Show).
As illustrated in fig. 8e, according to determine from real time communication session start through predefining the period, equipment 600 stop The size for indicating 822 and increasing user's expression 806 is overflowed in display.In some embodiments, when removing spilling indicates 822, It is main to indicate that 810 are resized and/or move.It is main to indicate that 810D is moved downward in the embodiment of illustration.One In a little embodiments, equipment 600 is less than or equal to according to the quantity of the determining participant for being currently joined into real time communication session to be participated in The number of thresholds (for example, five participants, user including equipment 600) of person overflows expression 822 to stop display.
In the time for corresponding to Fig. 8 E, real time communication session joined there are two participant.In some embodiments, In After further time quantum, equipment 600 stops main one indicated in 810 of display and by real time communication user interface 804 layout changes into 3 pair of 1 layout.In some embodiments, 4 pair of 1 layout and 3 pair of 1 layout are in one group of predetermined cloth In office, there is the similar staggered relatively of main expression 810.In some embodiments, having mainly indicates relatively similar Placement includes ought there are transition (for example, indicating position in a layout with the expression same between layout in this group Position overlapping in one group another layout) when, mainly indicate putting in the analogous location being maintained on user display 602 It sets.
In some embodiments, equipment 600 determines which, which to remove, mainly indicates based on the following factors: corresponding to participate in Whether person has been added real time communication session (for example, not removing expression if corresponding participant has been added), whether has reminded ginseng (for example, before predetermined time quantum after capable of indicating is shown in the corresponding prompting of selection, it will not remove and remind with person The expression of participant), participant in participant list placement (for example, in list earlier the expression of participant in list In participant later expression before remove) and placement of the expression in real time communication user interface 804 (for example, more Expression close to 602 top of display removes before lower expression on display 602).As shown in Figure 8 F, equipment 600 is stopped Only display is main indicates 810A and moves up main expression 810B (for example, to reduce the weight indicated between 810B and 810C It is folded), while the Relative vertical order for maintaining remaining to indicate (for example, indicating that 810B is greater than and is higher than indicates 810C, and indicates 810C, which is higher than, indicates 810D and with it with identical size).Fig. 8 F, which also shows prompting and shows, can indicate that 808A is modified to " again Jingle bell " indicates that corresponding participant has been alerted and is not alerted at least once and currently.Prompting show can indicate 808B also by Modify (for example, from " jingle bell " to " the tinkle of bells " and different colours) to indicate that corresponding participant had previously been alerted, but be not by The user reminding of equipment 600.
In the time for corresponding to Fig. 8 F, still only real time communication session joined there are two participant.In some embodiment party In case, after further time quantum, equipment 600 stops display, and another mainly indicates 810 and by real time communication user The layout at interface 804 changes into 2 pair of 1 layout (for example, 4 in the same group of predetermined layout shown in Fig. 8 E to Fig. 8 F To 1 layout and 3 pair of 1 layout).As shown in fig. 8g, equipment 600 stops showing main expression 810D and amplifies main expression 810B And 810C.Fig. 8 G shows three participants (for example, 2 pair 1) layout, wherein two other than the user of equipment 600 are joined Main expression with person is equivalent size, and a vertical alignment on top of the other.Remaining main indicates 810B and 810C Vertical order, wherein mainly indicate 810B mainly indicate 810C on.In the layout of Fig. 8 F, do not have between main expression Overlapping, and user indicates that 806 indicate that 810C is Chong Die with main.
In the time for corresponding to Fig. 8 G, still only real time communication session joined there are two participant.In some embodiment party In case, after further time quantum, equipment 600 stops display, and another is mainly indicated and by real time communication user interface 804 layout changes into 1 pair of 1 layout.As illustrated in figure 8h, equipment 600 stops the main expression 810C of display and will mainly indicate 810B is amplified to full-sized.In some embodiments, equipment 600 is shown preparatory true after capable of indicating 808A in selection prompting The layout of Fig. 8 H is changed at the timing area of a room from the layout of Fig. 8 G.Fig. 8 H shows two participants (for example, 1 pair 1) layout, In remaining mainly indicate 810B be displayed in full screen and by user indicate 806 overlapping.As illustrated in figure 8h, extremely scheme with Fig. 8 E respectively 4 pair of 1 layout, 3 pair of 1 layout shown in 8G, the main expression 810 in 2 pair of 1 layout and user indicate 806 square shape It compares, remaining mainly indicates that 810B and user indicate that 806 be the rectangle (for example, being orientated along profile) of vertical elongated.
Fig. 8 I is gone to, third participant Stephen joined real time communication session.As shown in fig. 81, in response to participant Real time communication session is added in Stephen, and equipment 600 is converted to 2 pair of 1 layout from the display of 1 pair of 1 layout of Fig. 8 H, wherein recently The main expression 810E of the participant of addition is on the expression 810B of existing participant (other than the user of equipment 600). In some embodiments, the display of equipment 600 notifies 812 and/or generates to indicate that real time communication session has been added in participant Audio or tactile output, as shown in fig. 81.In some embodiments, since real time communication is added as participant Stephen New main expression is shown when session, therefore equipment 600 avoids display from notifying 812 or generates audio or tactile output, this be because Mainly to indicate that 810E is used as the instruction that real time communication session has been added in participant.
Real time communication session is added with only audio mode in participant Stephen, and provides the real-time audio number of not video According to.The only voice communication state of participant Stephen is by mainly indicating 810E and notifying 812 instructions.It is main to indicate that 810E passes through The name and associated with participant head portrait for showing participant indicate only voice communication state.812 are notified to indicate with text Communications status (" this calling is added with audio ").As illustrated by figure 8j, equipment 600 stops display and notifies 812 (for example, in participant Stephen is added after the predetermined time quantum after real time communication session).
In some embodiments, it is being actively engaged in response to receiving instruction by the participant that only audio is added The data of (for example, speech), equipment 600 show visually indicating in the expression of the participant to talk.In Fig. 8 J, mainly Indicate that the presentation graphics (for example, head portrait) in 810E are animated (for example, having " voice circle ") to indicate participant Stephen is talking.
Fig. 8 K is gone to, the 4th participant Marcel joined real time communication session.As shown in figure 8k, it is participated in response to the 4th Real time communication session is added in person, and equipment 600 is converted to 3 pair of 1 layout (for example, as shown in Figure 8 F) from 2 pair of 1 layout, wherein recently The main expression 810F of the participant of addition existing participant (other than the user of equipment 600) main expression 810B and On 810E.
Real time communication session is added with video mode in participant Marcel, and provides real-time video and audio data.It participates in The video communication state of person Marcel is by mainly indicating that 810F is indicated.It is main to indicate 810F by display live video stream to indicate Video communication state.As shown in figure 8k, equipment 600 does not show that real time communication session has been added in instruction participant Marcel immediately Notice.
Fig. 8 L is gone to, the 5th participant Hoan joined real time communication session.As shown in Fig. 8 L, in response to participant Hoan Real time communication session is added, equipment 600 is converted to 4 pair of 1 layout from 3 pair of 1 layout, wherein the main table for the participant being recently joined Show 810G as the minimum top and right side mainly indicated and be located in main expression 810F.In some embodiments, according to Determine that participant Marcel and Hoan joined real time communication session in predetermined time window, equipment 600 is shown The combination that real time communication session has been added in instruction participant Marcel and Hoan notifies 816, as shown in Fig. 8 L.In some embodiment party In case, it is added according to determining participant with identical communications status (for example, video) to show that combination notifies 816.
Such as mainly indicating that the live video stream shown in 810G and the expression 816A of the video camera notified in 816 are signified Show, real time communication session is added with video mode in participant Hoan, so that equipment 600 receives reality associated with participant Hoan When video and audio data.
As shown in Fig. 8 L, (for example, detection) is received at the position of main expression 810F of the equipment 600 on display 602 User inputs 850C (for example, double-click).In some embodiments, as shown in Fig. 8 M, 850C is inputted in response to receiving user, Equipment 600 main will indicate 810F be amplified to it is maximum indicate size, and main will indicate that 810E narrows down to medium size and (while tieing up Hold 4 pair of 1 layout).As shown in Fig. 8 M, equipment 600 stops display and notifies 816 (for example, real time communication meeting is added in participant Hoan After predetermined time amount after words).
In some embodiments, the number for being actively engaged in real time communication session in response to receiving instruction participant According to dynamically changing the configuration of layout.In Fig. 8 M, equipment 600 receives instruction participant corresponding with main expression 810G The data that Hoan is being actively engaged in.As shown in Fig. 8 M, in response to receiving instruction participant corresponding with main expression 810G Main expression 810G is amplified to medium expression size from small expression size and will by the data being actively engaged in, equipment 600 It is main to indicate 810B from medium expression size reduction to small expression size.In this way, 4 pair of 1 layout maintains a small ruler Very little expression, two medium sizes indicate and a large scale indicates.
As shown in Fig. 8 M, (for example, detection) is received at the position of main expression 810G of the equipment 600 on display 602 User inputs 850D (for example, double-click).In some embodiments, as seen in fig. 8n, 850D is inputted in response to receiving user, Equipment 600 amplifies the foreground before the main main expression for indicating 810G and main expression 810G being taken to other displays.In addition to User indicates except 806 that the backstage after the main expression 810G of amplification is graying.
As shown in Fig. 8 O, equipment 600 shows that can indicate to receive (for example, detection) user on 818 inputs 850E (example in cancellation Such as, it taps).As shown in Fig. 8 P, 850E is inputted in response to receiving user, equipment 600 reduces the main size for indicating 810G simultaneously And be laid out display back to 4 pair 1 shown before user inputs 850D, the difference is that main indicate that 810G replaces being reduced Main expression 801F to medium expression size occupies large scale mainly and indicates.
Fig. 8 Q is gone to, real time communication session is added in the 6th participant.As in figure 8q, it is added in response to the 6th participant real When communication session, and be more than the predetermined threshold value of five participants according to the quantity of connected participant, equipment 600 is being overflow It shows that the spilling of the 6th participant indicates 822A and user's expression 806 is moved to overflow area 820 in region 820 out, and overflows Indicate that 822A is moved to user and indicates 806 left side out.As in figure 8q, overflowing indicates that 822A and user indicate that 806 are overflowing Align center in region 820.In some embodiments, equipment 600 is with the participant's (for example, the 6th participant) being newly added It indicates to replace the smallest main expression (for example, 810B), and shows the overflow table for the participant being replaced in overflow area 820 Show (for example, indicating that the size of 810B reduces and is moved to the position overflowed in overflow area 820 and indicate 822A).In some realities It applies in scheme, real time communication session is added in response to the 6th participant, and be more than five according to the quantity of connected participant The predetermined threshold value of participant, equipment 600 show that the participant being currently joined into or all participants are (current in an overflow region Sum being added is invited to) spilling expression, as shown in such as Fig. 8 AY.
As in figure 8q, overflow area 820 and main expression 810 are not overlapped.In some embodiments, in order to be overflow Region vacating space, equipment 600, which reduces size and/or vertical translation, one or more of mainly to be indicated.In some embodiment party In case, equipment 600 reduces the size that medium size mainly indicates and maintains small main expression and the big size mainly indicated.One In a little embodiments, equipment 600 reduces vertical interval and/or increases vertically superposed between main indicate.
As shown in Fig. 8 R, real time communication session is added in the 7th participant.Real time communication meeting is added in response to the 7th participant The spilling of the 7th participant in overflow area 820 is indicated that 822B is added to and overflows the left side for indicating 822A by words, equipment 600. As shown in Fig. 8 R, overflowing indicates that 822A and user indicate that 806 move right (compared with Fig. 8 Q) so that overflow indicate 822A and 822B and user's expression 806 align center in overflow area 820.In some embodiments, overflow indicate 822A and User indicates between 806, and the spilling of the 7th participant is indicated that 822B is added to and overflows the right side for indicating 822A.In Fig. 8 AZ institute In the embodiment shown, real time communication session is added in response to the 7th participant, equipment 600 is minimum with mainly indicating that 810I is replaced It is main to indicate (for example, 810H in Fig. 8 AY), and will indicate that 822F changes from placeholder corresponding to the spilling of participant Pablo Become live video stream.
As shown in Fig. 8 S, real time communication session is added in the 8th participant.Real time communication meeting is added in response to the 8th participant The spilling of the 8th participant in overflow area 820 is indicated that 822C is added to and overflows the left side for indicating 822B by words, equipment 600. As shown in Fig. 8 S, overflowing indicates that 822A and 822B and user indicate that 806 move right (compared with Fig. 8 Q), so that overflowing indicates 822A, 822B and 822C and the align center in overflow area 820 of user's expression 806.
As shown in Fig. 8 T, real time communication session is added in the 9th participant.Real time communication meeting is added in response to the 9th participant The spilling of the 9th participant in overflow area 820 is indicated that 822D is added to and overflows the left side for indicating 822C by words, equipment 600. As shown in Fig. 8 T, display, which is overflowed, completely indicates that 822D, 822C and 822B and user indicate 806.Being shown partially in spilling indicates 822A, wherein right part indicates that 806 is Chong Die with user.Overflow indicate 822A by shadowed (for example, have hatching effect), Middle spilling indicates that 822A meets user and indicates 806.In the embodiment of illustration, the ruler for indicating 822 and user's expression 806 is overflowed Very little holding is identical as the seven, the 8th and the 9th participant of real time communication session is added.Fig. 8 T shows the display combination of equipment 600 Notify 824, instruction participant Mary, Mick, John and Tom be added real time communication (respectively by overflow indicate 822A, 822B, 822C and 822D are indicated).In some embodiments, when additional participant (for example, exceeding the 9th participant) is added in fact When communication session when, the spilling of the participant that is newly added indicates to be added to the existing left end overflowed and indicated, and existing overflow table Show that right shift is that new spilling indicates vacating space, and user indicates that 806 keep being fully displayed in overflow area 820 The right side (for example, existing overflow the user's expression 806 for indicating to be rolled to right side " lower section ") of overflow area 820.Some In embodiment, the spilling for the participant being newly added indicates to be added to the existing right end for overflowing and indicating, wherein user indicates 806 Keep the right side for being fully displayed at overflow area 820.In some embodiments, if overflow area is fully occupied, no Display immediately is added to the expression of the new addition participant of the existing right end for overflowing and indicating (for example, overflowing in response to side roll to the left It indicates out and shows new spilling expression).
Fig. 8 U is gone to, equipment 600 receives instruction participant corresponding with expression 822B is overflowed and is actively engaged in real time communication meeting The data of words.In some embodiments, equipment 600 is in response to determining that the activity level for the participant being actively engaged in has increased Indicate that replacement mainly indicates with the participant being actively engaged in the activity level for being higher than participant representated by main indicate. In the embodiment of illustration, real time communication is actively engaged in response to receiving instruction participant corresponding with expression 822B is overflowed The data of session, equipment 600 indicate the minimum master for mainly indicating and showing and being replaced of replacement with the participant's being actively engaged in Indicate that the spilling of corresponding participant indicates.As shown in Fig. 8 U, mainly indicate 810B (the smallest main expression) by shadowed Or start to fade out；The spilling of the participant being actively engaged in indicates 822B by shadowed or starts to fade out and size reduces；And When new spilling corresponding with the main expression participant of 810B (the main expression being replaced) indicates 822E from a left side for display 602 Edge enters overflow area 820 to when overflowing the left side for indicating 822D, and overflowing indicates 822C and 822D right shift.In some realities It applies in scheme, it is main to indicate that 810B slightly amplifies (for example, to emphasize that it is just being replaced).As shown in Fig. 8 V to Fig. 8 W, main table Show the main expression 810H replacement for the participant that 810B is actively engaged in；Overflowing indicates that 822B continues to shrink until it is removed； And overflowing indicates that 822C and 822D continues right shift and indicates that 822E is fully displayed until overflowing, and overflows expression 822C Occupy the position previously occupied by the spilling expression 822B removed in overflow area 820.In Fig. 8 U to Fig. 8 W, overflowing is indicated 822A and user indicate that 806 are maintained at same position.
In some embodiments, it is actively engaged in response to the user of equipment 600, equipment 600 visually distinguishes user Indicate 806.In the embodiment shown in Fig. 8 AZ, equipment 600 shows that user indicates the bold border around 806 and protrudes aobvious Show that user indicates the initials item of 806 bottoms.
Fig. 8 X is gone to, equipment 600 rotates to laterally (level) orientation from longitudinal (vertical) orientation.As shown in Fig. 8 X, response It is rotated in equipment 600, equipment 600 maintains the expression in the corresponding region of the display 602 occupied before the rotation of equipment 600 (for example, user indicates and each main expression and spilling indicate to occupy the display 602 before and after the rotation of equipment 600 On same area).Rotation indicate in image so that participant look like relative to horizontal orientation it is upright.Some In embodiment, equipment 600 shows that main expression or the animation of the image in main expression rotate (for example, the rotation with equipment 600 Turn contrary 90 degree).It is indicated in some embodiments for not being square overflowing, the video data of participant is cropped And/or translation is to include (for example, placed in the middle) by the face of user in the rotation of equipment 600 in corresponding spilling indicates.In master Indicate it is not in some embodiments of square, the video data of participant is cropped and/or translates to revolve in equipment 600 The face of user is centered in corresponding main expression when turning.In some embodiments, the video data of participant is cut out Be cut to include user face (for example, no matter whether equipment 600 is rotated, such as when received video data have it is aobvious with it When showing the different aspect ratio of the expression at place or format).
As shown in Fig. 8 X, participant corresponding with main expression 810H starts to be actively engaged in.In response, equipment 600 is put Big main expression 810H simultaneously reduces the main size for indicating 810E, as shown in Fig. 8 Y.The embodiment party shown in Fig. 8 BA to Fig. 8 BB In case, equipment 600, which visually distinguishes spilling corresponding with participant is enlivened, indicates 822F (for example, the initials emphasized Item).
In some embodiments, real time communication session is added in response to participant, equipment 600 is with the participant being newly added Main expression replace existing main expression, and the participant of replacement is moved to overflow area 820.In Fig. 8 Z, participate in Real time communication session is added in person Pablo.In response, equipment 600 indicates that 810I is replaced with the main of the participant Pablo being newly added The smallest main expression 810E (Stephen) is changed, and shows that spilling indicates 822F (Stephen) in overflow area 820.In order to 822F vacating space is indicated to overflow, and rolling to overflow indicates 822A, 822C, 822D and 822E (for example, according to shown in Fig. 8 Z The orientation of equipment 600 is upward).As shown in Fig. 8 Z, it is no longer visible to overflow expression 822A, and overflow expression 822C to be indicated by user 806 parts hide.
As shown in Fig. 8 AA, equipment 600 rotates to longitudinal (vertical) orientation from laterally (level) orientation.In response, equipment The image of participant in 600 rotation expressions, while maintaining region occupied by each expression on display (for example, equipment 600 It rotates to horizontal orientation from machine-direction oriented in response to equipment and inverts performed rotation).
In Fig. 8 AB, participant corresponding with main expression 810G leaves (disconnection) real time communication session.Optionally, such as Shown in embodiment shown in Fig. 8 BC, in response, the display instruction participant of equipment 600 has left real time communication session Notify 828.Since the quantity of participant is remained above five, equipment 600 maintains 4 pair of 1 cloth with the display of overflow area 820 Office.It is main to indicate 810G by the main expression of the participant Stephen with next highest activity level as shown in Fig. 8 AC 810J replacement, and spilling corresponding with participant Stephen indicates that 822F is removed from overflow area 820.When removal overflow table When showing 822F, remaining, which is overflowed, indicates that 822A, 822C, 822D and 822E are rolled to the left.In the embodiment shown in Fig. 8 BD, The spilling of participant Hoan indicates that 822D is updated to show that the not connected placeholder of instruction participant Hoan and request participate in The prompting that person Hoan is rejoined, which is shown, to be indicated.
In some embodiments, if participant's quantity when participant leaves real time communication session in real time communication Five are reduced to from six, then equipment 600 stops display overflow area 820, and optionally, increases by one or more in main indicate A size and/or perpendicular separation.
In Fig. 8 AC, equipment 600, which receives (for example, detection), indicates the selection pair of 822E with the spilling of participant Allison The user answered inputs 850F (for example, double-click).As shown in Fig. 8 AD, 850F is inputted in response to receiving user, equipment 600 is being worked as The amplification of display participant Allison indicates 830 before 4 pair of 1 layout of the main expression 810 of preceding display.Amplification indicates 830 Display indicates the display of 810G similar to the amplification in Fig. 8 N, the difference is that amplification expression 830 is not according in 4 pair of 1 layout What main expression proposed.
As shown in Fig. 8 AD, equipment 600 receives (for example, detection) to be shown with closing can indicate that the 818 corresponding user of selection is defeated Enter 850G (for example, tap).As shown in Fig. 8 AE to Fig. 8 AF, 850F is inputted in response to receiving user, the display of equipment 600 participates in The main expression 810K of person Allison is instead of the main expression 810J of participant Stephen (for example, having minimum active state Participant replace with the currently participant that mainly indicates).It is main to indicate that 810K (at least initially) includes along edge bottom Band (or sash) 842A comprising the name of participant and for amplify indicate show and can indicate (for example, such as Fig. 8 AD institute Show).Fig. 8 AE to Fig. 8 AF is shown with mainly indicating that 810K replacement mainly indicates the exemplary animation of 810J.It is main to indicate 810K Initially indicate that size shows and (indicates 810J similar to main in Fig. 8 AC), is then amplified to medium expression size with small, and It is main indicate 810H from it is medium be reduced to it is small.Corresponding to participant Allison spilling indicate 822E size reduce and/or it is light Out, the spilling for the participant being replaced indicates that 822F is moved in overflow area 820.In some embodiments, stop aobvious Show it is main indicate 810J and show it is main indicate 810K include from it is main indicate 810J to the main cross fade for indicating 810K and/ Or the tone for creating " flash of light " effect.Fig. 8 BE to Fig. 8 BH, which is shown, removes the main reality for indicating 810J using flash effect Apply scheme.
In some embodiments, overflow area 820 is rotatable.The embodiment party with reference to shown in Fig. 8 BH to Fig. 8 BI Case, equipment 600, which rolls the spilling in overflow area 820 in response to being actively engaged in for participant, indicates 822, and does not show currently Or display is not indicated for corresponding overflow of the participant completely.As shown in Fig. 8 BH, the spilling of participant indicates that 822F is (corresponding In mainly expression 810I) it is partially hidden in overflow area 820.In Fig. 8 BH, corresponding to the main participant for indicating 810I It is actively engaged in real time communication session.As shown in Fig. 8 BI, corresponding to the main participant for indicating 810I just in response to receiving instruction In the data being actively engaged in, equipment 600 amplifies main expression 810I, and rolling to overflow indicates 822, so that it is complete to overflow expression 822F Full display (for example, near centre of the display portion of overflow area 820), and the bottom 822F is indicated by highlighting spilling Initials item and visually indication overflow indicate 822F.
Fig. 8 AG, the detection contact 850H of equipment 600 are gone to, which moves horizontally from right to left in overflow area 820, It maintains and the contact of display 602 simultaneously.As shown in Fig. 8 AH, in response to receiving contact 850H, equipment 600 is in display 602 On roll to overflow to the left and indicate 822 so that overflowing indicates that 822F partly removes the left edge of display 602, and overflow table Show that 822A no longer indicates that 806 is Chong Die with user.ISE, contact 850H and spilling indicate 822 mobile identical horizontal amounts.
It is main to indicate that 810 remain unchanged when rolling overflow area 820.Moreover, user indicates that 806 are maintained at overflow area In domain 820.
In the embodiment shown in Fig. 8 BJ to Fig. 8 BK, 850H is contacted since the left side of overflow area 820 and water It is flat to move right.In response, equipment 600, which scrolls right to overflow on display 602, indicates 822.In Fig. 8 BJ to Fig. 8 BK, When equipment 600, which reaches user, indicates 806, which removes (for example, no longer showing) from display 602 and indicates.
Fig. 8 AH is gone to, equipment 600 does not show that reception is (for example, inspection at the position of main expression 810 on display 602 Survey) user's input 850I (for example, tap).As shown in Fig. 8 AI, 850I is inputted in response to receiving user, the display of equipment 600 is exhaled Be control menu 832 comprising calling show can indicate 834 (for example, to leave real time communication session), effect show can indicate 836, Menu show can indicate 838 and processing show and can indicate 840.
As shown in Fig. 8 AI, Call- Control1 menu 832 is not Chong Die with overflow area 820 and main expression 810.In order to exhale It is control 832 vacating space of menu, the minification of equipment 600 and/or vertical translation mainly indicate one or more of 810 (for example, the size for reducing canvas area 811) and overflow area 820 is moved up on display 602.In some embodiments In, equipment 600 reduces the size that medium size mainly indicates and maintains small main expression and the big size mainly indicated.Some In embodiment, equipment 600 reduces vertical interval and/or increases vertically superposed between main indicate.
850I is inputted in response to receiving user, equipment 600 also shows band 842A to 842D in each main expression.Band 842A to 842D includes that the name of corresponding participant and showing for amplification expression can indicate (for example, as shown in Fig. 8 O).In some realities It applies in scheme, the vertically superposed amount between the vertical height with 842A to 842D and main indicate is identical.
As shown in Fig. 8 AJ, equipment 600 receives (for example, detection) to be shown with menu can indicate that the 838 corresponding user of selection is defeated Enter 850J (for example, tap).In some embodiments, it includes start from Call- Control1 menu 832 upward that user, which inputs 850J, Gently sweep (for example, showing in processing can indicate on 840).As shown in Fig. 8 AK, 850J, the extension calling of equipment 600 are inputted in response to user Menu 832 is controlled to show additional information relevant to real time communication session and control, similar to about the calling control in Fig. 6 H Additional information and control described in menu 630 processed.
As shown in Fig. 8 AL, equipment 600 receives (for example, detection) to be shown with effect can indicate that the 836 corresponding user of selection is defeated Enter 850K (for example, tap).As shown in Fig. 8 AM, 850K is inputted in response to receiving user, equipment 600 is set from the display of camera 603 The enlarged drawing 843 of standby 600 user simultaneously modifies Call- Control1 menu 832.It is prominent in the Call- Control1 menu 832 of modification Display effect, which is shown, can indicate 836, and display effect option shows 844A can be indicated to 844E.In Fig. 8 AN, equipment 600 receives (example Such as, detect) show with effect option and can indicate that the corresponding user of the selection of 844A inputs 850L (for example, tap).Such as Fig. 8 AO institute Show, inputs 850L in response to receiving user, equipment 600 modifies Call- Control1 menu 832 to include that head portrait option shows and can indicate 846A to 846C.In Fig. 8 AO, equipment 600 receives (for example, detection) to be shown with head portrait can indicate the corresponding user of the selection of 846C Input 850M (for example, tap).As shown in Fig. 8 AP, in response to receive user input 850M, equipment 600 in image 843 Display effect 843A in the face of user, and roll Call- Control1 menu 832 by currently selected head portrait show can indicate 846C occupy In be placed in Call- Control1 menu 832.In Fig. 8 AP, equipment 600 is shown in cancellation can indicate to receive (for example, detection) on 848 User inputs 850N (for example, tap).As shown in Fig. 8 AQ, 850N is inputted in response to receiving user, equipment 600 controls calling Menu 832 processed returns to the configuration of Fig. 8 AN, and the display of effect 843A is maintained on enlarged drawing 843.In Fig. 8 AQ, if Standby 600 receive (for example, detection) user's input on the display 602 except enlarged drawing 843 and Call- Control1 menu 832 850O (for example, tap).As shown in Fig. 8 AR, 850O is inputted in response to receiving user, equipment 600 stops display enlarged drawing 843 and user indicate 806 in user image on display effect 843A.When effect 843A is activated, such as in Fig. 8 AR, The live video stream of the user of equipment 600 includes effect 843A.
As shown in Fig. 8 AR, Call- Control1 menu 832 shows including media can indicate 844E.Equipment 600 receives (for example, inspection Survey) show with media and can indicate that the corresponding user of the selection of 844E inputs 850P (for example, tap).As shown in Fig. 8 AS, in response to connecing It receives user and inputs 850P, equipment 600 modifies Call- Control1 menu 832 to include that media option is shown 852A can be indicated to 852D. In Fig. 8 AS, equipment 600 receives (for example, detection) to be shown with media option can indicate the corresponding user of the selection of 852A (sketch) Input 850Q (for example, tap).850Q is inputted in response to receiving user, equipment 600 is shown in canvas area 811 and media Option shows the expression that can indicate the associated media content of 852A.
As shown in Fig. 8 AT, 850Q is inputted in response to receiving user, the selected media item of equipment 600 indicates that 810L is replaced Change owner will indicate 810K (for example, minimum main expression).In Fig. 8 AT, equipment 600 receives (for example, detection) and call disestablishment The corresponding user of request for controlling menu 832 inputs 850R (for example, on Call- Control1 menu 832 downward gently sweep).Such as figure Shown in 8AU, 850R is inputted in response to receiving user, equipment 600 stops display Call- Control1 menu 832.
In some embodiments, selection media option, which is shown, can indicate that 852A makes associated media content and leads in real time Believe that other participants of session share (for example, showing in the canvas area on the display of the equipment of another participant).In In some embodiments, the participant of real time communication session can be interacted with sharing media content (for example, via real time communication application The sequence of one or more inputs in the user interface of program).Fig. 8 AV is shown in which that supplementary features 810L-1 has been added To the embodiment for the media content for indicating 810L.In some embodiments, in response to associated by participant and expression 801L Media content interaction, the amplification of equipment 600 indicates 801L (for example, being similar to the side that main expression is amplified and then is selected The mode that formula or participant corresponding to the expression are actively engaged in).As shown in Fig. 8 AV, in response to participant with it is associated The interaction of media content indicates that 810L is amplified (compared with Fig. 8 AT).Fig. 8 AV is also shown to be shared in real time communication session Other content (for example, game between two participants) expression 810M.
Fig. 9 A to Fig. 9 K is the flow chart for showing the method according to some embodiments.In the equipment (example with display Such as, 100,300,500,600) place executes method 900.Some operations in method 900 are optionally combined, time of some operations Sequence is optionally changed, and some operations are optionally omitted.
As described below, method 900 is especially provided for the intuitive side of adaptive and intelligent real time communication user interface Formula.In addition, the in particular user of method 900 provides the intuitive manner with real time communication user interface interaction.This method is by being based on Participant's quantity in real time communication provides adaptively and intelligent real time communication interface reduces the cognitive load of user, to create Build a more effective man-machine interface.For battery-driven calculating equipment, allow users to based on preparatory in real time communication Determining participant's quantity more rapidly and more effectively checks real time communication user interface, saves power and increases battery two Interval between secondary charging.
At frame 901, equipment receives request (for example, 850A) to show for the reality between two or more participants When communication session user interface.
At frame 902, in response to receiving display for the real time communication session between two or more participants The request of user interface, equipment show real time communication user interface (for example, 804), including show in real time communication session simultaneously Participant multiple expressions (for example, 810A to 810D), wherein multiple expressions of the participant in display real time communication session Including frame 902 and 903.In some embodiments, real time communication session subscriber interface includes associated with first participant Placeholder (for example, 810A of Fig. 8 C) (for example, graphical placeholder (for example, image or avatars), text placeholder (for example, Name or initials)), which has been invited into real time communication session but real time communication session is not yet added (for example, the participant other than the user of electronic equipment；Wait that communication session is added and is sent communication session is added The participant of invitation).In some embodiments, if participant is sharing the real-time media stream including live video stream, Then the expression includes the image (for example, 810B of Fig. 8 D) of live video stream.In some embodiments, if participant Real-time media stream is shared with only audio, then the expression includes the head portrait (for example, 810E of Fig. 8 I) of corresponding participant.In some realities It applies in scheme, multiple expressions of the participant in real time communication session include selected from only audio status (for example, participant is used only Audio is communicated), video state (for example, participant is had conversation using video and audio), video pause state (for example, The video pause of participant), the non-decoded state of video be (for example, the problem of due to delay issue, video stream format etc., can not locate Manage participant's video flowing), leave state (for example, participant has left real time communication session), and wait it is to be added (for example, ginseng With person be invited into real time communication session but not yet be added real time communication) state user's communications status.
It include participant's quantity greater than threshold value according to the participant's quantity determined in real time communication session at frame 903, Equipment shows that more than first in addition to the user of electronic equipment is a to be greater than one or more sizes of respective threshold size simultaneously Participant expression (for example, 810A to 810D), and in overflow area (for example, 820) more than second a participants of display table Show (for example, 822), wherein a participant more than second includes the one or more participations being not included in more than first in a participant Person, and the expression of other the one or more participants shown in an overflow region is aobvious with the size for being less than respective threshold size Show.In some embodiments, current active participant (for example, 810E of Fig. 8 J) (for example, most active participant, with Satisfaction enlivens the participant that the mode of participant's standard group is talked or moved) it is shown as the part of a participant more than first.In In some embodiments, the participant of current active is shown in the main region of user interface (for example, 811) (for example, display the The region of a participant more than one) in.In some embodiments, if the participant's (example not shown in main region currently Such as, the participant corresponding to the 822B of Fig. 8 U) become current active participant, then the expression of the participant is shown in main region In domain (for example, 810H of Fig. 8 V).In some embodiments, the expression (for example, 810) of a participant more than first with two or More different sizes are shown (for example, wherein more active participant or more recent enlivening participant with more active than less Participant or the bigger size of participant of enlivening more remote show), and a participant more than second in overflow area Indicate that (for example, 822) are shown with identical size.
In some embodiments, overflow area (for example, 820) further include a participant more than first expression (for example, 810A to 810D) (for example, the expression of the participant shown in the main region of user interface is also shown in an overflow region) (for example, Fig. 8 AX).In some embodiments, overflow area (for example, 820) further includes being not currently connected to real time communication meeting The placeholder of the real time communication session invitee of words indicates (for example, 822A in Fig. 8 AW) (for example, head portrait, name, name contract It writes).Based on whether meeting rated condition (for example, whether including the quantity for being greater than participant based on real time communication session) Lai Butong Ground shows the expression that the expression of participant allows user to be easier or more effectively check participant in real time communication session.One Group condition, which has been satisfied, enhances the operability of equipment without executing optimized operation when further user input, And make user-equipment interface it is more effective (for example, by help user provide suitably input and reduce operation equipment/with set Standby user's mistake when interacting), this is further through allowing users to faster and more effectively reduce electric power using equipment Using and extend the battery life of equipment.
Optionally, at frame 904 and 905, equipment with first rate update the indicating of more than first a participants (for example, 810A to 810D) display (for example, indicate arrangement, position, size, position, visually indicate, vision content and/or video feedback Send) (for example, the expression of a participant more than first is the frame rate such as 24,30 to be equal to or more than exemplary video refresh rate Or 60 frame update per second real-time video feeding), and a participant more than second is updated in an overflow region with the second rate Indicate (for example, 822) display (for example, indicate arrangement, position, size, position, visually indicate, the vision content of expression And/or video feed), wherein the second rate is lower than first rate.(for example, the thumbnail in overflow area updates at any time, but It is every using the rate such as 1,5 of the video refresh rate of the expression for the multiple participants being less than in canvas area 811 or 10 frames Second).Being updated with different rates indicates that the display of group allows equipment to provide a user about the current shape indicated in real time communication The feedback of state, while reservation process ability is indicated with indicating that faster rate provides one group than another group, thus, it was demonstrated that Yong Huneng It is enough to indicate group with what higher rate checked higher priority.For user provide that improved visual feedback just enhances equipment can Operability, and keep user-equipment interface more efficient (for example, setting by helping user to provide input appropriate and reduce operation Standby/user's mistake when being interacted with equipment), this reduces electricity further through allowing users to more rapidly and efficiently using equipment Power uses and extends the battery life of equipment.
Optionally, at frame 906, expression of the equipment as more than second a participants of display in overflow area (for example, 820) When (for example, 822), the view (for example, self checking type camera view) of the camera of electronic equipment in an overflow region is shown It indicates (for example, 806).Show that the expression of the view of the camera of electronic equipment provides a user the active view about user's camera The visual feedback (for example, content that other participants of user into real time communication session show) of state.It provides and changes for user Into visual feedback just enhance the operability of equipment, and make user-equipment interface it is more efficient (for example, by help use User's mistake when family provides input appropriate and reduces operation equipment/interact with equipment), this is further through allowing users to faster Speed and efficiently use equipment and reduce electricity usage and extend the battery life of equipment.
Optionally, at frame 907,908,909 and 910, equipment detection corresponds to the input (for example, 850H) of overflow area (for example, horizontal finger dragging).In response to detecting input corresponding with overflow area, device rolling overflow area (for example, The expression of more than second a participants is rolled along first direction according to input), it is participated in stopping display (for example, hiding) more than second The subset (for example, 822G and 822H in Fig. 8 BJ to Fig. 8 BK) of the expression of person.While rolling overflow area, equipment is maintained The display (for example, Fig. 8 BJ to Fig. 8 BK) of the expression (for example, 806) of the view of the camera of electronic equipment in an overflow region.In In some embodiments, rolling overflow area includes the one or more animations indicated for showing more than second a participants, is seen Get up in the expression scrolls beneath of the view of the camera of equipment.The camera of electronic equipment is maintained while rolling overflow area The display that view indicates provides a user the constant feedback of the active view state of the camera about electronic equipment (for example, user The content that other participants into real time communication session show) while also user being allowed to roll through in overflow area The expression of participant.Improved visual feedback is provided for user and enhances the operability of equipment, and makes user-equipment interface More effectively (for example, wrong by the user for helping user to provide when suitably inputting and reduce operation equipment/interact with equipment Accidentally), this is further through allowing users to faster and more effectively reduce electricity usage using equipment and extend equipment Battery life.
Optionally, at frame 911,912 and 913, in detection and overflow area in multiple expressions of more than second a participants Indicate the corresponding input (for example, 850F) of selection of (for example, 822E of Fig. 8 AC) (for example, corresponding in overflow area 820 The Flick gesture of expression or be more than intensity threshold corresponding with the expression in overflow area 820 characteristic strength gesture). In response to the corresponding input of the selection that detects with indicate in multiple expressions of a participant more than second in overflow area, equipment is used More than first a participants are replaced in the expression (for example, 810K of Fig. 8 AE) of participant corresponding with the selected expression in overflow area Multiple expressions in indicate (for example, 810J in Fig. 8 AC) display (for example, replacement basic display unit in ginseng active recently With person).In some embodiments, with in overflow area selected by indicate that corresponding participant's indicates replacement more than first The display indicated in multiple expressions of participant includes stopping showing in an overflow region in the expression of more than second a participants Selected expression.In some embodiments, stop display in an overflow region is selected to indicate to include the selected animation for indicating to shrink. In some embodiments, other expressions in overflow area are translated to fill the selected gap indicated of display.In some implementations In scheme, with in overflow area selected by indicate corresponding participant indicate replacement more than first the multiple of a participant indicate The display of middle expression includes showing to indicate indicating for corresponding participant with selected with first size, then will be with selected expression pair The expression of the participant answered is amplified to the second size greater than first size.In some embodiments, the table of selected participant Show the minimum expression (corresponding to the participant in canvas area with minimum activity level) in replacement canvas area, then size It is automatically increased (for example, increasing to medium expression size), and (corresponding to has the second low activity in canvas area for another expression Horizontal participant) size reduce (for example, being reduced to small expression size).
It replaces the display of expression corresponding with the selected expression in overflow area and/or stops in display overflow area Selected expression provide additional control option without keeping UI mixed and disorderly, and for user provide to equipment it is more control and It is quickly indicated with the selected displaying of user replacement by help user to show and indicate.There is provided to the additional control of equipment without by So that UI is enhanced the operability of equipment in a jumble in the control of additional display, and make user-equipment interface it is more effective (for example, By help user provide suitably input and reduce operation equipment/with equipment interact when user's mistake), this further through It allows users to faster and more effectively reduce electricity usage using equipment and extends the battery life of equipment.
Optionally, at frame 914, equipment shows the first animation effect (for example, Fig. 8 AA to Fig. 8 AC) (for example, indicating to become Greatly, then become smaller, optionally there is cross fade and the tone for creating " flash of light " effect), to influence more than first ginsengs With the expression in the expression of person.Optionally, at frame 915, equipment display influences selected in the expression of more than second a participants The second animation effect indicated, second animation effect and the first animation effect share animation feature (for example, Fig. 8 BF extremely schemes 8BH) (for example, can correspond to overflow the flashing rate in display from the rate that basic display unit fades out/shrinks).Display corresponds to Second animation of the first animation provides visual feedback and is indicated with quickly identifying switching and/or which is replaced.It is provided for user Improved visual feedback just enhances the operability of equipment, and keeps user-equipment interface more efficient (for example, passing through help User's mistake when user provides input appropriate and reduces operation equipment/interact with equipment), this is further through allowing users to more Quickly and efficiently reduce electricity usage using equipment and extends the battery life of equipment.
Optionally, at frame 916, further, in response to detect correspond to overflow area input (for example, 850H), equipment shows new (for example, the previously hiding) expression of a participant more than second (for example, Fig. 8 AG in an overflow region 822A into Fig. 8 AH).Show that the new expression of participant allows user by increasing institute in an overflow region in response to input The accessibility of the expression of display more easily and more effectively checks expression.One group of condition be satisfied without into Optimized operation is executed when user's input of one step and enhances the operability of equipment, and keeps user-equipment interface more effective (for example, by help user provide suitably input and reduce operation equipment/with equipment interact when user's mistake), this Further through allowing users to faster and more effectively reduce electricity usage using equipment and extend the battery longevity of equipment Life.
Optionally, at frame 917, real time communication meeting has been added or has been added in response to receiving the new participant of instruction The data of words, the new participant of display is being overflow at a series of ending of expressions of more than the second a participants of equipment in an overflow region Expression in region (for example, 822B in Fig. 8 R) out.
Optionally, at frame 918,919,920 and 921, equipment is received and the first participant in more than second a participants Horizontal relevant second data of activity (for example, speech, mobile).In response to receiving the second data, equipment is according to more than second The activity level of first participant in a participant increases, and emphasizes the visual signature (example that (for example, increase or initiate) indicates Such as, size, highlight expression (for example, display bezel), animation (for example, pulse head portrait), keep image preposition), the expression with The expression of first participant is corresponding (for example, 822B of Fig. 8 U) in multiple expressions of a participant more than second in overflow area.It rings Ying Yu receives the second data, and equipment is reduced according to the activity level of the first participant in more than second a participants, do not emphasized Visual signature that (for example, increase or initiate) indicates is (for example, size, highlight expression (for example, display bezel), animation (example Such as, pulse head portrait), keep image preposition), the first participant in the expression and overflow area in the expression of more than second a participants It is corresponding.It emphasizes and/or does not emphasize to visually indicate that user is allowed quickly to identify that participant is (including most active when a given condition is fulfilled And/or most sluggish participant) activity level.It has been satisfied in one group of condition and has been inputted without further user Shi Zhihang optimized operation enhances the operability of equipment, and keeps user-equipment interface more effective (for example, passing through help User provides user's mistake when suitably inputting and reduce operation equipment/interact with equipment), this is further through enabling users to Enough battery lifes for faster and more effectively reducing electricity usage using equipment and extend equipment.In addition, when meeting The visual feedback that the activity level for indicating each participant is provided for user is emphasized and/or not emphasized when rated condition.For user Improved visual feedback is provided and just enhances the operability of equipment, and keeps user-equipment interface more efficient (for example, passing through User's mistake when user being helped to provide input appropriate and reduce operation equipment/interact with equipment), this is further through enabling users to It reaches more rapidly and efficiently uses equipment and reduce electricity usage and extend the battery life of equipment.
Optionally, at frame 922 and 923, when showing real time communication user interface, equipment detects more than second a participants In second participant become the participant of current active and (for example, most active participant, enliven participant's standard group to meet The mode participant that talks or move).In response to detecting that the second participant more than second in a participant becomes current living The participant of jump and according to determining that the expression of second participant in more than second a participants do not show currently in an overflow region Show, device rolling overflow area with show in an overflow region the indicating of the second participant more than second in a participant (for example, The 822F of Fig. 8 BH to Fig. 8 BI).Dynamic rolling overflow area to show that enlivening participant allows in overflow area in an overflow region Check the expression for enlivening participant without selecting any additional input in domain.Reduce the quantity of input needed for executing operation The operability of equipment is enhanced, and keeps user-equipment interface more effective (for example, suitable defeated by helping user to provide User's mistake when entering and reducing operation equipment/interact with equipment), this is further through allowing users to faster and more effectively Reduce electricity usage using equipment and extends the battery life of equipment.
It include the participation less than or equal to threshold value according to the participant's quantity determined in real time communication session at frame 924 Person's quantity, equipment show the participant in addition to the user of electronic equipment with the one or more sizes for being greater than respective threshold size Expression, without be less than or equal to respective threshold size one or more sizes show in addition to the user of electronic equipment The expression (for example, Fig. 8 P) of participant.In some embodiments, electronic equipment receives (for example, being used for real time communication in display Before the user interface of session) it is associated multiple real-time to the corresponding participant of multiple participants of live video communication session Media Stream.In some embodiments, the quantity of received real-time media stream, which corresponds to, is currently joined into live video communication session In participant quantity, do not include the participant using the equipment.Based on whether meeting rated condition (for example, based on real When communication session whether include showing the quantity indicated less than or equal to the participant of not no overflow area) be displayed differently from ginseng Expression with person allows user to be easier or more effectively check the expression of participant in real time communication session.One group of condition It is satisfied and enhances the operability of equipment without executing optimized operation when further user input, and used Family-equipment interface is more effective (for example, suitably inputting and reducing operation equipment/hand over equipment by helping user to provide User's mistake when mutually), this further through allow users to faster and more effectively to reduce using equipment electricity usage and Extend the battery life of equipment.
It in some embodiments, include being less than second threshold according to the participant's quantity determined in real time communication session Participant's quantity, no one of multiple expressions of remote participant (for example, participant in addition to a user) (example that overlaps each other Such as, Fig. 8 J).It in some embodiments, include being greater than or equal to the according to the participant's quantity determined in real time communication session Participant's quantity of two threshold values, at least some of multiple expressions of remote participant (for example, participant in addition to a user) Overlap each other (for example, Fig. 8 K).
It optionally, include less than second according to participant's quantity in determining real time communication session at frame 925 and 926 Participant's quantity (for example, two participants, user (for example, other participants) including electronic equipment) of threshold value, if At least one in the indicating of the standby participant with the first aspect ratio (for example, rectangular aspect ratio) display other than the user of equipment A (for example, 810B of Fig. 8 H).According to the participation that the participant's quantity determined in real time communication session includes greater than second threshold Person's quantity (for example, three or more participants), equipment is with second aspect ratio different from the first aspect ratio (for example, square Shape aspect ratio) at least one of the expression of participant of the display other than the user of equipment (for example, 810B of Fig. 8 I).Base In meeting rated condition to show indicating for participant based on the different aspect ratios of participant's quantity in real time communication session, allow Equipment provides intelligent real time communication user interface by maximizing the tissue of the size and/or expression that indicate to provide and be based on ginseng With the more preferable user experience of person's quantity, thus allow user be easier and more effectively check the expression in equipment.In one group of item Part, which has been satisfied, enhances the operability of equipment without executing optimized operation when further user input, and Make user-equipment interface it is more effective (for example, by help user provide suitably input and reduce operation equipment/and equipment into User's mistake when row interaction), this is further through allowing users to faster and more effectively reduce electricity usage using equipment And extend the battery life of equipment.
It optionally, include less than second according to participant's quantity in determining real time communication session at frame 927 and 928 Participant's quantity (for example, two participants) of threshold value, equipment show equipment with third aspect ratio (for example, rectangular aspect ratio) The expression (for example, 806 of Fig. 8 H) of the view (for example, self view of the camera from equipment) of camera.It is real-time according to determining Participant's quantity in communication session includes more than or equal to participant's quantity of second threshold (for example, three or more are joined With person), equipment is with the view of the camera of fourth aspect ratio (for example, square aspect ratio) display equipment different from third aspect ratio Scheme the expression (for example, 806 of Fig. 8 J) of (for example, self view of the camera from equipment).Based on meeting rated condition with base The view of the camera of the different aspect ratio display devices of participant's quantity in real time communication session allows equipment to pass through maximization The size of the expression of the view of camera come provide a user intelligent real time communication user interface and allow user be easier and more have Check to effect the expression in equipment.It has been satisfied in one group of condition without executing optimization when further user input Operation enhances the operability of equipment, and keeps user-equipment interface more effective (for example, suitable by helping user to provide Input and user's mistake when reducing operation equipment/interact with equipment), this is further through allowing users to more rapidly and have Effect ground is reduced electricity usage using equipment and extends the battery life of equipment.
It optionally, is first participant's quantity according to participant's quantity in real time communication session at frame 929 and 930, Equipment indicates (for example, Fig. 8 J) with the first layout (for example, arrangement or pattern) display participant.In some embodiments, First layout is one in multiple predefined layouts, wherein at the different location of the expression anchoring of user in the user interface (for example, randomly selected layout).In some embodiments, is determined based on participant's quantity in real time communication session The layout (for example, arrangement) of the expression of a participant (for example, other than user of equipment) more than one.According to real time communication session In participant's quantity be second participant quantity, equipment is indicated with the second layout display participant for being different from the first layout (for example, Fig. 8 K) (for example, layout changes as participant is added/leaves real time communication session).In some embodiments, First layout and second layout in a series of predetermined layouts, the expression with user it is similar staggered relatively.(example Such as, a series of predetermined layouts include the layout of no-float together).In some embodiments, a series of true in advance Fixed layout includes the layout member for multiple possible participant's quantity (for example, for a non-equipment user participant The first member (for example, 1 or more is laid out), the second member (for example, 2 or more are laid out) for two non-equipment user participants Etc..In some embodiments, electronic equipment includes multiple predetermined layout series, wherein the member of each series is shared Common trait).It similar staggered relatively indicates to allow to use with what different layout displays had that user indicates based on the quantity of user Family is easier by increased visibility and more effectively checks expression.It has been satisfied in one group of condition without further User's input when execute optimized operation and enhance the operability of equipment, and make the more effective (example of user-equipment interface Such as, by help user provide suitably input and reduce operation equipment/with equipment interact when user's mistake), this is again Electricity usage faster and more effectively can be reduced using equipment by using family and extend the battery life of equipment.
Optionally, at frame 931,932,923 and 934, the expression of the participant in addition to the user of equipment is being shown Meanwhile equipment receives the increased data of participant's quantity in instruction real time communication session.It is real-time in response to receiving instruction Participant's quantity in communication session increased data, equipment reduce one of the participant other than the user of equipment or The size (for example, Fig. 8 J to Fig. 8 K) of multiple expressions.In response to having received participant's quantity in instruction real time communication session Increased data, equipment show that new participant before original display (for example, be just added the group recently in real time communication session Participant) one or more indicate (for example, Fig. 8 J to Fig. 8 K).
Reducing one or more sizes indicated and showing newly indicates to provide a user about the current of real time communication session The feedback of state, and provide a user the visual feedback for indicating that real time communication has been added in new participant.It is provided for user improved Visual feedback just enhances the operability of equipment, and keeps user-equipment interface more efficient (for example, by helping user to mention User's mistake when for input appropriate and reducing operation equipment/interact with equipment), this further through allow users to more rapidly and It efficiently uses equipment and reduces electricity usage and extend the battery life of equipment.
Optionally, at frame 935 and 936, equipment, which receives to remind corresponding to first, shows the selection that can indicate (for example, 808) Input (for example, 850B).Show the corresponding input of the selection that can indicate in response to receiving to remind with first, equipment is joined to first The new notice (for example, calling, the tinkle of bells etc.) different with the previous notification for being sent to first participant is sent (for example, second from person Notice invasion property more than the first notice).Show in response to the prompting of previous notification for receiving with being different from being sent to participant Can indicate selection it is corresponding input and to first participant send notify, reduce execute operation needed for input quantity. What the quantity (for example, number of options needed for user reminding participant) for reducing input needed for executing operation enhanced equipment can Operability, and keep user-equipment interface more effective (for example, setting by helping user to provide suitably to input and reduce operation It is standby/with equipment interact when user's mistake), this further through allow users to faster and more effectively using equipment and reduce Electricity usage and the battery life for extending equipment.In addition, operation is executed automatically, it is such as defeated in not further user It is sent in the case where entering and notifies different notices from first, enhance the operability of equipment, and make user-equipment interface more Efficiently (for example, user's mistake when by helping user to provide input appropriate and reduce operation equipment/interact with equipment), this Reduce electricity usage further through allowing users to more rapidly and efficiently using equipment and extend battery longevity of equipment Life.
Participant a more than first placeholder and indicate in the embodiment of the first layout placement, optionally, in frame 937, at 938 and 939, real time communication session is not added up to predetermined time quantum (example according to determining first participant for equipment Such as, real time communication is not added after the predetermined period), then stop showing placeholder (for example, Fig. 8 E and Fig. 8 F).This Outside, real time communication session is not added up to predetermined time quantum according to determining first participant, equipment participates in more than first The arrangement of the expression of person change into different from the first layout the second layout (e.g., including the one of amplification and/or mobile participant A or multiple expressions (for example, display layout, wherein first participant does not include in this set-up)).In some embodiments, Equipment receives data from server or another equipment, to determine that real time communication session is not yet added up to predetermined in first participant Time quantum.In some embodiments, equipment can determine first participant be not yet added real time communication session up to predefine Time quantum because it does not show any expression of user.When meeting defined condition, stops display placeholder and simultaneously change The arrangement of expression allows user quickly to identify when other users are added or are added without calling, while maximizing shown participation The visibility of person and/or placeholder.It has been satisfied in one group of condition optimal without being executed when further user input Change operation and enhance the operability of equipment, and keeps user-equipment interface more effective (for example, by helping user to provide conjunction Suitable input and user's mistake when reducing operation equipment/interact with equipment), this further through allow users to more rapidly and Equipment is efficiently used and reduces electricity usage and extends the battery life of equipment.In addition, stopping display placeholder simultaneously The arrangement for changing display provides a user the feedback of the current state about real time communication session, and it is anti-to provide a user vision Feedback, instruction will execute certain operations associated with notice and/or real time communication session.Improved visual feedback is provided for user The operability of equipment is enhanced, and keeps user-equipment interface more effective (for example, suitable defeated by helping user to provide User's mistake when entering and reducing operation equipment/interact with equipment), this is further through allowing users to faster and more effectively Reduce electricity usage using equipment and extends the battery life of equipment.
Optionally, at frame 940, real time communication session has been added according to determining first participant, equipment is participated in first The expression (for example, 810B of Fig. 8 C to Fig. 8 D) of person modifies the display of the first placeholder (for example, placeholder can have participant's surname Name).In some embodiments, equipment can receive data from server or another equipment to carry out the determination.In some implementations In scheme, equipment can carry out the determination, because it does not show any expression of user.In some embodiments, occur dynamic It draws.Display using expression the first placeholder of modification of participant allows user quickly to identify that calling has been added in which user, to Feedback of user's offer about the current state of real time communication session, and visual feedback is provided a user, instruction will execute and lead to Know and/or the associated certain operations of real time communication session.Improved visual feedback, which is provided, for user enhances grasping for equipment The property made, and make user-equipment interface it is more effective (for example, by help user provide suitably input and reduce operation equipment/ User's mistake when being interacted with equipment), this is further through allowing users to faster and more effectively reduce using equipment Electricity usage and the battery life for extending equipment.
Optionally, at frame 941 and 942, when showing multiple expressions of the participant in real time communication session, equipment inspection Survey the change of participant's quantity in real time communication session.In response to detecting changing for participant's quantity in real time communication session Become, the notice (for example, 812) that participant's quantity in display instruction real time communication session has changed.In some embodiments In, when participant leaves real time communication session, notice is that the expression of the participant left is replaced by then (for example, pre- After the first determining time) animation of the placeholder figure that disappears (e.g., including the name of participant or the figure of initials). In some embodiments, when real time communication session is added in new participant, notice is then (for example, when predetermined Between after) by new participant expression (for example, expression based on the video data transmitted from the equipment of new participant) replace The display of the placeholder (e.g., including the figure of participant's name or initials) of new participant.In some embodiments, It is more than participant's quantity of threshold value (for example, working as overflow area packet according to the participant's quantity determined in addition real time communication session When including expression (for example, 3 or 4 other than the user of electronic equipment indicate) greater than predefined maximum quantity), display Indicate the notice that participant's quantity in real time communication session has changed.In some embodiments, it is real-time to abandon display for equipment The notice that participant's quantity in communication session has changed is (for example, when being added or having left real time communication session there are participant Another visible instruction, such as show the expression for the participant being newly added or remove the aobvious of the expression for having left the participant of session When showing, notice is not shown).In some embodiments, (for example, when the person of being not engaged in has been added or has left real time communication session Visible instruction when), equipment provides the audio output that has changed of participant's quantity in instruction real time communication session without aobvious Show notice.In some embodiments, notice include be added or left real time communication session participant instruction (for example, Name).In some embodiments, if multiple participants are added in threshold amount of time and/or leave real time communication session, Then equipment generates single notice, which indicates the participant for being added and/or leaving real time communication session in threshold amount of time Quantity.In some embodiments, the participant of real time communication session is added and leaves the participant point of real time communication session It is not combined in individually notice.In some embodiments, when participant leaves or real time communication session is added, equipment is prolonged Display notice late, until threshold amount of time is pass by, to determine whether notice should include logical in real time about being added and/or leaving Believe the information of other participants of session.The notice that has changed of participant's quantity in display instruction real time communication session is to user Feedback about the current state of participant's quantity in real time communication is provided, and is provided a user in instruction real time communication session The visual feedback that participant's quantity has changed.Improved visual feedback, which is provided, for user just enhances operating for equipment Property, and make user-equipment interface it is more efficient (for example, by help user provide it is appropriate input and reduce operation equipment/with User's mistake when equipment interaction), this is further through allowing users to more rapidly and efficiently use equipment and reducing electric power makes With and extend the battery life of equipment.
Optionally, at frame 943,944,945 and 946, equipment shows the view (example of the camera of equipment with first size Such as, self view) one as multiple expressions of participant in real time communication session of expression (for example, 806 of Fig. 8 AH) Point.Show real time communication session in participant multiple expressions when, equipment receive with for show the first menu (for example, 832) corresponding first input (for example, 850I) of selection is (for example, sweeping based on the single tap in background or gently upwards and touching Hair).In response to receiving input corresponding with for showing the selection of the first menu, equipment is in real time communication user interface (example Such as, the first menu (for example, Call- Control1 menu 832) is shown in 804), wherein the first menu includes in real time communication meeting The option (for example, 836) of visual effect (for example, intention camera) is shown in words in the expression of participant.In some embodiments In, menu includes other options, such as terminating to call switching camera (for example, front camera to rear camera). In response to receive with for show the selection of option of visual effect (for example, 842) it is corresponding second input (for example, 850K), equipment is indicated with the view of the camera of the second size (for example, 842) display equipment (for example, the table of covering participant Show), wherein the second size is greater than first size.
Optionally, at frame 947,948 and 949, equipment receives and for amplifying the participant's in real time communication session First in multiple expressions indicates the corresponding third input (for example, 850C) of the selection of (for example, 810F of Fig. 8 L) (for example, In The double-click in expression in main region or the input with the characteristic strength for being more than intensity threshold).In response to receiving and using The first selection corresponding third input indicated in multiple expressions of participant in amplification real time communication session, equipment amplification the One indicates (for example, 810F of Fig. 8 M).The apparatus reduces different in multiple expressions of participant in real time communication user interface In the size (for example, 810E of Fig. 8 L to Fig. 8 M) that at least one that first indicates indicates.Dynamically reduce the table of one group of display Show middle one or more sizes indicated and amplify selected expression and provide improved feedback, allows user to focus and be easy Ground checks the expression of amplification while minimizing reduced expression.For user provide that improved visual feedback just enhances equipment can Operability, and keep user-equipment interface more efficient (for example, setting by helping user to provide input appropriate and reduce operation Standby/user's mistake when being interacted with equipment), this reduces electricity further through allowing users to more rapidly and efficiently using equipment Power uses and extends the battery life of equipment.
Optionally, at frame 950,951,952 and 953, the orientation that equipment receives instruction electronic equipment changes (for example, edge First direction (for example, change of 90 degree of clocks)) data.The number that orientation in response to receiving instruction electronic equipment changes The display (for example, Fig. 8 W to Fig. 8 X) of multiple expressions of the participant in real time communication session is rotated (for example, along the according to, equipment Two directions (for example, opposite with first direction to maintain image be upright)).The equipment remains real while rotating display device When communication session in participant multiple expressions previously shown relative priority (for example, another display size, Yong Hujie In face of position, the arrangement etc. as (for example, expression)).Automatic slewing and maintenance had previously been shown when some condition is fulfilled The relative priority (for example, instruction user is just in data of slewing) of expression allows user in the identical position previously shown In check that the expression of rotation has arrangement, size or the position of the expression of minimum vision interference without changing.One group of condition It is satisfied and enhances the operability of equipment without executing optimized operation when further user input, and used Family-equipment interface is more effective (for example, suitably inputting and reducing operation equipment/hand over equipment by helping user to provide User's mistake when mutually), this further through allow users to faster and more effectively to reduce using equipment electricity usage and Extend the battery life of equipment.
Optionally, at frame 954, in response to receive instruction electronic equipment orientation change (for example, from it is machine-direction oriented to Horizontal orientation) the first data, equipment be based on first indicate septum reset (for example, 822C of Fig. 8 AZ to Fig. 8 BA) position change The first display indicated (for example, if indicating it is not square, adjusts in multiple expressions of participant in real time communication session Display to move on to participant when laterally indicating from longitudinal expression (wherein the face of participant may be biased to the top indicated) Face be located at indicate center).
Optionally, at frame 955 and 956, show participant multiple expressions in second indicate when, equipment reception changes Become the request of at least one aspect ratio indicated of multiple expressions of the participant in real time communication session.Change in response to receiving Become the request for the aspect ratio that at least one in multiple expressions of participant indicates in real time communication session, what equipment was indicated second Aspect ratio changes into the 7th aspect ratio (for example, square or round aspect ratio) from the 6th aspect ratio, wherein second indicates simultaneously With the 7th aspect ratio include by the received real-time media stream of electronic equipment second part (for example, be less than first part part, Cut part), which is selected as including the face that detects in real-time media stream (for example, Fig. 8 AZ to Fig. 8 BA 822C).In some embodiments, electronic equipment feeds the aspect ratio for changing and indicating based on real-time video, and passes through selection A part of real-time video feeding including the face detected cuts a part of real-time video feeding, so that detecting Face is not cut away from expression.It maintains the view of face to provide a user feedback in the expression of participant, allows to use The face of other participants is checked at family when changing aspect ratio.For user provide that improved visual feedback just enhances equipment can Operability, and keep user-equipment interface more efficient (for example, setting by helping user to provide input appropriate and reduce operation Standby/user's mistake when being interacted with equipment), this reduces electricity further through allowing users to more rapidly and efficiently using equipment Power uses and extends the battery life of equipment.
Optionally, at frame 957,958,959,960 and 961, when display real time communication session in participant it is multiple When expression, equipment reception the 4th input (for example, 850I) corresponding with for showing the request of menu (for example, 832) (for example, It sweeps based on the single tap in background or gently and triggers upwards).In response to receiving the 4th input, equipment is for logical in real time Believe in the user interface of session and shows menu (for example, 832).Multiple expressions of participant in the mobile real time communication session of the equipment In at least third indicate (for example, 810H of Fig. 8 AI) (for example, in order to accommodate menu；In order to avoid be overlapped with menu).This sets Standby at least the 4th expression changed in real time communication session in multiple expressions of participant is (for example, indicate identical table with third Show；Different expressions is indicated from third) size of (for example, 810F of Fig. 8 AI) is (for example, in order to accommodate menu；In order to avoid with Menu overlapping).In some embodiments, when showing menu in the bottom of screen, expression is moved upward and adjusts size. In some embodiments, when the menu is displayed, the size that at least one in multiple expressions indicates is not adjusted.In some embodiment party In case, expression is that there are three the expressions of predefined picture size (for example, 1 " small ", 2 " medium " and 1 " big ") for tool, and only adjust A kind of size of the expression of size (for example, " medium " image), while " small " and " big " expression are kept into identical size.When aobvious When showing menu, mobile one or more sizes for indicating and changing one or more expressions in real time communication session provide attached Add control option, while maintaining the visibility indicated in real time communication user interface.There is provided additional control option without due to The control of additional display makes real time communication user interface enhance the operability of equipment in a jumble, and makes user-equipment interface More efficient (for example, user's mistake when by helping user to provide input appropriate and reduce operation equipment/interact with equipment), This reduces electricity usage and extends the battery of equipment further through allowing users to more rapidly and efficiently using equipment Service life.
In some embodiments, equipment receives the first ginseng in more than second a participants that instruction indicates in an overflow region And the activity level of person is increased above the work of participant corresponding with the first expression in multiple expressions of more than first a participants Dynamic level is (for example, the participant indicated in an overflow region falls into talk, so that his activity level is increased above painting canvas area The activity level of one of the participant indicated in domain) data, and in response: being participated in in more than second a participants first The first display indicated is (for example, the table of the participant with speech in multiple expressions of expression more than first a participants of replacement of person Show the participant with minimum activity level in replacement canvas area)；And stop showing the table of first participant in overflow area Show.In some embodiments, the expression for stopping showing first participant in an overflow region includes the animation for indicating to shrink.In In some embodiments, other expressions in overflow area are translated to fill the gap that display indicates.In some embodiments, Replacing the first display indicated in multiple expressions of a participant more than first includes first shown in an overflow region with replacement Indicate the expression of corresponding participant.
In some embodiments, when show real time communication session in participant multiple expressions when, equipment receive with Show the corresponding input of request of menu.In response to receiving input corresponding with the display request of menu, equipment is led in real time Menu is shown in credit household interface, wherein menu includes the option for selecting content item (for example, game, video, sketch).In In some embodiments, equipment receives the input for corresponding to the selection of content item, and corresponds to content item in response to receiving Selection input, if additional size be greater than respective threshold size content item expression (for example, content item is shown in painting canvas area In a position in domain) replacement indicates with the size participant that shows for being greater than respective threshold size (for example, canvas area The expression of middle participant).
In some embodiments, the expression for showing more than second a participants in an overflow region includes: according to determining the The participant (for example, 6 participants in total) of one quantity participates in real time communication session, shows first group of expression in an overflow region (for example, two expressions, expression of the view of the camera of expression and equipment including most sluggish participant)；And according to true Surely the participant (for example, 7 participants in total) different from the second quantity of the participant of the first quantity is participating in logical in real time Believe session, in an overflow region display be different from first group of expression second group of expression (for example, three expression, including two most The expression of the view of the camera of the expression and equipment of sluggish participant), wherein second group of expression includes and first group of expression The expression of different number.
It may be noted that the details above in relation to process described in method 900 (for example, Fig. 9 A to Fig. 9 K) can also similar side Formula is suitable for hereafter/method as discussed above.For example, method 900 is optionally included above with reference to 700,1100,1300 and of method The one or more features of various methods described in 1500.
Figure 10 A to Figure 10 S, which is shown, adjusts real time communication session for especially dynamic according to some embodiments The example technique of user interface.User interface in these attached drawings be used to show including the process in Figure 11 A to Figure 11 F Process described below.
Figure 10 A shows the equipment 600 for showing the user interface 1000 of movable real time communication session.User interface 1000 Expression including five participants in the real time communication session that is shown in canvas area 1001.Indicate that 1010A includes not connecting It is connected to the placeholder of the participant of real time communication session.Indicate that 1010B includes only providing the representativeness of the participant of audio data Image.Indicate that 1010C and 1010D includes the real-time video feeding from two corresponding participants.Indicate that 1002 include coming to set The view of standby 600 camera 603.
Each participant has the activity level for determining the layout of user interface 1000.In some embodiments, The activity level of participant based on from participant the received audio of equipment and/or video feed (for example, dynamic in video feed Work or sound).In some embodiments, the activity level of participant is based on the received audio feed of equipment from participant Through filter audio (for example, filtered out ambient noise or filtered with isolation/amplification speech audio audio).In some embodiment party In case, for the activity level of participant based on the movement identified in the received video feed of equipment from participant, the movement is full The mobile standard of one group of foot is (for example, certain type of movement is (for example, hand is mobile, head/face is mobile, is not the shifting of background movement Dynamic (for example, non-participants is mobile)).
In some embodiments, the activity level of participant is detected at the equipment of first participant based on instruction Movable data are (for example, instruction first participant executes the data of user interface action (for example, note at the equipment of participant Meaning power, which is shown, to be indicated or visual effect shows the selection that can be indicated)).In some embodiments, external electronic device (for example, Server) at determine activity level, be then transmitted to equipment 600.
As shown in Figure 10 A, the activity level of the participant of real time communication session is by the following sequence indicated, from up to most It is low are as follows: to indicate 1010C, indicate 1010D, indicate 1010B and indicate 1010A.Table shown on the right side of Figure 10 A to Figure 10 O is by activity Horizontal descending from high to low lists participant, wherein the participant being located at the top of list (1) has highest activity level, and position People in list (4) bottom has the minimum activity level of the participant indicated in canvas area 1001.In Figure 10 A to Figure 10 O In, expression corresponding with the participant with highest activity level is that maximum in canvas area 1001 indicates, and with have The participant of the minimum activity level of the participant indicated in canvas area 1001 is corresponding to be indicated to be the smallest.In Figure 10 A It is corresponding with the participant with the second high activity level and the high activity level of third to indicate with most into Figure 10 O The size of the expression of the participant of high activity level and the minimum movable water with the participant indicated in canvas area 1001 Size between the size of the expression of flat participant.
In some embodiments, the activity level of participant changes relative to other participants.In display user circle When face 1000, equipment 600, which receives instruction activity level corresponding with the participant Stephen of 1010B is indicated and has increased to, to be higher than The data of the activity level of participant Nicholas corresponding with 1010D is indicated.As shown in Figure 10 B, the movable water of 1010B is indicated It is flat to be increased above the activity level for indicating 1010D, but since it has second to arrive highest activity level without changing ruler It is very little.In fig. 1 ob, participant Stephen continues to talk and becomes the participant with highest activity level.In response, table Show that the size of 1010B increases, indicates that the size of 1010C reduces, and indicate 1010A right shift (for example, to avoid due to table Show 1002B size increase and with indicate 1010B it is excessively overlapped), as illustrated in figure 10 c.In some embodiments, equipment 600 Change three or more sizes indicated and/or the position in user interface 1000.
As illustrated in figure 10 c, indicate that 1010A to 1010D overlaps each other.In some embodiments, 1010A is indicated extremely One or more sizes in 1010D reduce, and/or the movement when size increases, to avoid the face for blocking participant And/or it avoids generating the overlapping more than predefined Maximum overlap.In some embodiments, indicate 1010A to 1010D's Front and back (also referred to as " z-order ") does not change when (for example, when participant is actively engaged in) indicating that size increases.In some implementations In scheme, indicate 1010 when changing size (for example, due to correspondence participant be actively engaged in or the selection table at equipment 600 The user's input (for example, inputting 850C in response to user described in figure 8 above M) shown) change one or more times indicated Sequence (for example, being moved to below from moving behind front or from front).For example, in some embodiments, when Figure 10 B extremely In Figure 10 C when increased in size, indicate that 1010B is moved before indicating 1010C and/or indicating 1010A.
In some embodiments, when changing size and/or position, the expression of participant is maintained on display 602 Identical general area in.As shown in Figure 10 A to Figure 10 C, indicate 1010A to 1010D change size and/or position it Afterwards, each expression occupies at least part in its occupied region before changing size and/or position.
Figure 10 D is gone to, indicates that the participant Stephen of 1010B starts to shut up, and as shown in figure 10e, is indicated It is mobile that participant in the real-time video feeding of 1010A starts physics.The movement of participant is detected in real-time video feeding.In In some embodiments, based on the movement detected in the feeding of the real-time video of participant (for example, the physics of instruction participant Mobile movement), determine that participant is actively engaged in real time communication session.In some embodiments, the movement of participant causes The increase of the activity level of participant.In some embodiments, one or more standards are met (for example, continuing according to movement Time, size, direction or predetermined gesture), determine that participant is actively engaged in.
As shown in figure 10f, the data being actively engaged in response to receiving the participant of instruction expression 1010A, equipment 600 increase the size for indicating 1010A and reduce the size for indicating 1010D.Indicate that 1010B keeps identical size and to moving down It is dynamic.In this way, indicate that 1010A is transitioned into medium size from minimum expression and indicates, and indicate that 1010D is transitioned into become most Small expression.
In some embodiments, indicate the participant of 1010A (for example, having minimum activity in response to receiving instruction Horizontal participant) data that are being actively engaged in, the size for indicating 1010A increases to maximum from minimum expression by equipment 600 It indicates, and reduces the size for indicating 1010B.In some embodiments, according to participant's conduct corresponding with 1010A is indicated Nearest participant is actively engaged in, and indicates that 1010A increases to maximum expression from minimum expression.For example, in some embodiments, root It is actively engaged according to the corresponding participant of 1010A is indicated as nearest participant, equipment 600 layout and phase shown in Figure 10 E To layout shown in expression dimensional transitions to Figure 10 G and opposite expression size (being discussed below).
In some embodiments, participant is supplied to the real-time view of real time communication session by the way that visual effect to be applied to Frequency feeding is to be actively engaged in real time communication session.In some embodiments, user circle that participant passes through the equipment of participant Apply visual effect in face.In some embodiments, the visual effect for including in the real-time media feeding based on participant, determines Participant is actively engaged in real time communication session.As shown in figure 10f, visual effect (robot face) is applied to indicate 1010A Real-time video feeding.As a result, determining indicates that the participant of 1010A is actively engaged in and increases the activity level of participant.Such as Shown in Figure 10 F to Figure 10 H, indicate that the participant of 1010A is being actively engaged according to determining, equipment 600 will indicate the ruler of 1010A It is very little to increase to maximum expression size and indicate that 1010B's is reduced in size to medium expression size.Indicate 1010A's and 101B Relative z-order also changes because indicate 1010A due to indicate that the activity level of the associated participant of 1010A increases and It is mobile before indicating 1010B.
In Figure 10 H, equipment 600 receives (for example, detection) user on indicating 1010C and inputs 1050A (for example, light It hits).As shown in Figure 10 I to Figure 10 J, 1050A is inputted in response to user, the size for indicating 1010C is increased to maximum by equipment 600 Indicate size.In addition, corresponding to indicates that the participant of 1010D starts to be actively engaged in (for example, talk), make to indicate that 1010D increases To medium size, and make to indicate that 1010B is reduced to small size.
In Figure 10 J, equipment 600 receives (for example, detection) user at the position of display 602 and inputs 1050B (example Such as, tap), do not show the expression of participant at this location.As shown in Figure 10 K, 1050B, equipment 600 are inputted in response to user It shows Call- Control1 menu 1004, is similar to above-mentioned Call- Control1 menu 630.
As shown in Figure 10 K, when showing Call- Control1 menu 1004, the size of canvas area 1001 reduces, so that calling It is not Chong Die with expression 1010 to control menu 1004.In order to be 1004 vacating space of Call- Control1 menu, equipment 600 is by reducing ruler Very little and/or upward vertical translation one or more indicates to modify the layout for indicating 1010.In some embodiments, equipment 600 reduce the size of the expression 1010A and 1010D of medium sizes, and maintain respectively small and big main expression 1010B and The size of 1010C.In some embodiments, equipment 600 reduces between expression (for example, indicating between 1010A and 1010C) Vertical interval.
1050B is inputted in response to receiving user, other than the expression 1002 of the user of equipment 600, equipment 600 also exists Band 1012A to 1012D is shown in each expression.Each band includes the name of corresponding participant and showing and capable of indicating for amplification expression.
Figure 10 L is gone to, equipment 600 is shown with laterally (level) orientation.In the opposite activity level and Figure 10 J of participant It is identical.Compared with (vertical) orientation in the longitudinal direction in Figure 10 J, indicate to occupy the equally oriented with longitudinal (vertical) of display 602 Corresponding region, and indicate in image rotated so that participant look like it is upright.In some embodiments, it rings Should in equipment 600 from the machine-direction oriented horizontal orientation for rotating to Figure 10 L of Figure 10 J, animation rotation that the display of equipment 600 indicates or Image (for example, 90 degree opposite with the direction of rotation of equipment 600) in expression.
Figure 10 M is gone to, since being laid out shown in Figure 10 J with activity level, the 6th participant Pablo is added logical in real time Believe session.As shown in Figure 10 M, real time communication session is added in response to the 6th participant, and lead in real time according to determining to be connected to Believe that participant's quantity of session is greater than the predetermined threshold of five connection participants, the image of the participant newly connected of equipment 600 Replacement indicates image corresponding with having the participant of minimum activity level in 1010B.Equipment 600, which optionally provides, newly to be connected The notice of real time communication session has been added in participant, and shows that the spilling of the participant newly connected indicates in overflow area 1020 1022A.As shown in Figure 10 M to Figure 10 O, when showing overflow area 1020, the size of canvas area 1001 reduces.It indicates At least some of 1010A to 1010D dimensionally reduces and/or moves on display 602.
As shown in Figure 10 N, overflowing indicates that 1010A corresponds to not by mainly indicating (for example, the overflow area in Figure 10 N 1020 top expression) indicate participant.In Figure 10 N, equipment 600 receives (for example, detection) on overflowing expression 1022A User inputs 1050C (for example, tap).As shown in fig. 10o, 1050C is inputted in response to user, equipment 600 is selected with corresponding to The spilling selected indicates that the image replacement of the participant of 1022A indicates the image in 1010A, and shows the participation for corresponding to and being replaced The spilling of person (Hoan) indicates 1022B.
In some embodiments, in response to indicating that 1022A corresponding participant is actively engaged in (for example, generation with overflowing 1050C is inputted in response to user), equipment 600 indicates that the image of the corresponding participant of 1022A replaces Figure 10 N's with overflowing Indicate the image in 1010A.In some embodiments, the work in response to determining with overflowing the corresponding participant of expression 1022A Dynamic level has increased to the activity level for being higher than participant corresponding with the expression 1010A of Figure 10 N, and equipment 600 is used and overflow table Show that the image of the corresponding participant of 1022A replaces the image of Figure 10 N indicated in 1010A.
Figure 10 P is gone to, the equipment 1008 with display 1014 is shown.In some embodiments, display 1014 With the feature (for example, size and/or aspect ratio) different from the display 602 of equipment 600.Equipment 1008 shows user interface 1016, it is divided into two parts 1016A (for example, canvas area) and 1016B.Part 1016A has the display with equipment 600 The roughly the same aspect ratio of device 602.According to the aspect ratio of part 1016A, part 1016A includes indicating 1030A to 1030D, right It should be in the participant of the expression 1010A to 1010D in Figure 10 O.As shown in Figure 10 P, the layout (example of 1030A to 1030D is indicated Such as, relative size, position, interval and overlapping) it is similar to the layout for indicating 1010A to 1010D in Figure 10 O.
As shown in Figure 10 P, equipment 1008 receives (for example, detection) user and inputs 1050D (for example, to the right gently sweep) with will Separator 1018 moves right.In response, equipment 1008 reduces the size of part 1016B and increases the size of part 1016A. As shown in Figure 10 P to Figure 10 S, indicate that 1030 are amplified and indicate that the horizontal interval between 1030 increases.As shown in Figure 10 S, Equipment 1008 shows menu 1024, show with effect can indicate 1026, calling show can indicate 1028 and menu show and can indicate 1034 (its effect similarly as described above, which is shown, can indicate 632, calling show can indicate 634 and menu show can indicate 636).
Figure 11 A to Figure 11 F is the flow chart for showing the method according to some embodiments.In the equipment with display Method 1100 is executed at (for example, 100,300,500,600).Some operations in method 1100 are optionally combined, Yi Xiecao The order of work is optionally changed, and some operations are optionally omitted.
As described below, method 1100 is provided for providing the intuitive side of adaptive and intelligent real time communication user interface Formula, the user interface emphasize that one or more of real time communication session enlivens participant.This method alleviates user and checks Cognitive load when enlivening participant in real time communication session, to create a kind of more effective man-machine interface.For electricity The calculating equipment of pond driving, allows users to more rapidly and more effectively checks enliven participant, save power and increase Battery twice charge between interval.
At frame 1102, equipment shows live video communication user interface (for example, 1000) comprising while display is real-time The expression (for example, 1010C) of first participant in video communication sessions, second participant in live video communication session Indicate the expression (for example, 1010D) of (for example, 1010B) and the third participant in live video communication session.In some realities It applies in scheme, if participant is sharing the real-time media stream including live video stream, which includes live video stream Image (for example, 1010C).It in some embodiments, should if participant shares real-time media stream with only audio Indicate the head portrait (for example, 1010B) including corresponding participant.In some embodiments, if reality is not added currently in participant When video communication sessions, then the expression includes placeholder image (for example, head portrait) (for example, 1010A).In some embodiments In, real time communication interface includes canvas area (for example, 1001) (for example, canvas area (for example, main region) is different from overflowing Region).In some embodiments, canvas area is close to overflow area.In some embodiments, it is shown in canvas area The expression of first participant in real time communication session, the expression and real time communication of second participant in real time communication session The expression (for example, Figure 10 A) of third participant in session.
At frame 1104, while showing live video communication user interface, equipment receives instruction live video communication The activity level of second participant (for example, 1010B) has increased in session participates in higher than in live video communication session first The data of the activity level of person (for example, 1010C).In some embodiments, it receives and indicates from external source (for example, server) The data that first participant is being actively engaged in.In some embodiments, equipment is based on received data (for example, coming from The real-time video or real-time audio of associated with first participant real-time media feeding) determine the first participant actively It participates in.In some embodiments, increased in the activity level for receiving the second participant in instruction live video communication session It is added to before the data of the activity level for the first participant being higher than in live video communication session, first participant, the second ginseng Indicate Chong Die at least one other expression of participant (for example, Figure 10 A with one or more in person and third participant 1010B, 1010C and 1010D).
In some embodiments, at least the activity level of first participant is based on (for example, according to) from first participant The received audio of equipment and/or video feed (for example, movement or sound in video feed) and/or instruction first participate in The movable data detected at the equipment of person are (for example, instruction first participant executes user at the equipment of first participant The data of interface action (for example, Selective attention power, which is shown, to be indicated)).In some embodiments, in external electronic device (example Such as, server) at determine activity level, be then transmitted to electronic equipment.In some embodiments, it at least first participates in The activity level of person is based on the received audio feed of equipment from first participant through filter audio (for example, having filtered out background Noise is filtered with isolation/amplification speech audio audio).In some embodiments, at least activity of first participant Level is based on the movement identified in the received video feed of equipment from first participant (for example, identifying at server；In Identified at electronic equipment), which meets one group of mobile standard (for example, 1010A of Figure 10 E) (for example, certain type of movement (for example, hand is mobile, head/face is mobile, is not the movement (for example, non-user is mobile) of background movement).
At frame 1106, in response to receiving the activity level (example of second participant in instruction live video communication session Such as, the 1010B of Figure 10 A) activity level higher than first participant in live video communication session is had increased to (for example, Figure 10 A 1010C) data, equipment change live video communication user interface in first participant expression size and/or position (at frame 1108), change live video communication user interface in second participant expression size and/or position (in frame At 1110), and change the expression (for example, 1010D) of third participant in live video communication user interface size and/or Position (at frame 1112).The size and/or position for changing expression when a given condition is fulfilled allow user quickly to identify participation The activity level of person (including most active and/or most sluggish participant).One group of condition be satisfied without into Optimized operation is executed when user's input of one step and enhances the operability of equipment, and keeps user-equipment interface more effective (for example, by help user provide suitably input and reduce operation equipment/with equipment interact when user's mistake), this Further through allowing users to faster and more effectively reduce electricity usage using equipment and extend the battery longevity of equipment Life.In addition, only changing the size indicated and/or position when meeting rated condition reduces user's determination and enlivens participant or pass The quantity of input needed for note enlivens participant.The quantity for reducing input also enhances the operability of equipment, and makes user- Equipment interface is more effective (for example, when suitably inputting and reduce operation equipment/interact with equipment by helping user to provide User's mistake), this is further through allowing users to faster and more effectively reduce electricity usage using equipment and extend The battery life of equipment.
In some embodiments, the movable water of second participant in live video communication session is indicated in response to receiving It puts down and has increased to the data higher than the activity level of first participant in live video communication session, in real time communication user interface The size of the expression (for example, 1010C of Figure 10 A) of first participant changes, second participant in real time communication user interface Indicate (for example, 1010B of Figure 10 A) size change, and in real time communication user interface third participant expression (example Such as, the 1010D of Figure 10 A) size change.In some embodiments, if most sluggish participant becomes most active Participant's (for example, be based on activity score), then equipment increases the size of the expression of most active participant, and reduce it is all its The size (for example, Figure 10 E to Figure 10 F) that he indicates.In some embodiments, if most active participant becomes least living Participant's (for example, being based on activity score) of jump, then equipment reduces the size of the expression of most sluggish participant, and increases institute The size for thering are other to indicate.In some embodiments, in response to receiving the second participation in instruction live video communication session The activity level of person has increased to the data higher than the activity level of first participant in live video communication session: real time communication The position change of the expression of first participant in user interface；The position of the expression of second participant in real time communication user interface Change；And the position change of the expression of third participant in real time communication user interface.In some embodiments, if most Sluggish participant becomes most active participant (for example, being based on activity score), then exchanges all positions (for example, mostly The expression that number enlivens participant is moved up or is removed, and other expressions are moved down or removed).In some embodiments, If most active participant becomes most sluggish participant's (for example, based on activity score), most sluggish participant Expression move down and/or remove, and other expressions are moved up and/or are removed.
In some embodiments, with the highest movable water of (for example, currently having) the first, second, and third participant First, second and the in a part of the expression (for example, 1010C of Figure 10 A) of flat participant and real time communication user interface A part overlapping of the expression of another participant in three participants is (for example, a part of most active expression is under other expressions Side).In some embodiments, display has the expression of the participant of minimum activity level without with any other participant's Expression overlapping (for example, because the expression of the participant with minimum activity level is the minimum expression in the expression of participant, and And be therefore maintained on the expression of other participants, to avoid the table for excessively hiding the hidden participant with minimum activity level Show).In some embodiments, the expression (for example, 1010B of Figure 10 I) of the participant with minimum activity level is shown It is Chong Die at least part of expression (for example, 1010A of Figure 10 I) with the participant with highest activity level.Some In embodiment, the table of the participant of the minimum activity level with (for example, currently having) the first, second, and third participant Show a part of (for example, 1010D of Figure 10 H) with it is another in the first, second, and third participant in real time communication user interface A part overlapping of the expression (for example, 1010C of Figure 10 H) of participant is (for example, a part of most sluggish expression is at other Indicate lower section).
Optionally, at frame 1114, equipment is based on video feed (and/or the view of third participant from second participant Frequency is fed) obtain the face recognition data of (for example, by analysis acquisition) change the size of the expression of first participant and/or Position, wherein change first participant expression size and/or position not will lead to hide hidden second participant face (and/ Or hide the face of hidden third participant).Change in real time communication session in the case where not hiding the face of hidden another participant The size of the expression of participant and/or position provide a user the feedback of the current active level about participant, without hiding The face of other participants in real time communication session, thus the size for the participant for allowing user to have changed in activity level or position Set the face that other participants are checked when change.Improved visual feedback, which is provided, for user just enhances the operability of equipment, And make user-equipment interface it is more efficient (for example, by help user provide it is appropriate input and reduce operation equipment/with set User's mistake when standby interaction), this reduces electricity usage further through allowing users to more rapidly and efficiently using equipment And extend the battery life of equipment.
Optionally, at frame 1116, equipment changes the expression (for example, 1010B of Figure 10 A to Figure 10 C) of first participant Size and/or position include that the size of expression is gradually increased into (for example, gradually increasing whithin a period of time) from first size To the second size.Change the size indicated and/or position provides a user the feedback of the current active level about participant, together When minimize user interface change.Improved visual feedback is provided for user and just enhances the operability of equipment, and is used Family-equipment interface it is more efficient (for example, by help user provide it is appropriate input and reduce operation equipment/when being interacted with equipment User's mistake), this reduces electricity usage and extends further through allowing users to more rapidly and efficiently using equipment The battery life of equipment.The size for gradually increasing expression provides a user visual feedback, and instruction is when its activity level changes The activity level of participant is changing, while minimizing the sudden change of size to interference caused by user security risk.It is provided for user Improved visual feedback just enhances the operability of equipment, and keeps user-equipment interface more efficient (for example, passing through help User's mistake when user provides input appropriate and reduces operation equipment/interact with equipment), this is further through allowing users to more Quickly and efficiently reduce electricity usage using equipment and extends the battery life of equipment.
Optionally, at frame 1118, equipment detects canvas size and changes event (for example, 1050B；Figure 10 J to Figure 10 K) (for example, the rotation of equipment；By other elements (for example, one or more callings for contacting participant are shown and can be indicated；Addition The expression of the participant shown outside canvas area) it is introduced into live video communication user interface；With change canvas area size Request corresponding user input).Optionally, at frame 1120, in response to detecting that canvas size changes event, equipment changes The size of canvas area (for example, 1001) (optionally, at frame 1124).In response to detecting that canvas size changes event, if The layout of the expression (for example, 1010B, 1010C and 1010D of Figure 10 K) of the first, second, and third participant of standby modification.One In a little embodiments, the layout (example of expression is modified by the size for indicating the displacement of expression and/or change Such as, so as to the interval when the size of canvas area reduces between reduction expression).Size based on canvas area changes intelligently Modification layout allows user when canvas size changes to check indicating for user to the shown minimum change indicated, thus The interference to user is reduced, while user being allowed to check shown displaying.One group of condition be satisfied without into Optimized operation is executed when user's input of one step and enhances the operability of equipment, and keeps user-equipment interface more effective (for example, by help user provide suitably input and reduce operation equipment/with equipment interact when user's mistake), this Further through allowing users to faster and more effectively reduce electricity usage using equipment and extend the battery longevity of equipment Life.
Optionally, at frame 1126, equipment is additionally in response to detect that canvas size changes event, changes the first, second He One or more sizes for indicating (for example, 1010B, 1010C and 1010D of Figure 10 K) of third participant.In some embodiment party In case, the activity level of second participant is had increased to logical higher than real-time video in receiving instruction live video communication session Believe in session before the data of the activity level of first participant, first, second, and third participates in real time communication user interface The expression (for example, 1010B, 1010C and 1010D of Figure 10 K) of person is shown according to z-order (for example, " z-order " refers to participating in The expression of person relative to the imaginary axis (for example, the axis come out from screen) in the face perpendicular to display order (for example, when two tables When the position overlapping shown, the z-order of expression determines which object is shown in front of another object).At frame 1128, equipment dimension Hold the z-order of the expression (for example, 1010B, 1010C and 1010D of Figure 10 K) of the first, second, and third participant.
In some embodiments, change real time communication user interface in first participant expression (for example, Figure 10 B 1010B) size and/or position after, the expression of the change of first participant occupy at least part in region (for example, General area, position, pixel on display etc.), the region is in the size or position for changing the expression of first participant It is occupied before at least one by the expression of first participant in real time communication user interface.In some embodiments, changing After size, participant show in the display in participant previously locating identical region (for example, at least one indicated (for example, part, pixel, position etc.) is divided to continue to contact the position on the display of its prior exposure before changing).Some In embodiment, indicate the arrangement/general position for maintaining to indicate while emphasizing to enliven participant (for example, indicating with it Size increase/reduction and shift but maintain position relative to other images).In some embodiments, changing In real time communication user interface after the size of the expression (for example, 1010C of Figure 10 B) of second participant and/or position, second The expression of the change of participant occupies at least part (for example, general area, position, pixel on display etc.) in region, The region is before changing at least one of the size of expression of second participant or position by real time communication user interface The expression of second participant occupies.In some embodiments, after changing size, participant show in the display with ginseng (for example, at least part (for example, part, pixel, position etc.) indicated continues to contact in identical region locating for previous with person Position on the display of its prior exposure before changing).In some embodiments, it indicates emphasizing to enliven participant's Arrangement/the general position indicated is maintained (still to maintain phase for example, indicating to shift with their size increase/reduction simultaneously For the position of other images).In some embodiments, change real time communication user interface in third participant expression After the size of (for example, 1010D of Figure 10 B) and/or position, the expression of the change of third participant occupies at least the one of region Partially (for example, general area, position, pixel on display etc.), size of the region in the expression for changing third participant Or it is occupied before at least one of position by the expression of third participant in real time communication user interface.In some embodiments In, after changing size, participant show in the display in participant previously locating identical region (for example, indicating At least part (for example, part, pixel, position etc.) continue position of the contact before changing on the display of its prior exposure It sets).In some embodiments, expression maintains their arrangement and/or general position (example while emphasizing to enliven participant Such as, it indicates to shift the position still maintained relative to other images with their size increase/reduction).
Display occupies at least part in their previously occupied regions after changing the size indicated and/or position Expression, allow user the activity level of participant is identified with the minimum change to user interface, thus increase user concern The ability of the expression of participant in real time communication session.It has been satisfied in one group of condition and has been inputted without further user Shi Zhihang optimized operation enhances the operability of equipment, and keeps user-equipment interface more effective (for example, passing through help User provides user's mistake when suitably inputting and reduce operation equipment/interact with equipment), this is further through enabling users to Enough battery lifes for faster and more effectively reducing electricity usage using equipment and extend equipment.
Optionally, at frame 1130, equipment show real time communication user interface in the 4th participant expression (for example, The 1010D of Figure 10 E).Optionally, in response to having received the activity level of second participant in instruction live video communication session The data of the activity level of first participant in live video communication session are increased above, equipment changes real time communication user circle The size of the expression of 4th participant and/or position in face (at frame 1132).In some embodiments, when every other table Show that the 4th indicates to change size and/or position when changing size or position.In some embodiments, it the 4th indicates to change (example Such as, it has increased in response to receiving the activity level of second participant in instruction real time communication session higher than real time communication session First data of the activity level of middle first participant).In some embodiments, some to indicate that the such as the 4th indicates to maintain Its size and position, even if other expressions are changing.
Optionally, at frame 1134,1136,1138,1140 and 1142, the in changing live video communication user interface After the size of the expression of one participant and/or position, equipment receives first participant in instruction live video communication session Activity level has increased to the second data higher than the activity level of second participant in live video communication session.In response to connecing The activity level for receiving first participant in instruction live video communication session has increased to higher than in live video communication session Second data of the activity level of second participant, equipment change the expression of first participant in live video communication user interface Size and/or position.Equipment changes the size of the expression of second participant and/or position in live video communication user interface It sets.Equipment maintains the size of the expression of third participant and position in real time communication session.In some embodiments, in response to The change of the activity level of participant maintains the expression of some expressions shown in painting canvas (for example, main region), even if its He indicates to change size or position (for example, two indicate that position can be exchanged based on other expression its sizes of change and position is determined It sets).In some embodiments, maintained to indicate to correspond to the movable water with the participant indicated is changed than corresponding to Put down the participant of low activity level.When a given condition is fulfilled, it is tieed up while changing the size and/or expression that other are indicated The size and/or position permission user of holding some expressions quickly identify the movable water of participant with the minimum change to user interface It is flat, to increase the ability that user pays close attention to the expression of participant in real time communication session.One group of condition be satisfied without Optimized operation is executed when further user being needed to input and enhances the operability of equipment, and makes user-equipment interface More effectively (for example, wrong by the user for helping user to provide when suitably inputting and reduce operation equipment/interact with equipment Accidentally), this is further through allowing users to faster and more effectively reduce electricity usage using equipment and extend equipment Battery life.
Optionally, at frame 1144 and 1146, in response to receiving the work of second participant in instruction real time communication session Dynamic level has increased to the first data higher than the activity level of first participant in real time communication session, equipment emphasize (for example, Increase or initiate) expression of second participant's (for example, when second participant currently more enlivens) in real time communication user interface Visually indicate (for example, highlight expression (for example, display bezel), animation (for example, pulse head portrait), make image it is preposition, increase Add size).Activity level in response to receiving second participant in instruction real time communication session has increased to be higher than leads in real time Believe that the first data of the activity level of first participant in session, equipment do not emphasize that (for example, reduce or stop) real time communication is used The expression of first participant's (for example, when first participant currently less enlivens) visually indicates (for example, prominent in the interface of family Display indicates (for example, display bezel), animation (for example, pulse head portrait), makes image postposition, reduces size).Item is provided when meeting It emphasizes and/or does not emphasize to visually indicate that user is allowed quickly to identify participant's (including most active and/or most sluggish ginseng when part With person) activity level.It has been satisfied in one group of condition without executing optimized operation when further user input The operability of equipment is enhanced, and keeps user-equipment interface more effective (for example, suitable defeated by helping user to provide User's mistake when entering and reducing operation equipment/interact with equipment), this is further through allowing users to faster and more effectively Reduce electricity usage using equipment and extends the battery life of equipment.In addition, emphasize when a given condition is fulfilled and/ Or do not emphasize the visual feedback that the activity level for indicating each participant is provided for user.Improved visual feedback is provided for user The operability of equipment is just enhanced, and keeps user-equipment interface more efficient (for example, appropriate by helping user to provide User's mistake when inputting and reducing operation equipment/interact with equipment), this is further through allowing users to more rapidly and efficiently make Reduce electricity usage with equipment and extends the battery life of equipment.
In some embodiments, the movable water of second participant in instruction live video communication session is received in equipment It is flat to have increased to higher than before the data of the activity level of first participant in live video communication session (for example, in the past): the The expression of one participant is with first size (for example, the intermediate sizes in one group of 3 or more predefined size；In the group Full-size) display, the expression of second participant is to be less than the second size of first size (for example, one group 3 or more are pre- Define the minimum dimension in size；Intermediate sizes in the group) display (for example, small expression).Optionally, at frame 1148, if The size of the standby expression by second participant increases to first size from the second size.By the size of expression and/or position from compared with Small size increases to larger size and provides a user visual feedback, indicates the activity level of the participant when its activity level changes Changing.Improved visual feedback is provided for user and just enhances the operability of equipment, and makes user-equipment interface more Efficiently (for example, user's mistake when by helping user to provide input appropriate and reduce operation equipment/interact with equipment), this Reduce electricity usage further through allowing users to more rapidly and efficiently using equipment and extend battery longevity of equipment Life.
Optionally, at frame 1150 and 1152, the work of second participant in receiving instruction live video communication session Dynamic level has increased to higher than after the data of the activity level of first participant in live video communication session, equipment detection with The corresponding input (for example, 850C) (for example, tap notationally) of the expression of first participant.In response to detecting and first The corresponding input of the expression of participant is (for example, user can be changed in real time communication user interface (for example, in overflow area, drawing In cloth region etc.) display any expression size, equipment increase first participant activity level (for example, Fig. 8 L to Fig. 8 M 810F) value of activity level (for example, increase).
At frame 1154 and 1156, the 4th participant's ratio that equipment receives in instruction real time communication session has first, the Two and third participant minimum activity level the more active data of participant.In response to receiving instruction real time communication session In the 4th participant than the minimum activity level with the first, second, and third participant the more active data of participant, There is the first, second, and third participant most with the display replacement for indicating (for example, 1010A of Figure 10 O) of the 4th participant The display of the expression (for example, 1010A of Figure 10 N) of the participant of low activity level.In some embodiments, in frame 1158 Place, equipment show indicating for the 4th participant with the following size, which, which is greater than, has the first, second, and third participant most The size of the expression of the participant of low activity level.When a given condition is fulfilled, with the participation not shown in indicating at first group Person's replacement has the display of the expression of the participant of minimum activity level, and user is allowed to pay close attention to most active ginseng in real time communication session User is reduced to real-time and when not shown user more enlivens than the user shown in first group of expression with the expression of person The concern of most sluggish participant in communication session.It has been satisfied in one group of condition and has been inputted without further user Shi Zhihang optimized operation enhances the operability of equipment, and keeps user-equipment interface more effective (for example, passing through help User provides user's mistake when suitably inputting and reduce operation equipment/interact with equipment), this is further through enabling users to Enough battery lifes for faster and more effectively reducing electricity usage using equipment and extend equipment.What increase was newly shown The size of participant provides a user about relevant to new participant participant shown by other (for example, the first of expression In subset) current active level feedback, while minimize user interface change.Improved visual feedback is provided just for user The operability of equipment is enhanced, and keeps user-equipment interface more efficient (for example, appropriate defeated by helping user to provide User's mistake when entering and reducing operation equipment/interact with equipment), this is further through allowing users to more rapidly and efficiently use Equipment and reduce electricity usage and extend the battery life of equipment.
Optionally, at frame 1160, equipment shows the view of the camera of electronic equipment in real time communication session subscriber interface Scheme the expression (for example, 1002) of (for example, self view).Optionally, at frame 1162, equipment is additionally in response to receive instruction in fact When communication session in the activity level of second participant have increased to the movable water higher than first participant in real time communication session The first flat data: optionally, at frame 1164, the view of the camera of electronic equipment is maintained in real time communication session subscriber interface The size of the expression (for example, 1002) of figure.In some embodiments, self view will not be based on the movable water of other participants Flat change and change.The size of the expression of the view of the camera of equipment is maintained, while other expressions can meet one group of item Change when part, user is allowed to check that their expression simultaneously continuously minimizes the change to layout.One group of condition by Meet and enhance the operability of equipment without executing optimized operation when further user input, and makes user- Equipment interface is more effective (for example, when suitably inputting and reduce operation equipment/interact with equipment by helping user to provide User's mistake), this is further through allowing users to faster and more effectively reduce electricity usage using equipment and extend The battery life of equipment.
It may be noted that can also be with class above in relation to the details of process described in method 1100 (for example, Figure 11 A to Figure 11 F) It is suitable for hereafter/method as discussed above like mode.For example, method 1100 optionally include above with reference to method 700,900, The one or more features of various methods described in 1300 or 1500.
Figure 12 A to Figure 12 N is shown according to some embodiments for especially initiating in fact from instant message application program When communication session exemplary user interface.User interface in these attached drawings be used to show including in Figure 13 A to Figure 13 D The process described below of process.
Figure 12 A shows the (institute as above of equipment 600 of the instant message user interface 1200 of display instant message application program It states).Instant message user interface 1200 includes the participation in the group (" Dream Team ") of 15 participants for participating in message conversation The message in message region 1202 between person.Instant message user interface 1200 includes for initiating and the institute in message conversation Have the calling of the real time communication session of participant (for example, all participants in " Dream Team " group) all show can indicate 1206 with And group's contact information for accessing additional information relevant to " Dream Team " group is shown can indicate 1208.
As illustrated in fig. 12, the reception of equipment 600 (for example, detection) is all shown with calling can indicate that 1206 selection is corresponding User inputs 1250A (for example, tap).1250A is inputted in response to receiving user, equipment 600 starts and as real time communication The real time communication session of all participants in the message conversation that participant in session includes.In some embodiments, it responds 1250A is inputted in receiving user, equipment 600 sends instruction, and the instruction is so that equipment associated with the member of this group is shown The notice that the real time communication session is in active state.In this way, calling, which is all shown, can indicate that 1206 immediately begin to and disappear The real time communication session of all participants in breath dialogue group.
Figure 12 B shows the instant message user interface of the instant message application program when initiating real time communication session 1200.Instant message user interface 1200 includes that the vision at 602 top of message 1212 and display in message region 1202 refers to Show symbol 1214, instruction real time communication session is in active state.
In some embodiments, 1250A is inputted in response to receiving user, equipment 600 starts real time communication application journey Sequence.As indicated in fig. 12 c, after real time communication session start, equipment 600 shows that the real time communication of real time communication application program is used Family interface 1216.Real time communication user interface 1216 corresponds to the real time communication session started with " Dream Team " group.
As indicated in fig. 12 c, the main expression 1210A of real time communication user interface 1216 including four corresponding participants is extremely The user of 1210D (being referred to as 1210) (for example, first four participant in the participant list of " Dream Team " group) and equipment 600 User indicate 1218.Real time communication user interface 1216 further includes overflow area 1220, which has and lead in real time Believe that the corresponding spilling of the participant (for example, each member of " Dream Team " group) of session indicates 1222.It is main to indicate that 1210B includes The real time communication session with video data has been added in real-time video from participant Allison, instruction Allison.
It is main to indicate that 1201A, 1210C and 1210D include placeholder, indicate that corresponding participant is not yet added (for example, not It is connected to) real time communication session.Placeholder include participant's name text representation (for example, name or first and last A initial).Optionally, placeholder includes indicating that participant is waiting to connect to the status indicator of real time communication session, And it can indicate for showing to the prompting of the corresponding participant transmission available prompting of real time communication session (for example, " the tinkle of bells " 808A).In some embodiments, it reminds and shows that can indicate initially does not include on placeholder, and according to determining preparatory true Real time communication session is not added to show in corresponding participant after the fixed period.
In some embodiments, the detection of equipment 600 is shown that the selection that can be indicated is corresponding with prompting and is inputted (for example, mentioning Wake up and show the tap on capable of indicating), and in response, instruction is sent to cause newly to notify (for example, calling, the tinkle of bells etc.), is referred to Show that real time communication session can be used at the associated equipment of corresponding participant, wherein new notice is different from being previously transmitted to participate in The notice of person.In some embodiments, new notice invasion property more than previous notification is (for example, defeated with no audio or tactile Banner notice out is compared, audio the tinkle of bells, tactile output or full frame notice).In some embodiments, it reminds and shows and can indicate Overflow indicate display in 1222 (for example, according in main indicate display alarm show the identical standard that can be indicated).One A bit in embodiments, selection prompting, which is shown, on overflowing expression can indicate to obtain and above for the prompting shown in main expression Show identical response (for example, new, more invasion property the notice issued to corresponding participant) described in energy list item.
In some embodiments, real time communication session has been added by one of the participant that placeholder indicates according to determining, Equipment 600 modifies associated with the user indicate (for example, replacing placeholder with real-time video；See, for example, Fig. 8 C to Fig. 8 D). In some embodiments, modification indicates to include animation.In some embodiments, equipment 600 is from server or other equipment Data are received to determine whether participant has been added session.In some embodiments, equipment 600 is based on receiving and participant's phase Associated real time media data determines that session has been added in participant.
In some embodiments, real time communication session is not yet added according to one of determining participant indicated by placeholder Up to predetermined time quantum, equipment 600 changes the layout (example of the expression shown in real time communication session subscriber interface 1216 Such as, equipment 600 stops display and reaches the corresponding expression of the participant of predetermined time quantum with real time communication session is not added, and Display has the less layout mainly indicated or indicates to replace indicating for the participant with another participant；See, for example, figure 8E to Fig. 8 F).In some embodiments, equipment 600 receives data from server or other equipment whether to determine participant Real time communication session is not yet added and reaches predetermined time quantum.In some embodiments, equipment 600 be based on do not receive with The associated real time media data of participant come determine participant whether not yet be added real time communication session up to it is predetermined when The area of a room.
Figure 12 D shows real time communication user circle in the time later of the real time communication ession for telecommunication with " Dream Team " group Face 1216.As indicated in fig. 12d, real time communication user interface 1216 includes Call- Control1 menu 1224, is similar to above-mentioned calling Control menu 630.As described above, in some embodiments, Call- Control1 menu 1224 is inputted in response to user (for example, aobvious Show the tap on device 602 or gently sweep upwards) and be shown.As indicated in fig. 12d, equipment 600 receives (for example, detection) and exhales with extension The corresponding user of request of control menu 1224 is made to input 1250B (for example, gently sweeping upwards).As shown in figure 12e, in response to user 1250B is inputted, equipment 600 extends Call- Control1 menu 1224.The Call- Control1 menu 1224 of extension is similar to about Fig. 6 I extremely The Call- Control1 menu 630 of extension described in Fig. 6 K.As shown in figure 12e, Call- Control1 menu 1224 includes and participant Hoan Associated prompting, which is shown, can indicate 1226.In some embodiments, equipment 600, which is detected, shows the selection that can indicate 1226 with prompting Corresponding input (for example, showing the tap that can be indicated on 1226 reminding), and in response, instruction is sent to cause newly to notify (for example, calling, the tinkle of bells etc.) indicates that real time communication session can be used at equipment associated with participant Hoan, wherein newly Notice is different from being previously transmitted to the notice of participant Hoan.In some embodiments, new notice is more invaded than previous notification Immunity (for example, compared with banner notice that no audio or tactile export, audio the tinkle of bells, tactile output or full frame notice).
Figure 12 F again illustrates real time communication user circle before starting real time communication session with group of messages " Dream Team " Face 1216.As shown in Figure 12 F, equipment 600 receive (for example, detection) show with group contact person can indicate 1208 the corresponding use of selection Family inputs 1250C (for example, tap).As shown in fig. 12g, 1250C is inputted in response to receiving user, equipment 600 shows group's connection It is people's user interface 1228, shows additional information relevant to " Dream Team " group, the real time communication including participant in this group Session, which is shown, can indicate 1230 and list 1232.Group's contacts user interface 1228 further includes that addition contact person shows and can indicate 1234. In some embodiments, show that can indicate that 1234 selection is corresponding inputs (for example, light with addition contact person in response to detecting Hit), equipment 600 initiates the process for new contact person to be added to real time communication session (for example, having text defeated by display Enter field for using the user interface of keyboard input participant's name and/or for from the menu of such as contacts list The addition participant of selection participant, which shows, can indicate (for example, as described in below with reference to Figure 14 F)).
As shown in Figure 12 H, equipment 600 receive (for example, detection) show with real time communication session can indicate 1230 selection pair The user answered inputs 1250D (for example, tap).1250D is inputted in response to receiving user, equipment 600 starts real time communication and answers With program and the real time communication session of beginning and the participant in " Dream Team " group (for example, as indicated in fig. 12 c).
User can also select the expression of the participant in group in group's contacts user interface 1228, with access with it is corresponding The relevant additional information of participant and option.As shown in figure 12i, equipment 600 receives (for example, detection) and participant Allison The corresponding user of selection of list items 1232B input 1250E (for example, tap).1250E, equipment 600 are inputted in response to user Display has additional information associated with selected participant and for individually being communicated with participant by various communication patterns Option 1238A to 1238E independent contacts user interface 1236.As shown in Figure 12 J, independent contacts user interface 1236 It especially include showing and capable of indicating 1238A, 1238B and 1238C, for initiating to talk with the single message of participant Allison respectively, Independent call and independent real time communication session (for example, video communication).
In some embodiments, whether the characteristic strength that equipment 600 determines that user inputs 1250E is more than threshold intensity. It is more than threshold intensity according to the characteristic strength for determining that user inputs 1250E, equipment 600 is in group contact person user as shown in Figure 12 K Menu 1240 is shown on interface 1228, wherein option 1242A to 1242E corresponds to and individually led to participant Allison The various communication patterns of letter.
Figure 12 L to Figure 12 N respectively illustrates illustrative 4 pair 1,3 pairs 1 and 2 pair 1 layout.In some embodiments, root According to the real time communication session including five participants (user including equipment 600), equipment 600 is shown shown in Figure 12 L just Begin one of layout.In some embodiments, according to the real time communication meeting including four participants (user including equipment 600) Words, equipment 600 show one of initial layout shown in Figure 12 M.In some embodiments, according to including three participants The real time communication session of (user including equipment 600), equipment 600 show one of initial layout shown in Figure 12 N.Some In embodiment, equipment 600 shows the expression of the participant in one of layout shown in Figure 12 L to Figure 12 N, as when based on quilt Initial layout when the quantity initiation real time communication session for the participant for inviting and/or connecting is (for example, instead of shown in Figure 12 C Layout).For example, in some embodiments, according to the real time communication session including a total of four participant, equipment 600 According to the expression of one of layout display participant shown in Figure 12 M.
Figure 13 A to Figure 13 D is the flow chart for showing the method according to some embodiments.In the equipment with display Method 1300 is executed at (for example, 100,300,500,600).Some operations in method 1300 are optionally combined, Yi Xiecao The order of work is optionally changed, and some operations are optionally omitted.
As described below, method 1300 is provided for initiating the real time communication session between multiple participants and interacting Intuitive manner.This method is by providing the intelligence initiation to the real time communication session between multiple participants and interacting reduction The cognitive load of user, to create more effective man-machine interface.For battery-driven calculating equipment, allow users to More rapidly and more effectively interaction and initiate real time communication session, save power and increase battery charge twice between Every.
At frame 1302, equipment shows the instant message user interface (for example, 1200) of instant message application program.Immediately Message user interface includes message region (for example, 1202) comprising participant in message conversation (including three or more Participant) between multiple message.Instant message user interface include separated with message region (for example, 1202) show and can indicate (for example, 1206), for starting live video communication session in live video communication application program (for example, message details are shown It can indicate or what the figure with live video communication application program indicated shows and can indicate).In some embodiments, showing can table Show it is that calling is all shown and can be indicated (for example, 1206), will immediately begin to and all participants in instant message application program Real time communication session.In some embodiments, show that can indicate is to participate in for showing group's contacts user interface to invite The group contact person that person carries out real time communication session shows and can indicate (for example, 1208).
At frame 1340, equipment detection corresponds to the input (for example, 1250A) for showing the selection that can indicate (for example, 1206). At frame 1306, in response to detecting and showing the corresponding input of the selection that can indicate, equipment is in live video communication application program The middle process initiated for starting live video communication session, the live video communication session include the participant of message conversation. In some embodiments, start live video communication application program (for example, 1216), and in response to showing the choosing that can be indicated It selects and directly starts new live video communication session.In some embodiments, in response to being shown to the selection that can be indicated is shown Show the menu of instant message application program (for example, 1228), and selects menu option (for example, 1230) to start real-time video Communication application program simultaneously starts new live video communication session.(for example, passing through detection input) when a given condition is fulfilled, from The dynamic process initiated for starting live video communication session in live video communication application program allows user to start in real time Communication session.When one group of condition has been satisfied without further user input, execution optimized operation, which is enhanced, sets Standby operability, and keep user-equipment interface more effective (for example, suitably inputting and reducing by helping user to provide User's mistake when operation equipment/interacted with equipment), this is further through allowing users to faster and more effectively using equipment And reduces electricity usage and extend the battery life of equipment.Further it is provided that additional controls (for example, showing can indicate) are to send out It plays the process for starting real time communication and additional control option (for example, showing can indicate) is provided, without due to additional display Control (for example, group chat UI) keeps UI mixed and disorderly, and which enhance the operability of equipment, and makes the more effective (example of user-equipment interface Such as, by help user provide suitably input and reduce operation equipment/with equipment interact when user's mistake), this is again Electricity usage faster and more effectively can be reduced using equipment by using family and extend the battery life of equipment.
Optionally, at frame 1308, equipment is converted from instant message user interface (for example, 1200) (for example, in response to inspection Measure user input and convert immediately) arrive real time communication application program user interface (for example, 1216).Optionally, in frame At 1310, equipment starts real time communication session.Optionally, at frame 1312, equipment display group's contacts user interface (for example, 1228), this group of contacts user interface are (for example, show when continuing and showing at least part of instant message user interface Interface or the discrete user interface for replacing instant message user interface) it include group contact person for starting real time communication session Showing can indicate (for example, 1230).(for example, passing through detection input) when a given condition is fulfilled, automatically from instant message user circle Face is converted and starting live video communication session in live video communication application program allows user to immediately begin to real time communication Session.It has been satisfied in one group of condition and has enhanced equipment without executing optimized operation when further user input Operability, and keep user-equipment interface more effective (for example, suitably inputting by helping user to provide and reducing operation User's mistake when equipment/interacted with equipment), this is further through allowing users to faster and more effectively subtract using equipment Lack electricity usage and extends the battery life of equipment.In addition, when a given condition is fulfilled, from instant message user interface Converting and starting live video communication session in live video communication application program reduces the input number executed needed for operating Amount.The quantity for reducing input needed for executing operation enhances the operability of equipment, and keeps user-equipment interface more effective (for example, by help user provide suitably input and reduce operation equipment/with equipment interact when user's mistake), this Further through allowing users to faster and more effectively reduce electricity usage using equipment and extend the battery longevity of equipment Life.
In some embodiments, group's contacts user interface further include: multiple participants show and can indicate (for example, 1232) comprising associated with the first participant in participant's (for example, list for the person of participating individually in) of message conversation One participant, which shows, to be indicated (for example, 1232B)).In some embodiments, group's contacts user interface further includes new connection People, which shows, to be indicated (for example, 1234).Optionally, at frame 1314, equipment starts real time communication in real time communication application program Session.Show group's contacts user interface provide additional control option (for example, showing can indicate) and not when needing additional controls Since the control of additional display keeps UI mixed and disorderly, which enhance the operability of equipment, and keep user-equipment interface more efficient (for example, user's mistake when by helping user to provide input appropriate and reduce operation equipment/interact with equipment), this is again logical It crosses and allows users to more rapidly and efficiently use equipment and reduce electricity usage and extend the battery life of equipment.
Optionally, at frame 1316, equipment shows real time communication session subscriber interface (for example, 1216) (for example, for real The interface of Shi Tongxin placeholder) comprising (for example, initially including) placeholder (for example, 1210A of Figure 12 C) is (for example, figure Placeholder (for example, image or avatars), with the first participant in real time communication sessions participant (for example, except electronics is set The participant of participant, the invitation for waiting addition communication session and being sent addition communication session except standby user) phase Associated text placeholder (for example, name or initials)).In some embodiments, real time communication session subscriber interface It further include the second placeholder associated with second participant.In some embodiments, the first placeholder includes for real-time The first prompting of the first participant of communication session participant, which is shown, can indicate (for example, jingle bell button) (for example, Figure 12 C).One In a little embodiments, real time communication session subscriber interface further include the second placeholder associated with second participant (for example, 1210C).In some embodiments, the first placeholder and the second placeholder are arranged according to the first layout.When real time communication is opened Show that real time communication session subscriber interface provides a user the feedback of the current state about real time communication session when the beginning, and to Family provides the visual feedback for indicating executable operation associated with real time communication interface.Improved visual feedback is provided for user The operability of equipment is just enhanced, and keeps user-equipment interface more efficient (for example, appropriate by helping user to provide User's mistake when inputting and reducing operation equipment/interact with equipment), this is further through allowing users to more rapidly and efficiently make Reduce electricity usage with equipment and extends the battery life of equipment.
Optionally, at frame 1318, real time communication session has been added according to determining first participant, equipment is participated in first The expression (for example, 1210A of Figure 12 D) of person replaces the display (for example, 1210A of Figure 12 C) of the first placeholder (for example, occupy-place Symbol can have participant's name).In some embodiments, animation (for example, Fig. 8 T to Fig. 8 W) occurs.In some embodiments In, equipment receives data from server or another equipment to carry out the determination.In some embodiments, equipment carries out this really It is fixed, because it does not show any expression of user.Display using expression the first placeholder of modification of participant allows user Quickly identify that calling has been added in which user, provide a user the feedback of the current state about real time communication session, and to Family provides the visual feedback that instruction will execute certain operations associated with notice and/or real time communication session.It is provided for user Improved visual feedback enhances the operability of equipment, and keeps user-equipment interface more effective (for example, by helping to use Family provides user's mistake when suitably inputting and reduce operation equipment/interact with equipment), this is further through allowing users to Faster and more effectively reduce electricity usage using equipment and extends the battery life of equipment.
Optionally, at frame 1320, equipment is not yet added real time communication session according to determining first participant and reaches in advance really Fixed time quantum (for example, real time communication is not added after the predetermined period), equipment stop (optionally, in frame At 1322) the first placeholder of display, and optionally at frame 1324, equipment, which changes into the arrangement of the second placeholder, to be different from The second layout of first layout (for example, display layout, wherein first participant does not include in this set-up).In some embodiment party In case, equipment can receive data from server or another equipment to carry out the determination.In some embodiments, equipment can carry out The determination, because it does not show any expression of user.When meeting defined condition, stops display placeholder and simultaneously change table The arrangement shown allows user quickly to identify when other users are added or are added without calling, while maximizing shown participant And/or the visibility of placeholder.It has been satisfied in one group of condition without executing optimization when further user input Operation enhances the operability of equipment, and keeps user-equipment interface more effective (for example, suitable by helping user to provide Input and user's mistake when reducing operation equipment/interact with equipment), this is further through allowing users to more rapidly and have Effect ground is reduced electricity usage using equipment and extends the battery life of equipment.In addition, stopping display placeholder and changing The arrangement for becoming display provides a user the feedback of the current state about real time communication session, and it is anti-to provide a user vision Feedback, instruction will execute certain operations associated with notice and/or real time communication session.Improved visual feedback is provided for user The operability of equipment is enhanced, and keeps user-equipment interface more effective (for example, suitable defeated by helping user to provide User's mistake when entering and reducing operation equipment/interact with equipment), this is further through allowing users to faster and more effectively Reduce electricity usage using equipment and extends the battery life of equipment.
Optionally, at frame 1326 and 1328, equipment, which receives to remind corresponding to first, shows the choosing that can indicate (for example, 808) The input (for example, 850B) selected.Show the corresponding input of the selection that can indicate in response to receiving to remind with first, equipment is to first Participant's transmission new notice (for example, calling, the tinkle of bells etc.) different from the previous notification for being sent to first participant (for example, the Two notices invasion property more than the first notice).In response to receiving the prompting with the previous notification for being different from being sent to participant Show the corresponding input of the selection that can be indicated and send and notify to first participant, reduces the number of input needed for executing operation Amount.The quantity (for example, number of options needed for user calling participant) for reducing input needed for executing operation enhances equipment Operability, and keep user-equipment interface more effective (for example, suitably inputting by helping user to provide and reducing behaviour Make user's mistake when equipment/interact with equipment), this further through allow users to faster and more effectively using equipment and Reduce electricity usage and extends the battery life of equipment.In addition, executing operation automatically, do not using further such as Family sends in the case where inputting and notifies different notices from first, enhances the operability of equipment, and make user-equipment circle Face is more efficient (for example, user's mistake when by helping user to provide input appropriate and reduce operation equipment/interact with equipment Accidentally), this reduces electricity usage and extends equipment further through allowing users to more rapidly and efficiently using equipment Battery life.
Optionally, at frame 1330 and 1332, equipment detection shows energy with the group contact person for starting real time communication session Indicate the corresponding input (for example, 1250C) of selection of (for example, 1208).In response to detecting and being used to start real time communication meeting The group contact person of words shows the corresponding input of selection that can be indicated, equipment starts real time communication meeting in real time communication application program Words.(for example, the user interface for being transformed into real time communication application program).In some embodiments, group's contacts user interface List (for example, 1232) including the participant for real time communication session.In some embodiments, the column of participant are provided Table, without providing for being individually added into or starting the option with the video of given participant or audio conferencing and/or show to indicate. In some embodiments, instant message user interface includes for starting real-time video in live video communication application program Two of communication session, which show, to be indicated: cause to show that the first of group's contacts user interface shows and can indicate when selected, and Lead to the user interface for being transformed into real time communication application program and beginning (for example, immediately beginning to) real time communication when selected The second of session, which is shown, to be indicated.Showing in response to selection group contact person can indicate and start in real time communication application program logical in real time Letter session provides additional control option (for example, showing can indicate) and does not make due to adding the control shown when needing additional controls UI is mixed and disorderly, and which enhance the operability of equipment, and keeps user-equipment interface more efficient (for example, by helping user to provide Input appropriate and user's mistake when reducing operation equipment/interact with equipment), this is further through allowing users to more rapidly and height Effect ground is reduced electricity usage using equipment and extends the battery life of equipment.In addition, showing energy in response to group contact person The selection of expression and in real time communication application program start real time communication session reduce execute operation needed for input quantity. The quantity for reducing input needed for executing operation enhances the operability of equipment, and makes the more effective (example of user-equipment interface Such as, by help user provide suitably input and reduce operation equipment/with equipment interact when user's mistake), this is again Electricity usage faster and more effectively can be reduced using equipment by using family and extend the battery life of equipment.
Optionally, at frame 1334,1336,1338 and 1340, equipment is after starting communication: in response to determining the first ginseng Real time communication session is not yet added with person up to the predetermined period, shows reminder option associated with first participant. In some embodiments, the first participant that reminder option is shown in group contacts user interface, which shows, can indicate or attached Closely.Show that reminder option associated with participant allows user quickly to identify and calling is not yet added when a given condition is fulfilled User simultaneously calls those specific users.It has been satisfied in one group of condition optimal without being executed when further user input Change operation and enhance the operability of equipment, and keeps user-equipment interface more effective (for example, by helping user to provide conjunction Suitable input and user's mistake when reducing operation equipment/interact with equipment), this further through allow users to more rapidly and Equipment is efficiently used and reduces electricity usage and extends the battery life of equipment.It is related to first participant showing When the reminder option of connection, equipment receives input corresponding with the selection of reminder option.In response to receiving the choosing with reminder option Corresponding input is selected, equipment, which sends instruction real time communication session, can be used for the notice of first participant (for example, making first participant Equipment output (for example, the sense of hearing, vision, tactile) notice notice).In response to receiving corresponding to associated with participant The input of the selection of reminder option and to first participant send notify, reduce execute operation needed for input quantity.It reduces The quantity (for example, number of options needed for user calling participant) of input needed for executing operation enhances operating for equipment Property, and make user-equipment interface it is more effective (for example, by help user provide suitably input and reduce operation equipment/with User's mistake when equipment interacts), this is further through allowing users to faster and more effectively reduce electricity using equipment Power uses and extends the battery life of equipment.
Optionally, at frame 1342 and 1344, equipment detection shows the choosing that can indicate (for example, 1232B) with first participant Corresponding input (for example, 1250E) is selected, which, which shows, can indicate associated with the first participant of message conversation.In In some embodiments, these participants are the participants of real time communication session (for example, the participant of live communication session can wrap The one or more users different from the user from message conversation are included, and also may not include the participation from message conversation Person).In response to detecting that corresponding to first participant associated with the first participant of message conversation shows the selection that can be indicated Input, equipment show the feature including first participant the person's of participating individually in user interface (for example, 1236) (for example, individually Participant's user interface has the details about individual consumer (for example, user identifier, name, initials etc.； Contact information of customer；Contact the communication pattern of user)).In response to detecting input (for example, on group contact person UI The participant of each person of participating individually in shows and can indicate) show the person's of participating individually in user interface and when needing additional controls not due to The control of additional display makes UI (for example, group contact person UI) in a jumble, and which enhance the operability of equipment, and set user- Standby interface it is more efficient (for example, by help user provide it is appropriate input and reduce operation equipment/interact with equipment when user Mistake), this reduces electricity usage and extends equipment further through allowing users to more rapidly and efficiently using equipment Battery life.
Optionally, at frame 1346 and 1348, equipment detects and is used to select the option of communication pattern (for example, 1238A is extremely The corresponding input of selection 1238C) (for example, the input with the characteristic strength more than intensity threshold).In response to detect with The corresponding input of the selection of communication pattern option, equipment initiate the process for selecting the commutating mode of first participant.One In a little embodiments, initiating the process includes showing that multiple communications options are shown to indicate (for example, mail, which shows, can indicate, call and show Energy, which indicates, instant message shows to indicate).
Optionally, at frame 1350 and 1352, the corresponding input (example of the selection that can indicate is shown in equipment detection with new contact person Such as, 1234).In response to detecting the corresponding input of the selection showing with new contact person and can indicate, equipment is initiated for will newly contact People is added to the process (for example, showing the user interface for selecting new contact person to be added) of real time communication session.It initiates Process for new contact person to be added to real time communication session allows user quickly to identify which user's addition calling, Xiang Yong Family provide about real time communication session current state feedback, and provide a user instruction will execute with notice and/or in real time The visual feedback of the associated certain operations of communication session.Improved visual feedback, which is provided, for user enhances operating for equipment Property, and make user-equipment interface it is more effective (for example, by help user provide suitably input and reduce operation equipment/with User's mistake when equipment interacts), this is further through allowing users to faster and more effectively reduce electricity using equipment Power uses and extends the battery life of equipment.
It may be noted that can also be with class above in relation to the details of process described in method 1300 (for example, Figure 13 A to Figure 13 D) It is suitable for hereafter/method as discussed above like mode.For example, method 1300 optionally include with reference to method 700,900,1100 or The one or more features of various methods described in 1500.
Figure 14 A to Figure 14 I is shown according to some embodiments for especially initiating in fact from real time communication application program When communication session exemplary user interface.User interface in these attached drawings be used to show including in Figure 15 A to Figure 15 C The process described below of process.
Figure 14 A shows display and has shows the user interface 1400 that can be indicated (for example, home for start application program Screen or springboard) equipment 600 (as described above).As shown in Figure 14 A, equipment 600 receives (for example, detection) and answers with real time communication Being shown with program can indicate that the 1402 corresponding user of selection inputs 1450A (for example, tap).
As shown in Figure 14B, 1450A is inputted in response to receiving user, equipment 600 starts live video communication application program And show the live video communication user interface 1404 of live video communication application program.Live video communication user interface includes Participant corresponding with corresponding participant or participant group shows the list 1408 that can indicate 1408A to 1408K.In some embodiment party In case, based on in real time communication application program corresponding participant or participant group (for example, in addition to equipment 600 user it Two or more outer participants) previous (for example, nearest) real time communication session, participant shows can indicate to be included in column In table 1408.In some embodiments, previous real time communication session include by equipment 600 user or another participant at Communication function connection and attempted.Live video communication user interface 1404 includes for participating in for new real time communication session selection The new session of person, which shows, can indicate 1406.
In some embodiments, it is based on another application program (for example, (1408A can be indicated to message application by showing； For example, group chat participant list), (1408C can be indicated to telephony application by showing；For example, the participant of Conference Calling), calendar answers With program, (1408B can be indicated by showing；For example, the attendant of calendar event) or email application (1408D can be indicated by showing； For example, the reception list of Email)) associated participant group, participant, which shows, can indicate to be included in list 1408.Example Such as, participant shows the participation that can indicate that 1408A corresponds in previous or existing group's message conversation (for example, text or instant message) Person group, and it is shown as in list 1408 suggestion (for example, user need not manual creation participant group).In some embodiments In, participant shows the participant group that can indicate to correspond in Conference calling associated with telephony application.Participant Showing can indicate that 1408A to 1408K includes that the instruction of participant or participant group, status indicator, details are shown and can be indicated and can The relevant date of choosing.As shown in Figure 14B, status indicator mark participant or group source (for example, " come from message ", " come from Phone ", " come from calendar ", " coming from mail ") or associated communication communications status (for example, video, audio, unavailable Deng).In some embodiments, the source of participant or group are by figure or text instruction, color associated with specific application Or it is indicated to application relevant logo (for example, phone icon, video camera icon or message bubble icon).
In some embodiments, equipment 600 is visually distinguished and there are the faciation of movable real time communication session passes The participant of connection, which shows, to be indicated.In fig. 14b, participant, which shows, can indicate that 1408E and real time communication session are currently at moving type The faciation of state is associated with.With other participants show can indicate in the non-bold text that shows and contact details icon (for example, It 1408A-1) compares, showing in participant can indicate that movable real time communication session is by participant's name, video camera figure in 1408E Mark the bold text instruction of 1408E-1 and " tap is to be added " message.
In some embodiments, participant associated with previous real time communication session shows that the appearance that can be indicated is based on setting Whether standby user is added corresponding real time communication session.In fig. 14b, participant shows the use that can indicate 1408H Yu equipment 600 The previous real time communication session that family is not added is associated.As a result, participant, which shows, can indicate that 1408H includes italic text, such as with set Other associated participants of previous real time communication session that standby 600 user (for example, 1408F) is added, which show, can indicate non-oblique Body text is compared.
As shown in Figure 14 C, equipment 600 receive (for example, detection) with it is above-mentioned include 15 participants participant group " dream it The participant of team ", which shows, can indicate that the corresponding user of the selection of 1408A inputs 1450B (for example, tap).
1450B is inputted in response to receiving user, equipment 600 is initiated and show corresponding to selected participant to indicate 1408A The participant of (" Dream Team ") starts the process of new live video communication session.In some embodiments, in response to participating in Person shows the selection that can indicate 1408A, directly starts new live video communication session.For example, as shown in fig. 14d, in response to connecing It receives user and inputs 1450B, equipment 600 shows real time communication user interface 1410 and in the feelings of not further user's input The real time communication session with " Dream Team " group is initiated under condition.Real time communication user interface 1410 includes the table of the user of equipment 600 Show (for example, video of the camera from 600 front of equipment) and Call- Control1 menu 1412, similar to the Call- Control1 discussed Menu 630.
In some embodiments, it can indicate 1408A selection in response to showing participant and show live video communication The menu of application program, and menu option is selected to start new live video communication session.For example, in some embodiments In, equipment 600 shows real time communication user interface 1410 (as shown in fig. 14d), without initiating real time communication session.On the contrary, In When showing real time communication user interface 1410, equipment 600 receives (for example, detection) and shows with the calling on Call- Control1 menu 1412 It can indicate 1414 selection corresponding user input (for example, tap), and show in response to real time communication initiation and can indicate 1414 Selection initiate and the real time communication session of " Dream Team " group.
Figure 14 E also shows real time communication user interface 1404.Show and can indicate instead of receiving participant in list 1408 Selection (as shown in Figure 14 C), equipment 600 receive (for example, detection) show with new session can indicate 1406 the corresponding use of selection Family inputs 1450C (for example, tap).1450C is inputted in response to user, and can indicate 1406 pairs according to showing with selection new session The user answered inputs 1450C, and equipment 600 is initiated for selecting participant and between selected participant and the user of equipment 600 Start the process of new live video communication session.As shown in fig. 14f, real time communication user interface 1404 is by user interface 1416 Replacement, the user interface include for using keyboard 1420 to input the text entry field 1418 of participant's name, being used for from dish Addition participant of selection participant, which shows, in single (for example, contacts list), which can indicate 1422, only audio is shown can indicate 1424 and Video, which shows, can indicate 1426.
In Figure 14 F, in response to text " JOE " is inputted text entry field 1418, the display of equipment 600 is used for contact person Two the communications options 1417A and 1417B of " JOE SMITH ".In some embodiments, equipment 600 visually distinguish with The associated communications option of equipment of real time communication session can be added.In Figure 14 F, indicate logical by communications option 1417B Letter mode (for example, Email) is associated with the equipment that real time communication session can be added, as shown in bold text.On the contrary, The communication pattern (for example, home phone number) indicated by communications option 1417A is shown with normal (non-runic) text, to indicate to communicate Mode is not associated with the equipment that real time communication session can be added.
As shown in figure 14g, the participant of entitled Joe, Ashley and Sam are selected.Equipment 600 receives (for example, detection) Showing with video can indicate that the 1426 corresponding user of selection inputs 1450D (for example, tap).
1450D is inputted in response to receiving user, equipment 600 starts the new real time communication session with selected participant, and Real-time media stream (as shown in fig. 14h) including video and audio data is provided.In some embodiments, in response to receiving Show with only audio can indicate 1424 selection corresponding user input, equipment 600 starts the new real time communication with selected participant Session, and providing includes audio data without the real-time media stream of video data.
In some embodiments, after new real time communication session start, equipment 600 is removing real time communication application journey The expression of new real time communication session is shown in application program except sequence.Starting and participant Joe, Ashley and Sam After real time communication session, the user of equipment 600 navigates to instant message application program.As shown in Figure 14 I, starting to lead in real time After believing session, equipment 600 shows the instant message user interface 1428 of instant message application program.Instant message user interface 1428 displays include group's message session of the participant of real time communication session, and show that instruction has been opened in message region 1432 The message 1430 of beginning real time communication session.In some embodiments, after starting real time communication session, equipment 600 is in sight When message application message conversation list in display include real time communication session participant group.In some embodiments In, equipment 600 shows the (institute as above of instant message user interface 1428 in response to the selection to the group in message conversations list It states).
Figure 15 A to Figure 15 C is the flow chart for showing the method according to some embodiments.In the equipment with display Method 1500 is executed at (for example, 100,300,500,600).Some operations in method 1500 are optionally combined, Yi Xiecao The order of work is optionally changed, and some operations are optionally omitted.
As described below, method 1500 is provided for initiating the real time communication session between multiple participants and interacting Intuitive manner.This method is by providing the intelligence initiation to the real time communication session between multiple participants and interacting reduction The cognitive load of user, to create more effective man-machine interface.For battery-driven calculating equipment, allow users to More rapidly and more effectively interaction and initiate real time communication session, save power and increase battery charge twice between Every.
At frame 1502, equipment show live video communication application program live video communication user interface (for example, 1404).At frame 1504, live video communication user interface includes multiple ginsengs corresponding with corresponding one or more participant Showing with person can indicate that (for example, 1408A to 1408K), the multiple participant shows that can indicate includes corresponding to three or more The first participant of the group of participant, which shows, to be indicated (for example, 1408A).At frame 1506, live video communication user interface packet It includes the new session for starting new live video communication session and shows and can indicate (for example, 1406).In some embodiments, join Showing with person can indicate to correspond to (for example, successfully connecting (by participant using equipment or by participant associated with external equipment) Connect or attempt) previously (for example, nearest) video communication and including the participant's (for example, 1408F) to video communication Instruction.In some embodiments, participant, which shows, can indicate to represent with another communication application program (for example, instant message application Program or telephony application (for example, nearest Conference Calling)) associated group.In some embodiments, real time communication User interface includes the list of nearest real time communication session (for example, the list of nearest Video chat can be independent chat And/or group chat).Display includes that the real time communication user interface of nearest real time communication session list allows user quickly to access simultaneously It realizes that user or equipment participate in the nearest real time communication session of (for example, be invited to, be added, therefrom communication etc.), is mentioned to user For the feedback of the current state about real time communication session, and providing a user instruction will execution and notice and/or real time communication The visual feedback of the associated certain operations of session.Improved visual feedback, which is provided, for user enhances the operability of equipment, And make user-equipment interface it is more effective (for example, by help user provide suitably input and reduce operation equipment/with set Standby user's mistake when interacting), this is further through allowing users to faster and more effectively reduce electric power using equipment Using and extend the battery life of equipment.
In some embodiments, multiple participants show can indicate include and suggest participant group (for example, 1408B) (example Such as, the suggestion participant group for not manually selected/being created by user) corresponding second participant shows and can indicate, suggestion participant Group based on from second application program different from live video communication application program (for example, telephony application, instant message Application program) data (for example, using data) determine.
In some embodiments, it is proposed that participant group correspond to the group that carries out in the second application program and talk with (example Such as, group's instant messaging session, group's Conference Calling) in participant (for example, 1408A) or carried out in the second application program Participant (for example, 1408C) in Conference Calling.Display include with suggest participant group it is corresponding show can indicate it is real-time lead to Credit household interface allows user quickly to access and starts and suggest the real time communication of group, and reducing execution and establishing includes selected group Each member the associated operation of real time communication session needed for input quantity.Reduce the number of input needed for executing operation Amount enhances the operability of equipment, and keeps user-equipment interface more effective (for example, suitable by helping user to provide User's mistake when inputting and reducing operation equipment/interact with equipment), this is further through allowing users to more rapidly and effectively Ground is reduced electricity usage using equipment and extends the battery life of equipment.In addition, display is with suggestion, group is corresponding shows It can indicate to provide additional control operations, without keeping UI mixed and disorderly due to the control of additional display.There is provided additional control operations without by So that UI is just enhanced the operability of equipment in a jumble in the control of additional display, and makes the more efficient (example of user-equipment interface User's mistake when such as, by helping user to provide input appropriate and reduce operation equipment/interact with equipment), this further through It allows users to more rapidly and efficiently uses equipment and reduce electricity usage and extend the battery life of equipment.
In some embodiments, second participant, which shows, can indicate to include with the second application program (for example, the suggestion comes Source application) instruction (for example, figure or text instruction (for example, " as proposed by message "), with some application program phase The color of pass or logo relevant to the application program).Display further includes corresponding showing and capable of indicating with the participant group suggested The participant group of real time communication user interface, the suggestion corresponds to the group in the application program different from real time communication application program Participant in dialogue allows user quickly to access and starts and suggest the real time communication of group, and reduces execution with foundation and include Needed for the associated operation (for example, different application program of such as navigating) of the real time communication session of each member of selected group Input quantity.The quantity for reducing input needed for executing operation enhances the operability of equipment, and makes user-equipment interface More effectively (for example, wrong by the user for helping user to provide when suitably inputting and reduce operation equipment/interact with equipment Accidentally), this is further through allowing users to faster and more effectively reduce electricity usage using equipment and extend equipment Battery life.In some embodiments, multiple participants show can indicate include with second suggest participant group (for example, not by The suggestion participant group that user manually selects/creates) corresponding third participant shows and can indicate, the second suggestion participant crowd base In from different from live video communication application program and different with the second application program third application program (for example, electricity Talk about application program, instant message application program, email application, calendar applications) data (for example, using number According to) determine (for example, the group from nearest calendar event or group from nearest mail events).Display include suggest group from The second participant of the instruction of application program shows the feedback that can indicate to provide a user the current state about real time communication session, And provide a user the visual feedback for indicating certain operations associated with the second application program.Improved vision is provided for user Feedback enhances the operability of equipment, and keeps user-equipment interface more effective (for example, suitable by helping user to provide Input and user's mistake when reducing operation equipment/interact with equipment), this is further through allowing users to more rapidly and have Effect ground is reduced electricity usage using equipment and extends the battery life of equipment.In some embodiments, logical in real time Credit household interface further includes that participant's selection is shown and can be indicated (for example, "+").
At frame 1508, equipment detection corresponds to the input (for example, 1450B) of following request, that is, initiates to start new reality When video communication sessions process.At frame 1510,1512 and 1514, equipment is in response to detecting the input: according to first Participant, which shows, can indicate the corresponding input of the selection of (for example, 1408A), initiate the mistake for starting new live video communication session Journey, the new live video communication session include the group of three or more participants.In some embodiments, in response to right First participant shows the selection that can be indicated and directly starts new live video communication session (for example, Figure 14 D).In some implementations In scheme, in response to showing the selection that can be indicated to first participant, show live video communication application program menu (for example, 1416), and menu option is selected to start new live video communication session.In response to detect input, equipment according to The corresponding input of selection that can be indicated is shown in new session, initiates for selecting two or more participants and starting new reality When video communication sessions process, which includes participant associated with the equipment and two Or more selected by participant (for example, Figure 14 F).In some embodiments, user interface is replaced by another user interface, Another user interface include for select the first of participant show can indicate (for example, 1422) and for starting real-time video it is logical The second of letter session (for example, in response to showing that the selection that can be indicated directly starts live video communication session to second) is shown and can indicate (for example, 1424 and 1426).In some embodiments, by the way that contact information is input to the text field (for example, 1418) Or contact person is selected from contacts list to select participant.In some embodiments, in response to that can indicate (example to showing Such as, 1408A) selection directly start new live video communication session.It initiates when a given condition is fulfilled new for starting The process of real time communication allows equipment to provide a user different live video communication sessions based on the quantity of selected participant.In One group of condition, which has been satisfied, enhances operating for equipment without executing optimized operation when further user input Property, and make user-equipment interface it is more effective (for example, by help user provide suitably input and reduce operation equipment/with User's mistake when equipment interacts), this is further through allowing users to faster and more effectively reduce electricity using equipment Power uses and extends the battery life of equipment.In addition, being initiated when a given condition is fulfilled for starting new real time communication Various process reduce execute operation needed for input quantity.The quantity for reducing input needed for executing operation enhances equipment Operability, and keep user-equipment interface more effective (for example, suitably inputting by helping user to provide and reducing behaviour Make user's mistake when equipment/interact with equipment), this further through allow users to faster and more effectively using equipment and Reduce electricity usage and extends the battery life of equipment.
Optionally, at frame 1516,1518,1520,1522,1524 and 1526, equipment, which is received, shows energy with participant's selection Corresponding second input of the selection of expression.Show the corresponding second user of the selection that can indicate with participant's selection in response to receiving Input, equipment show that mass selection selects user interface (for example, 1416) comprising: the text of one or more participants for identification Input field (for example, 1418) (for example, extremely: field)；Keyboard (for example, 1420)；Only audio, which is shown, to indicate (for example, 1424)； And video shows and can indicate (for example, 1426).When showing that mass selection selects user interface, equipment receives third user input selection. In response to receiving third user input, equipment makes according to the selection that can be indicated corresponding third user input is shown with only audio Start new real-time lead to since the audio of electronic equipment (for example, the audio recorded by one or more microphones of equipment) Session is believed, without from electronic equipment video data.When the defined condition of satisfaction is (for example, showing in response to only audio can indicate Selected) when, allow user to be added in fact using audio using new real time communication session since the audio of electronic equipment When communication session, select or modify equipment without user it is additional output and input (for example, close camera and/or Open microphone).It has been satisfied in one group of condition without executing optimized operation enhancing when further user input The operability of equipment, and keep user-equipment interface more effective (for example, by helping user to provide suitable input simultaneously Reduce user's mistake when operation equipment/interact with equipment), this is further through allowing users to faster and more effectively use Equipment and reduce electricity usage and extend the battery life of equipment.In some embodiments, system display indicator, The indicator describes user and real time communication is added using only audio.In some embodiments, device prompts user shows phrase Such as " whether you want connect with video ", to determine that user is intended to that audio or video and audio is used only.This can make It obtains user to feel comfortable, because video is not shared).In response to receiving third user input, shows and can indicate according to video Selection corresponding third user input, the equipment real time communication new come using the audio and video since electronic equipment Session.When condition (for example, showing in response to video can indicate that option is selected) as defined in meet, using from electronic equipment Audio and video start real time communication session allow user using audio and video be added real time communication session, without user The additional of selection or modification equipment outputs and inputs (for example, open camera and/or open microphone).In one group of condition It has been satisfied and has enhanced the operability of equipment without executing optimized operation when further user input, and made User-equipment interface more effectively (suitably inputs and reduces operation equipment/and equipment progress for example, passing through and user being helped to provide User's mistake when interaction), this is further through allowing users to faster and more effectively reduce electricity usage using equipment simultaneously And extend the battery life of equipment.
Optionally, at frame 1528 and 1530, equipment starts new real time communication session.Starting new real time communication meeting After words, equipment display in fourth application program (for example, message or telephony application) different from real time communication session The expression of new real time communication session.In some embodiments, real time communication participant is shown as instant message application journey The group participant in group's message in message conversation list in sequence.In some embodiments, the group in message conversation list Message includes the instruction with the presence or absence of the activity real time communication session including group participant (for example, 1408E).In some embodiment party In case, grouping busy in the nearest call list that real time communication participant is shown as in telephony application or calling appl. Group participant in crying.In some embodiments, real time communication participant is shown as calendar event in calendar applications Invitee.Shown in the application program different from real time communication session the expression of real time communication session provide a user about The feedback of the current state of real time communication session in multiple application programs, in some embodiments, if user is only once Property activation indicate (for example, later since with previous real time communication session have identical participant another application program reality When communication application program in another new real time communication session), then providing a user instruction will execute and notice and/or logical in real time Believe the visual feedback of the associated certain operations of session.Improved visual feedback, which is provided, for user enhances operating for equipment Property, and make user-equipment interface it is more effective (for example, by help user provide suitably input and reduce operation equipment/with User's mistake when equipment interacts), this is further through allowing users to faster and more effectively reduce electricity using equipment Power uses and extends the battery life of equipment.
In some embodiments, multiple participants, which show, can indicate that (for example, 1408) include and three or more participations Corresponding 4th participant of the group of person shows and can indicate (for example, 1408A, 1408E).In some embodiments, according to determining the Four participants, which show, can indicate to correspond to movable real time communication session (for example, 1408E), and the 4th participant, which shows, can indicate with first Visual appearance is shown (for example, First look appearance is that participant shows the font color that can be indicated or font style or to participate in Person show can indicate at or near the icon or tag mark that show)；And it can indicate to correspond to according to determining that the 4th participant shows Inactive real time communication session (for example, 1408A), the 4th participant, which shows, can indicate to be different from the second of First look appearance the view Feel that appearance is shown (for example, the second visual appearance is the font face different from the font color of First look appearance or font style Color or font style or the second visual appearance do not include the icon or tag mark shown near the expression of potential participant Or include the icon or tag mark different from First look appearance).In some embodiments, with previous real time communication session Associated participant shows whether the appearance user based on equipment that can be indicated is added corresponding real time communication session (for example, wrong The participant for the real time communication session crossed, which shows, can indicate that (for example, 1408H) includes the real time communication being added with the user of equipment 600 The participant of session shows font color, font style, icon, the mark that can indicate that (for example, 1408F) is different or is not included therein Will symbol etc.；If the user adding to conversation of equipment, participant, which shows, can indicate the text including a kind of color, if equipment User then includes the text of different colours before the end that session is not added).In some embodiments, when session not (for example, literary from bold text and video camera icon to the non-runic of no video camera icon when again being at active state This), participant associated with activity real time communication session shows the outward appearance change that can be indicated.
In some embodiments, it initiates for selecting the process of two or more participants to include showing potential participation The expression (for example, 1417A, 1417B) of person (for example, contact person in contacts list).In some embodiments, according to true Fixed potential participant is associated with the equipment that real time communication session can be added, and the expression of potential participant is with First look appearance Display is (for example, First look appearance is for the font color or font style of the expression of potential participant or potential participant's Indicate the icon or tag mark nearby shown)；And according to determine participant not with setting for real time communication session can be added It is standby associated, the indicating (for example, the second vision of potential participant is shown to be different from the second visual appearance of First look appearance Appearance is the font color or font style or second vision different from the font color of First look appearance or font style Appearance does not include the icon shown near the expression of potential participant or tag mark or including different from First look appearance Icon or tag mark).
It may be noted that the details of the process above with reference to described in method 1500 (for example, Figure 15 A to Figure 15 C) are also with similar side Formula is suitable for method as discussed above.For example, method 1500 is optionally included above with reference to method 700,900,1100 or 1300 The one or more features of the various methods.
For illustrative purposes, the description of front is described by reference to specific embodiment.However, example above The property shown discussion is not intended to exhausted or limits the invention to disclosed precise forms.According to above teaching content, very More modifications and variations are all possible.It is to best explain these technologies that these embodiments, which are chosen and described, Principle and its practical application.Thus, it is possible to best utilize these technologies and have to be suitble to by others skilled in the art In the various embodiments of the various modifications of desired special-purpose.
Although having carried out comprehensive description to the disclosure and example referring to attached drawing, it should be noted that, it various change and repairs Changing will become obvious for those skilled in the art.It should be appreciated that such change and modification are considered being wrapped It includes in the range of the disclosure and example being defined by the claims.
As described above, the one aspect of this technology is to collect and uses the data for deriving from various sources, to improve more matchmakers Body communication.The disclosure is expected, in some instances, these data collected may include uniquely identify or can be used for connection or Position the personal information data of specific people.Such personal information data may include consensus data, location-based data, Telephone number, e-mail address, Twitter account, home address, data relevant to user health or the general level of the health or note Record (for example, life sign measurement, drug information, exercise information), date of birth or any other identification or personal information.
It is benefited the present disclosure recognize that may be used in family using such personal information data in the technology of the present invention.For example, Personal information data can be used for promoting the multimedia communication between user.Therefore, such personal information data is used to make user Such communication can be improved.In addition, the disclosure is it is also contemplated that personal information data are beneficial to other purposes of user.For example, healthy It can be used for providing opinion to the general health of user with workout data, or can be used as pursuing health objectives using technology Personal positive feedback.
Disclosure imagination is responsible for acquisition, analyzes, open, transmission, stores or other use the reality of such personal information data Body will comply with set privacy policy and/or privacy practice.Specifically, such entity should be carried out and adhere to using being acknowledged as It is met or exceeded by the privacy policy and practice to the privacy of maintenance personal information data and the industry of safety or administration request. User accesses such strategy in which can be convenient, and should update with the acquisition of data and/or using changing.The personal letter of user Breath should acquire the legal and reasonable employment for entity, without that should share or sell except these legal uses.In addition, receiving After user's informed consent, such acquisition/shared should be carried out.In addition, such entity is considered as taking any steps necessary, defendance and It ensures the access to such personal information data, and ensures that the people of other Internet access personal information data abides by its privacy policy And process.In addition, this entity can make itself to be subjected to third party's assessment with prove its abide by the privacy policy accepted extensively and Practice.In addition, policy and practice should be adjusted, so as to the certain types of personal information data for acquiring and/or accessing, and it is suitable for The applicable law specifically considered and standard including compass of competency.For example, the collection or access of certain health datas can in the U.S. It can constraint convenient by federal and/or state law such as health insurance and accountability act (HIPAA)；And the healthy number of other countries According to may be by the constraint of other regulations and policy, therefore it should give respective handling.Therefore, different should be kept in each country The different privacy practices of personal data type.
Regardless of afore-mentioned, the disclosure is it is also contemplated that user selectively prevents to use or access personal information data Embodiment.That is disclosure expection can provide hardware element and/or software element, to prevent or prevent to such personal information number According to access.For example, technology of the invention can be configured to during registration service or thereafter for multimedia communication Any time, allow user that " addition " or " exiting " is selected to participate in the collection to personal information data.For another example, user may be selected not Position data and/or availability are provided.In another example, user, which may be selected to limit, maintains the time of the record of communication session long Degree or total ban retain such data.Other than offer " selection is added " and " selection is exited " option, the disclosure is imagined Notice relevant to accessing or using personal information is provided.For example, user can notify user, a people when downloading application program Information data will be accessed, and then remind user again before personal information data are accessed by application program.
In addition, the purpose of the disclosure is that should manage and handle personal information data to minimize unintentionally or unauthorized access Or the risk used.Once no longer needing data, risk can be minimized by the acquisition of limitation data and deletion data.In addition, simultaneously And as applicable, it is included in certain healthy related applications, data go to identify the privacy that can be used for protecting user.In feelings appropriate It, can be by the amount or feature of the data of removal unique identifier (for example, date of birth etc.), control storage (for example, In under condition City-level rather than address level collect position data), the storage modes (for example, aggregated data between users) of control data and/ Or other methods promote to identify.
Therefore, although the disclosure, which is widely covered using personal information data, realizes that one or more is various disclosed Embodiment, but the disclosure it is also contemplated that various embodiments can also without access such personal information data in the case where quilt It realizes.That is, the various embodiments of the technology of the present invention will not due to lack such personal information data all or part of and It can not be normally carried out.For example, can the personal information based on non-personal information data or absolute bottom line (for example, with user's phase The requested content of associated equipment, other non-personal information available to communications service providers or publicly available information) come Establish communication session.

Claims (66)

1. a kind of electronic equipment, comprising:
Display；
One or more processors；With
Memory, the memory storage are configured as the one or more programs executed by one or more of processors, One or more of programs include instructions for performing the following operations:
Receive request of the display for the user interface of the real time communication session between two or more participants；And
In response to receiving user circle of the display for the real time communication session between two or more participants The request in face shows real time communication user interface, including showing that the participant's in the real time communication session is more simultaneously A expression,
Wherein:
Participant according to the quantity in the determination real time communication session includes the participant greater than number of thresholds, removes institute At least some of the multiple expression of the participant except the user of electronic equipment expression is stated to overlap each other；And
Participant according to the quantity in the determination real time communication session includes being less than or equal to the number of thresholds The multiple expression of participant, the participant in addition to the user of the electronic equipment all do not overlap one another.
2. electronic equipment according to claim 1, one or more of programs further include for performing the following operations Instruction:
After showing the real time communication session, the opposite activity level of the participant in the real time communication session is detected Variation；And
In response to detecting the variation of the opposite activity level of the participant in the real time communication session:
Participant according to the quantity in the determination real time communication session includes the participant greater than number of thresholds, adjustment The size of multiple expressions of participant relative to each other, while keeping multiple participants in addition to the user of the electronic equipment The multiple expression between at least some overlappings；And
Participant according to the quantity in the determination real time communication session includes being less than or equal to the number of thresholds Participant keeps the size of the multiple expression of participant relative to each other.
3. electronic equipment according to claim 1, wherein showing that the participant's in the real time communication session is described more A expression further include:
Participant according to the quantity in the determination real time communication session includes the participant greater than second threshold quantity, Show that more than first in addition to the user of the electronic equipment participate in be greater than one or more sizes of respective threshold size The expression of person, while the expression of more than second a participants is shown in an overflow region, wherein a participant includes more than described second One for being not included in one or more participants more than described first in a participant, and being shown in the overflow area Or the expression of other multiple participants is shown with the size for being less than the respective threshold size；With
Participant according to the quantity in the determination real time communication session includes being less than or equal to second threshold quantity Participant, be greater than one or more sizes of respective threshold size show in addition to the user of the electronic equipment described the The expression of a participant more than one is shown without one or more sizes to be less than or equal to the respective threshold size except described The expression of participant except the user of electronic equipment.
4. electronic equipment according to claim 3, wherein more than described first in addition to the user of the electronic equipment At least some of expression of participant expression overlaps each other, and wherein more than described second in the overflow area participation The expression of person does not all overlap one another.
5. electronic equipment according to claim 3, one or more of programs further include for performing the following operations Instruction:
The display indicated of a participant more than described first is updated with first rate；And
The display indicated in the overflow area of a participant more than described second is updated with the second rate, wherein described Second rate is lower than the first rate.
6. electronic equipment according to claim 3, one or more of programs further include for performing the following operations Instruction:
Detection corresponds to the input of the overflow area；And
Correspond to the input of the overflow area in response to detecting:
The overflow area is rolled to stop showing the subset indicated of a participant more than described second；And
While rolling the overflow area, maintain the expression of the view of the camera of the electronic equipment in the spilling Display in region.
7. electronic equipment according to claim 1, wherein showing that the participant's in the real time communication session is described more A expression further include:
Participant according to the quantity in the determination real time communication session includes the participant less than second threshold quantity, At least one of indicating for the participant in addition to the user of the electronic equipment is shown with the first aspect ratio；With And
Participant according to the quantity in the determination real time communication session includes the ginseng greater than the second threshold quantity With person, be different from first aspect ratio the second aspect ratio show in addition to the user of the electronic equipment described in At least one of expression of participant.
8. electronic equipment according to claim 1, wherein showing that the participant's in the real time communication session is described more A expression further include:
Participant according to the quantity in the determination real time communication session includes the participant less than third number of thresholds, Indicating for the view of the camera of the electronic equipment is shown with third aspect ratio；And
It include being greater than or equal to the third number of threshold values according to the participant of the quantity in the determination real time communication session The participant of amount is shown described in the camera of the electronic equipment with being different from the 4th aspect ratio of the third aspect ratio The expression of view.
9. electronic equipment according to claim 3, one or more of programs further include for performing the following operations Instruction:
While showing the expression of the participant in addition to the user of the electronic equipment, receives and indicate the reality When communication session in the participant quantity increased data；And
The increased data of quantity of the participant in the real time communication session are indicated in response to receiving:
Reduce one or more sizes indicated in the expression of the participant in addition to the user of the electronic equipment；With And
Show that the one or more of the new participant in the real time communication session indicate.
10. electronic equipment according to claim 1, wherein showing that the participant's in the real time communication session is described more A expression further include:
It is the participant of the first quantity according to the participant of the quantity in the real time communication session, with the first layout display The expression of the participant；And
It is the participant of the second quantity according to the participant of the quantity in the real time communication session, to be different from described the Second layout of two layouts shows the expression of the participant.
11. electronic equipment according to claim 3, wherein real time communication session subscriber interface further includes and invited It please arrive the real time communication session but the associated placeholder of first participant of the real time communication session is not yet added.
12. electronic equipment according to claim 11, wherein the placeholder includes for the first participant One prompting shows and can indicate that one or more of programs further include instructions for performing the following operations:
It receives and shows the corresponding input of the selection that can indicate with first prompting；And
Show the corresponding input of the selection that can indicate in response to receiving to remind with described first, Xiang Suoshu first participant sends out Send the new notice different from the previous notification for being sent to the first participant.
13. electronic equipment according to claim 11, wherein the expression of the placeholder and more than first a participant With the first layout placement, one or more of programs further include instructions for performing the following operations:
The real time communication session is not yet added up to predetermined time quantum according to the determination first participant:
Stop showing the placeholder；And
The arrangement of the expression of more than described first a participants is changed into the second layout different from first layout, packet Include one or more expressions in amplification and/or the expression of mobile participant.
14. electronic equipment according to claim 11, one or more of programs further include for performing the following operations Instruction:
The real time communication session has been added according to the determination first participant, has indicated modification institute with the first participant State the display of placeholder.
15. electronic equipment according to claim 3, one or more of programs further include for performing the following operations Instruction:
Detection with described second in the overflow area more than a participant expression in one indicate selection it is corresponding Input；And
In response to detecting the choosing with an expression in the expression of more than described second a participants in the overflow area Select the corresponding input:
Indicate that corresponding participant's indicates to replace a participant more than described first with selected in the overflow area Expression in one expression display.
16. electronic equipment according to claim 3, one or more of programs further include for performing the following operations Instruction:
When showing the real time communication user interface, detecting the second participant in more than second a participant becomes current Active participant；And
In response to detecting that the second participant more than described second in a participant becomes the participant of the current active And according to the expression of the second participant in a participant of determination more than described second currently not in the overflow area Display rolls the overflow area to show second participation more than described second in a participant in the overflow area The expression of person.
17. electronic equipment according to claim 3, wherein the overflow area further include be not currently connected to it is described in real time The placeholder of the invitee of the real time communication session of communication session indicates.
18. electronic equipment according to claim 1, one or more of programs further include for performing the following operations Instruction:
A part of the multiple expression as the participant in the real time communication session shows the electricity with first size The expression of the view of the camera of sub- equipment；
When showing the multiple expression of the participant in the real time communication session, receives and be used to show the first menu Select corresponding first input；
In response to receiving the input corresponding with for showing the selection of first menu, in the real time communication user First menu is shown in interface, wherein first menu includes for the participant in the real time communication session The option of visual effect is shown in expression；And
It is aobvious with the second size in response to receiving the second input corresponding with for showing the selection of the option of visual effect Show the expression of the view of the camera of the electronic equipment, wherein second size is greater than first ruler It is very little.
19. electronic equipment according to claim 1, one or more of programs further include for performing the following operations Instruction:
Receive the choosing indicated with first in the multiple expression for amplifying participant described in the real time communication session Select corresponding third input；
Described in the multiple expression in response to receiving and for amplifying participant described in the real time communication session The first corresponding third input of selection indicated, amplifying described first indicates；And
Reduce and is different from first expression in the multiple expression of participant described in the real time communication session at least The size of one expression.
20. electronic equipment according to claim 1, one or more of programs further include for performing the following operations Instruction:
The first data that orientation in response to receiving the instruction electronic equipment changes, indicate septum reset based on described first The first display indicated in the multiple expression of the position to change participant described in the real time communication session.
21. electronic equipment according to claim 3, one or more of programs further include for performing the following operations Instruction:
When showing the multiple expression of the participant in the real time communication session, receive corresponding with the display request of menu The 4th input；And
In response to receiving the 4th input:
Menu is shown in the user interface of the real time communication session；
At least third in the multiple expression of participant in the mobile real time communication session indicates；And
Change at least the 4th size indicated in the multiple expression of the participant in the real time communication session.
22. electronic equipment according to claim 3, one or more of programs further include for performing the following operations Instruction:
When showing the multiple expression of the participant in the real time communication session, the request with the first menu of display is received Corresponding input；
In response to receiving the input corresponding with the request of first menu is shown, in the real time communication user First menu is shown in interface, wherein first menu includes the option for selecting content item；
Receive input corresponding with the selection of content item；And
In response to receiving the input corresponding with the selection of the content item, it is greater than the respective threshold size with size The participant's that the expression replacement of the content item is shown with the size greater than the respective threshold size indicates.
23. a kind of non-transient computer readable storage medium, the non-transient computer readable storage medium storage is configured as The one or more programs that will be executed by the one or more processors of the electronic equipment with display, it is one or more of Program includes instructions for performing the following operations:
Receive request of the display for the user interface of the real time communication session between two or more participants；And
In response to receiving user circle of the display for the real time communication session between two or more participants The request in face shows real time communication user interface, including showing that the participant's in the real time communication session is more simultaneously A expression, in which:
Participant according to the quantity in the determination real time communication session includes the participant greater than number of thresholds, removes institute At least some of the multiple expression of the participant except the user of electronic equipment expression is stated to overlap each other；And
Participant according to the quantity in the determination real time communication session includes being less than or equal to the number of thresholds The multiple expression of participant, the participant in addition to the user of the electronic equipment all do not overlap one another.
24. non-transient computer readable storage medium according to claim 23, one or more of programs further include Instructions for performing the following operations:
After showing the real time communication session, the opposite activity level of the participant in the real time communication session is detected Variation；And
In response to detecting the variation of the opposite activity level of the participant in the real time communication session:
Participant according to the quantity in the determination real time communication session includes the participant greater than number of thresholds, adjustment The size of multiple expressions of participant relative to each other, while keeping multiple participants in addition to the user of the electronic equipment The multiple expression between at least some overlappings；And
Participant according to the quantity in the determination real time communication session includes being less than or equal to the number of thresholds Participant keeps the size of the multiple expression of participant relative to each other.
25. non-transient computer readable storage medium according to claim 23, wherein showing the real time communication session In participant the multiple expression further include:
Participant according to the quantity in the determination real time communication session includes the participant greater than second threshold quantity, Show that more than first in addition to the user of the electronic equipment participate in be greater than one or more sizes of respective threshold size The expression of person, while the expression of more than second a participants is shown in an overflow region, wherein a participant includes more than described second One for being not included in one or more participants more than described first in a participant, and being shown in the overflow area Or the expression of other multiple participants is shown with the size for being less than the respective threshold size；With
Participant according to the quantity in the determination real time communication session includes being less than or equal to second threshold quantity Participant, be greater than one or more sizes of respective threshold size show in addition to the user of the electronic equipment described the The expression of a participant more than one is shown without one or more sizes to be less than or equal to the respective threshold size except described The expression of participant except the user of electronic equipment.
26. non-transient computer readable storage medium according to claim 25, wherein removing the user of the electronic equipment Except at least some of the expression of more than described first a participants indicate to overlap each other, and wherein in the overflow area The expression of more than described second a participants all do not overlap one another.
27. non-transient computer readable storage medium according to claim 25, one or more of programs further include Instructions for performing the following operations:
The display indicated of a participant more than described first is updated with first rate；And
The display indicated in the overflow area of a participant more than described second is updated with the second rate, wherein described Second rate is lower than the first rate.
28. non-transient computer readable storage medium according to claim 25, one or more of programs further include Instructions for performing the following operations:
Detection corresponds to the input of the overflow area；And
Correspond to the input of the overflow area in response to detecting:
The overflow area is rolled to stop showing the subset indicated of a participant more than described second；And
While rolling the overflow area, maintain the expression of the view of the camera of the electronic equipment in the spilling Display in region.
29. non-transient computer readable storage medium according to claim 23, wherein showing the real time communication session In participant the multiple expression further include:
Participant according to the quantity in the determination real time communication session includes the participant less than second threshold quantity, At least one of indicating for the participant in addition to the user of the electronic equipment is shown with the first aspect ratio；With And
Participant according to the quantity in the determination real time communication session includes the ginseng greater than the second threshold quantity With person, be different from first aspect ratio the second aspect ratio show in addition to the user of the electronic equipment described in At least one of expression of participant.
30. non-transient computer readable storage medium according to claim 23, wherein showing the real time communication session In participant the multiple expression further include:
Participant according to the quantity in the determination real time communication session includes the participant less than third number of thresholds, Indicating for the view of the camera of the electronic equipment is shown with third aspect ratio；And
It include being greater than or equal to the third number of threshold values according to the participant of the quantity in the determination real time communication session The participant of amount is shown described in the camera of the electronic equipment with being different from the 4th aspect ratio of the third aspect ratio The expression of view.
31. non-transient computer readable storage medium according to claim 25, one or more of programs further include Instructions for performing the following operations:
While showing the expression of the participant in addition to the user of the electronic equipment, receives and indicate the reality When communication session in the participant quantity increased data；And
The increased data of quantity of the participant in the real time communication session are indicated in response to receiving:
Reduce one or more sizes indicated in the expression of the participant in addition to the user of the electronic equipment；With And
Show that the one or more of the new participant in the real time communication session indicate.
32. non-transient computer readable storage medium according to claim 23, wherein showing the real time communication session In participant the multiple expression further include:
It is the participant of the first quantity according to the participant of the quantity in the real time communication session, with the first layout display The expression of the participant；And
It is the participant of the second quantity according to the participant of the quantity in the real time communication session, to be different from described the Second layout of two layouts shows the expression of the participant.
33. non-transient computer readable storage medium according to claim 25, wherein the real time communication session subscriber Interface further includes and has been invited into the real time communication session but the first participant of the real time communication session is not yet added Associated placeholder.
34. non-transient computer readable storage medium according to claim 33, wherein the placeholder includes being used for institute It states the first of first participant and reminds and show and can indicate, one or more of programs further include finger for performing the following operations It enables:
It receives and shows the corresponding input of the selection that can indicate with first prompting；And
Show the corresponding input of the selection that can indicate in response to receiving to remind with described first, Xiang Suoshu first participant sends out Send the new notice different from the previous notification for being sent to the first participant.
35. non-transient computer readable storage medium according to claim 33, wherein the placeholder and described first With the first layout placement, one or more of programs further include finger for performing the following operations for the expression of multiple participants It enables:
The real time communication session is not yet added up to predetermined time quantum according to the determination first participant:
Stop showing the placeholder；And
The arrangement of the expression of more than described first a participants is changed into the second layout different from first layout, packet Include one or more expressions in amplification and/or the expression of mobile participant.
36. non-transient computer readable storage medium according to claim 33, one or more of programs further include Instructions for performing the following operations:
The real time communication session has been added according to the determination first participant, has indicated modification institute with the first participant State the display of placeholder.
37. non-transient computer readable storage medium according to claim 25, one or more of programs further include Instructions for performing the following operations:
Detection with described second in the overflow area more than a participant expression in one indicate selection it is corresponding Input；And
In response to detecting the choosing with an expression in the expression of more than described second a participants in the overflow area Select the corresponding input:
Indicate that corresponding participant's indicates to replace a participant more than described first with selected in the overflow area Expression in one expression display.
38. non-transient computer readable storage medium according to claim 25, one or more of programs further include Instructions for performing the following operations:
When showing the real time communication user interface, detecting the second participant in more than second a participant becomes current Active participant；And
In response to detecting that the second participant more than described second in a participant becomes the participant of the current active And according to the expression of the second participant in a participant of determination more than described second currently not in the overflow area Display rolls the overflow area to show second participation more than described second in a participant in the overflow area The expression of person.
39. non-transient computer readable storage medium according to claim 25, wherein the overflow area further includes working as Before be not attached to the real time communication session the real time communication session invitee placeholder indicate.
40. non-transient computer readable storage medium according to claim 23, one or more of programs further include Instructions for performing the following operations:
A part of the multiple expression as the participant in the real time communication session shows the electricity with first size The expression of the view of the camera of sub- equipment；
When showing the multiple expression of the participant in the real time communication session, receives and be used to show the first menu Select corresponding first input；
In response to receiving the input corresponding with for showing the selection of first menu, in the real time communication user First menu is shown in interface, wherein first menu includes for the participant in the real time communication session The option of visual effect is shown in expression；And
It is aobvious with the second size in response to receiving the second input corresponding with for showing the selection of the option of visual effect Show the expression of the view of the camera of the electronic equipment, wherein second size is greater than first ruler It is very little.
41. non-transient computer readable storage medium according to claim 23, one or more of programs further include Instructions for performing the following operations:
Receive the choosing indicated with first in the multiple expression for amplifying participant described in the real time communication session Select corresponding third input；
Described in the multiple expression in response to receiving and for amplifying participant described in the real time communication session The first corresponding third input of selection indicated, amplifying described first indicates；And
Reduce and is different from first expression in the multiple expression of participant described in the real time communication session at least The size of one expression.
42. non-transient computer readable storage medium according to claim 23, one or more of programs further include Instructions for performing the following operations:
The first data that orientation in response to receiving the instruction electronic equipment changes, indicate septum reset based on described first The first display indicated in the multiple expression of the position to change participant described in the real time communication session.
43. non-transient computer readable storage medium according to claim 25, one or more of programs further include Instructions for performing the following operations:
When showing the multiple expression of the participant in the real time communication session, receive corresponding with the display request of menu The 4th input；And
In response to receiving the 4th input:
Menu is shown in the user interface of the real time communication session；
At least third in the multiple expression of participant in the mobile real time communication session indicates；And
Change at least the 4th size indicated in the multiple expression of the participant in the real time communication session.
44. non-transient computer readable storage medium according to claim 25, one or more of programs further include Instructions for performing the following operations:
When showing the multiple expression of the participant in the real time communication session, the request with the first menu of display is received Corresponding input；
In response to receiving the input corresponding with the request of first menu is shown, in the real time communication user First menu is shown in interface, wherein first menu includes the option for selecting content item；
Receive input corresponding with the selection of content item；And
In response to receiving the input corresponding with the selection of the content item, it is greater than the respective threshold size with size The participant's that the expression replacement of the content item is shown with the size greater than the respective threshold size indicates.
45. a kind of method, comprising:
At the electronic equipment with display:
Receive request of the display for the user interface of the real time communication session between two or more participants；And
In response to receiving user circle of the display for the real time communication session between two or more participants The request in face shows real time communication user interface, including showing that the participant's in the real time communication session is more simultaneously A expression,
Wherein:
Participant according to the quantity in the determination real time communication session includes the participant greater than number of thresholds, removes institute At least some of the multiple expression of the participant except the user of electronic equipment expression is stated to overlap each other；And
Participant according to the quantity in the determination real time communication session includes being less than or equal to the number of thresholds The multiple expression of participant, the participant in addition to the user of the electronic equipment all do not overlap one another.
46. according to the method for claim 45, which comprises
After showing the real time communication session, the opposite activity level of the participant in the real time communication session is detected Variation；And
In response to detecting the variation of the opposite activity level of the participant in the real time communication session:
Participant according to the quantity in the determination real time communication session includes the participant greater than number of thresholds, adjustment The size of multiple expressions of participant relative to each other, while keeping multiple participants in addition to the user of the electronic equipment The multiple expression between at least some overlappings；And
Participant according to the quantity in the determination real time communication session includes being less than or equal to the number of thresholds Participant keeps the size of the multiple expression of participant relative to each other.
47. according to the method for claim 45, wherein showing that the participant's in the real time communication session is the multiple It indicates further include:
Participant according to the quantity in the determination real time communication session includes the participant greater than second threshold quantity, Show that more than first in addition to the user of the electronic equipment participate in be greater than one or more sizes of respective threshold size The expression of person, while the expression of more than second a participants is shown in an overflow region, wherein a participant includes more than described second One for being not included in one or more participants more than described first in a participant, and being shown in the overflow area Or the expression of other multiple participants is shown with the size for being less than the respective threshold size；With
Participant according to the quantity in the determination real time communication session includes being less than or equal to second threshold quantity Participant, be greater than one or more sizes of respective threshold size show in addition to the user of the electronic equipment described the The expression of a participant more than one is shown without one or more sizes to be less than or equal to the respective threshold size except described The expression of participant except the user of electronic equipment.
48. according to the method for claim 47, wherein more than described first in addition to the user of the electronic equipment are joined It overlaps each other at least some of the expression of person expression, and wherein a participant more than described second in the overflow area Expression all do not overlap one another.
49. according to the method for claim 47, which comprises
The display indicated of a participant more than described first is updated with first rate；And
The display indicated in the overflow area of a participant more than described second is updated with the second rate, wherein described Second rate is lower than the first rate.
50. according to the method for claim 47, which comprises
Detection corresponds to the input of the overflow area；And
Correspond to the input of the overflow area in response to detecting:
The overflow area is rolled to stop showing the subset indicated of a participant more than described second；And
While rolling the overflow area, maintain the expression of the view of the camera of the electronic equipment in the spilling Display in region.
51. according to the method for claim 45, wherein showing that the participant's in the real time communication session is the multiple It indicates further include:
Participant according to the quantity in the determination real time communication session includes the participant less than second threshold quantity, At least one of indicating for the participant in addition to the user of the electronic equipment is shown with the first aspect ratio；With And
Participant according to the quantity in the determination real time communication session includes the ginseng greater than the second threshold quantity With person, be different from first aspect ratio the second aspect ratio show in addition to the user of the electronic equipment described in At least one of expression of participant.
52. according to the method for claim 45, wherein showing that the participant's in the real time communication session is the multiple It indicates further include:
Participant according to the quantity in the determination real time communication session includes the participant less than third number of thresholds, Indicating for the view of the camera of the electronic equipment is shown with third aspect ratio；And
It include being greater than or equal to the third number of threshold values according to the participant of the quantity in the determination real time communication session The participant of amount is shown described in the camera of the electronic equipment with being different from the 4th aspect ratio of the third aspect ratio The expression of view.
53. according to the method for claim 47, which comprises
While showing the expression of the participant in addition to the user of the electronic equipment, receives and indicate the reality When communication session in the participant quantity increased data；And
The increased data of quantity of the participant in the real time communication session are indicated in response to receiving:
Reduce one or more sizes indicated in the expression of the participant in addition to the user of the electronic equipment；With And
Show that the one or more of the new participant in the real time communication session indicate.
54. according to the method for claim 45, wherein showing that the participant's in the real time communication session is the multiple It indicates further include:
It is the participant of the first quantity according to the participant of the quantity in the real time communication session, with the first layout display The expression of the participant；And
It is the participant of the second quantity according to the participant of the quantity in the real time communication session, to be different from described the Second layout of two layouts shows the expression of the participant.
55. according to the method for claim 47, wherein real time communication session subscriber interface further includes and has been invited to The associated placeholder of first participant of the real time communication session to the real time communication session but is not yet added.
56. method according to claim 55, wherein the placeholder includes mentioning for the first of the first participant It wakes up and shows and can indicate, which comprises
It receives and shows the corresponding input of the selection that can indicate with first prompting；And
Show the corresponding input of the selection that can indicate in response to receiving to remind with described first, Xiang Suoshu first participant sends out Send the new notice different from the previous notification for being sent to the first participant.
57. method according to claim 55, wherein the expression of the placeholder and more than first a participant are with One layout placement, which comprises
The real time communication session is not yet added up to predetermined time quantum according to the determination first participant:
Stop showing the placeholder；And
The arrangement of the expression of more than described first a participants is changed into the second layout different from first layout, packet Include one or more expressions in amplification and/or the expression of mobile participant.
58. method according to claim 55, which comprises
The real time communication session has been added according to the determination first participant, has indicated modification institute with the first participant State the display of placeholder.
59. according to the method for claim 47, which comprises
Detection with described second in the overflow area more than a participant expression in one indicate selection it is corresponding Input；And
In response to detecting the choosing with an expression in the expression of more than described second a participants in the overflow area Select the corresponding input:
Indicate that corresponding participant's indicates to replace a participant more than described first with selected in the overflow area Expression in one expression display.
60. according to the method for claim 47, which comprises
When showing the real time communication user interface, detecting the second participant in more than second a participant becomes current Active participant；And
In response to detecting that the second participant more than described second in a participant becomes the participant of the current active And according to the expression of the second participant in a participant of determination more than described second currently not in the overflow area Display rolls the overflow area to show second participation more than described second in a participant in the overflow area The expression of person.
61. according to the method for claim 47, wherein the overflow area further include be not currently connected to it is described logical in real time Believe that the placeholder of the invitee of the real time communication session of session indicates.
62. according to the method for claim 45, which comprises
A part of the multiple expression as the participant in the real time communication session shows the electricity with first size The expression of the view of the camera of sub- equipment；
When showing the multiple expression of the participant in the real time communication session, receives and be used to show the first menu Select corresponding first input；
In response to receiving the input corresponding with for showing the selection of first menu, in the real time communication user First menu is shown in interface, wherein first menu includes for the participant in the real time communication session The option of visual effect is shown in expression；And
It is aobvious with the second size in response to receiving the second input corresponding with for showing the selection of the option of visual effect Show the expression of the view of the camera of the electronic equipment, wherein second size is greater than first ruler It is very little.
63. according to the method for claim 45, which comprises
Receive the choosing indicated with first in the multiple expression for amplifying participant described in the real time communication session Select corresponding third input；
Described in the multiple expression in response to receiving and for amplifying participant described in the real time communication session The first corresponding third input of selection indicated, amplifying described first indicates；And
Reduce and is different from first expression in the multiple expression of participant described in the real time communication session at least The size of one expression.
64. according to the method for claim 45, which comprises
The first data that orientation in response to receiving the instruction electronic equipment changes, indicate septum reset based on described first The first display indicated in the multiple expression of the position to change participant described in the real time communication session.
65. according to the method for claim 47, which comprises
When showing the multiple expression of the participant in the real time communication session, receive corresponding with the display request of menu The 4th input；And
In response to receiving the 4th input:
Menu is shown in the user interface of the real time communication session；
At least third in the multiple expression of participant in the mobile real time communication session indicates；And
Change at least the 4th size indicated in the multiple expression of the participant in the real time communication session.
66. according to the method for claim 47, which comprises
When showing the multiple expression of the participant in the real time communication session, the request with the first menu of display is received Corresponding input；
In response to receiving the input corresponding with the request of first menu is shown, in the real time communication user First menu is shown in interface, wherein first menu includes the option for selecting content item；
Receive input corresponding with the selection of content item；And
In response to receiving the input corresponding with the selection of the content item, it is greater than the respective threshold size with size The participant's that the expression replacement of the content item is shown with the size greater than the respective threshold size indicates.
CN201910704856.XA 2018-05-07 2018-09-28 Multi-player real time communication user interface Pending CN110457104A (en)

Applications Claiming Priority (11)

Application Number Priority Date Filing Date Title
US201862668229P 2018-05-07 2018-05-07
US62/668,229 2018-05-07
US201862679925P 2018-06-03 2018-06-03
US62/679,925 2018-06-03
DKPA201870364 2018-06-12
DKPA201870362A DK180130B1 (en) 2018-05-07 2018-06-12 Multi-participant live communication user interface
DKPA201870363 2018-06-12
DKPA201870364A DK201870364A1 (en) 2018-05-07 2018-06-12 Multi-participant live communication user interface
DKPA201870362 2018-06-12
DKPA201870363 2018-06-12
CN201811136766.7A CN110457091A (en) 2018-05-07 2018-09-28 Multi-player real time communication user interface

Related Parent Applications (1)

Application Number Title Priority Date Filing Date
CN201811136766.7A Division CN110457091A (en) 2018-05-07 2018-09-28 Multi-player real time communication user interface

Publications (1)

Publication Number Publication Date
CN110457104A true CN110457104A (en) 2019-11-15

Family

ID=68466048

Family Applications (6)

Application Number Title Priority Date Filing Date
CN201910400180.5A Active CN110457096B (en) 2018-05-07 2018-09-28 Multi-participant real-time communication user interface
CN201811136766.7A Pending CN110457091A (en) 2018-05-07 2018-09-28 Multi-player real time communication user interface
CN202011243876.0A Active CN112214275B (en) 2018-05-07 2018-09-28 Multi-participant real-time communication user interface
CN201910704856.XA Pending CN110457104A (en) 2018-05-07 2018-09-28 Multi-player real time communication user interface
CN202010126661.4A Pending CN111367603A (en) 2018-05-07 2018-09-28 Multi-participant real-time communication user interface
CN201910400179.2A Active CN110457095B (en) 2018-05-07 2018-09-28 Multi-participant real-time communication user interface

Family Applications Before (3)

Application Number Title Priority Date Filing Date
CN201910400180.5A Active CN110457096B (en) 2018-05-07 2018-09-28 Multi-participant real-time communication user interface
CN201811136766.7A Pending CN110457091A (en) 2018-05-07 2018-09-28 Multi-player real time communication user interface
CN202011243876.0A Active CN112214275B (en) 2018-05-07 2018-09-28 Multi-participant real-time communication user interface

Family Applications After (2)

Application Number Title Priority Date Filing Date
CN202010126661.4A Pending CN111367603A (en) 2018-05-07 2018-09-28 Multi-participant real-time communication user interface
CN201910400179.2A Active CN110457095B (en) 2018-05-07 2018-09-28 Multi-participant real-time communication user interface

Country Status (3)

Country Link
JP (4) JP6609361B2 (en)
CN (6) CN110457096B (en)
WO (1) WO2019217477A1 (en)

Families Citing this family (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR102373510B1 (en) * 2017-08-11 2022-03-11 삼성전자주식회사 Display device for visualizing contents as the display is rotated and control method thereof
JP6946499B2 (en) * 2020-03-06 2021-10-06 株式会社日立製作所 Speech support device, speech support method, and speech support program
US11580984B2 (en) * 2020-03-20 2023-02-14 At&T Intellectual Property I, L.P. Virtual assistant-initiated conversations
WO2022035931A1 (en) * 2020-08-12 2022-02-17 DTEN, Inc. Mode control and content sharing
WO2022091298A1 (en) * 2020-10-29 2022-05-05 株式会社パルケ Online conference assistance device, online conference assistance program, and online conference assistance system
JP7043110B1 (en) * 2020-10-29 2022-03-29 株式会社パルケ Online conferencing support equipment, online conferencing support programs, and online conferencing support systems
CN112288996B (en) * 2020-11-13 2022-08-09 中铁广州工程局集团深圳工程有限公司 High-voltage offline early warning protection method and system, computer equipment and storage medium thereof
WO2022118746A1 (en) * 2020-12-04 2022-06-09 ソニーグループ株式会社 Information processing apparatus, information processing method, program, and information processing system
US11909921B1 (en) * 2020-12-21 2024-02-20 Meta Platforms, Inc. Persistent digital video streaming
US11943072B2 (en) * 2021-03-30 2024-03-26 Snap Inc. Providing a room preview within a virtual conferencing system
CN117321575A (en) * 2021-05-17 2023-12-29 苹果公司 Apparatus, method and graphical user interface for automatically providing shared content to an application

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103384235A (en) * 2012-05-04 2013-11-06 腾讯科技（深圳）有限公司 Method, server and system used for data presentation during conversation of multiple persons
CN103649985A (en) * 2011-05-26 2014-03-19 谷歌公司 Providing contextual information and enabling group communication for participants in a conversation
CN104010158A (en) * 2014-03-11 2014-08-27 宇龙计算机通信科技(深圳)有限公司 Mobile terminal and implementation method of multi-party video call
CN105094957A (en) * 2015-06-10 2015-11-25 小米科技有限责任公司 Video conversation window control method and apparatus
US20160014059A1 (en) * 2015-09-30 2016-01-14 Yogesh Chunilal Rathod Presenting one or more types of interface(s) or media to calling and/or called user while acceptance of call
US20170324784A1 (en) * 2016-05-06 2017-11-09 Facebook, Inc. Instantaneous Call Sessions over a Communications Application

Family Cites Families (39)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US3859005A (en) 1973-08-13 1975-01-07 Albert L Huebner Erosion reduction in wet turbines
US4826405A (en) 1985-10-15 1989-05-02 Aeroquip Corporation Fan blade fabrication system
US7185054B1 (en) * 1993-10-01 2007-02-27 Collaboration Properties, Inc. Participant display and selection in video conference calls
WO1999038149A1 (en) 1998-01-26 1999-07-29 Wayne Westerman Method and apparatus for integrating manual input
US7218226B2 (en) 2004-03-01 2007-05-15 Apple Inc. Acceleration-based theft detection system for portable electronic devices
US7688306B2 (en) 2000-10-02 2010-03-30 Apple Inc. Methods and apparatuses for operating a portable device based on an accelerometer
US6677932B1 (en) 2001-01-28 2004-01-13 Finger Works, Inc. System and method for recognizing touch typing under limited tactile feedback conditions
US6570557B1 (en) 2001-02-10 2003-05-27 Finger Works, Inc. Multi-touch system and method for emulating modifier keys via fingertip chords
JP4352381B2 (en) * 2003-09-19 2009-10-28 日本ビクター株式会社 Video phone equipment
US20050099492A1 (en) * 2003-10-30 2005-05-12 Ati Technologies Inc. Activity controlled multimedia conferencing
JP4994646B2 (en) * 2005-11-29 2012-08-08 京セラ株式会社 Communication terminal, communication system, and communication terminal display method
US7657849B2 (en) 2005-12-23 2010-02-02 Apple Inc. Unlocking a device by performing gestures on an unlock image
US8818175B2 (en) * 2010-03-08 2014-08-26 Vumanity Media, Inc. Generation of composited video programming
US8423058B2 (en) * 2010-04-07 2013-04-16 Apple Inc. Registering client computing devices for online communication sessions
WO2013169849A2 (en) 2012-05-09 2013-11-14 Industries Llc Yknots Device, method, and graphical user interface for displaying user interface objects corresponding to an application
CN103458215B (en) * 2012-05-29 2016-11-23 国基电子（上海）有限公司 System, mobile phone, electronic installation and the changing method of Switch Video call
AU2013368443B2 (en) 2012-12-29 2016-03-24 Apple Inc. Device, method, and graphical user interface for transitioning between touch input to display output relationships
US8848026B2 (en) * 2013-02-01 2014-09-30 Maitland M. DeLand Video conference call conversation topic sharing system
JP6111898B2 (en) * 2013-06-28 2017-04-12 富士電機株式会社 Image display device, monitoring system, and image display program
CN104869046A (en) * 2014-02-20 2015-08-26 陈时军 Information exchange method and information exchange device
US20150350141A1 (en) * 2014-05-31 2015-12-03 Apple Inc. Message user interfaces for capture and transmittal of media and location content
US9971500B2 (en) * 2014-06-01 2018-05-15 Apple Inc. Displaying options, assigning notification, ignoring messages, and simultaneous user interface displays in a messaging application
US9615058B2 (en) * 2014-06-05 2017-04-04 Reel, Inc. Apparatus and method for sharing content items among a plurality of mobile devices
JP2016001446A (en) * 2014-06-12 2016-01-07 モイ株式会社 Conversion image providing device, conversion image providing method, and program
CN104331246A (en) * 2014-11-19 2015-02-04 广州三星通信技术研究有限公司 Device and method for split screen display in terminal
US9380264B1 (en) * 2015-02-16 2016-06-28 Siva Prasad Vakalapudi System and method for video communication
JP6172610B2 (en) * 2015-03-17 2017-08-02 パナソニックＩｐマネジメント株式会社 Video conferencing system
US10061467B2 (en) * 2015-04-16 2018-08-28 Microsoft Technology Licensing, Llc Presenting a message in a communication session
US20160308920A1 (en) * 2015-04-16 2016-10-20 Microsoft Technology Licensing, Llc Visual Configuration for Communication Session Participants
CN104980578B (en) * 2015-06-11 2018-01-23 广东欧珀移动通信有限公司 A kind of event-prompting method and mobile terminal
CN105204846B (en) * 2015-08-26 2019-07-09 小米科技有限责任公司 Display methods, device and the terminal device of video pictures in more people's videos
CN105554429A (en) * 2015-11-19 2016-05-04 掌赢信息科技（上海）有限公司 Video conversation display method and video conversation equipment
WO2017218153A1 (en) * 2016-06-12 2017-12-21 Apple Inc. Devices and methods for accessing prevalent device functions
JP6634965B2 (en) * 2016-06-20 2020-01-22 株式会社リコー Communication terminal, communication system, communication control method, and program
JP6179834B1 (en) * 2016-09-27 2017-08-16 パナソニックＩｐマネジメント株式会社 Video conferencing equipment
US20180123986A1 (en) * 2016-11-01 2018-05-03 Microsoft Technology Licensing, Llc Notification of a Communication Session in a Different User Experience
CN107728876B (en) * 2017-09-20 2019-08-09 深圳市金立通信设备有限公司 A kind of method of split screen display available, terminal and computer readable storage medium
CN107704177B (en) * 2017-11-07 2020-04-21 Oppo广东移动通信有限公司 Interface display method and device and terminal
CN107992248B (en) * 2017-11-27 2020-09-29 北京小米移动软件有限公司 Message display method and device

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103649985A (en) * 2011-05-26 2014-03-19 谷歌公司 Providing contextual information and enabling group communication for participants in a conversation
CN103384235A (en) * 2012-05-04 2013-11-06 腾讯科技（深圳）有限公司 Method, server and system used for data presentation during conversation of multiple persons
CN104010158A (en) * 2014-03-11 2014-08-27 宇龙计算机通信科技(深圳)有限公司 Mobile terminal and implementation method of multi-party video call
CN105094957A (en) * 2015-06-10 2015-11-25 小米科技有限责任公司 Video conversation window control method and apparatus
US20160014059A1 (en) * 2015-09-30 2016-01-14 Yogesh Chunilal Rathod Presenting one or more types of interface(s) or media to calling and/or called user while acceptance of call
US20170324784A1 (en) * 2016-05-06 2017-11-09 Facebook, Inc. Instantaneous Call Sessions over a Communications Application

Also Published As

Publication number Publication date
CN111367603A (en) 2020-07-03
JP6998353B2 (en) 2022-01-18
CN110457096B (en) 2021-01-01
JP2020039139A (en) 2020-03-12
CN112214275B (en) 2021-10-29
CN110457095B (en) 2022-12-27
WO2019217477A1 (en) 2019-11-14
CN110457091A (en) 2019-11-15
JP7297044B2 (en) 2023-06-23
JP2023120280A (en) 2023-08-29
JP2019197520A (en) 2019-11-14
CN110457095A (en) 2019-11-15
CN110457096A (en) 2019-11-15
JP2022043185A (en) 2022-03-15
CN112214275A (en) 2021-01-12
JP6609361B2 (en) 2019-11-20

Similar Documents

Publication Publication Date Title
CN110457104A (en) Multi-player real time communication user interface
US11849255B2 (en) Multi-participant live communication user interface
CN113162843B (en) User interface for multi-user communication session
CN206649467U (en) A kind of electronic equipment
CN109219796A (en) Digital touch on real-time video
CN107430489A (en) The graphical configuration that shared user can configure
CN110109592A (en) Head portrait creation and editor
CN110460797A (en) Intention camera
CN106797415A (en) Telephone user interface
CN107637073A (en) Videograph and playback
AU2023248185B2 (en) Multi-participant live communication user interface
CN110096157A (en) Tactile output based on content
KR102541365B1 (en) Multi-participant live communication user interface

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
CB03 Change of inventor or designer information

Inventor after: M. van Oz

Inventor after: N.V.Jin

Inventor after: S.O. Lemme

Inventor after: J. Lausch

Inventor after: H. Enamel

Inventor after: L.S. Broughton

Inventor after: P. D. Callaway

Inventor after: P F Carol

Inventor after: J.W.Chang

Inventor after: A. Dreyer

Inventor after: A.C.Dai

Inventor after: C. M. Frederick

Inventor after: R. Garcia III

Inventor before: M. van Oz

Inventor before: N.V.Jin

Inventor before: S.O. Lemme

Inventor before: J. Lausch

Inventor before: H. Enamel

Inventor before: L. Broughton

Inventor before: P. Callaway

Inventor before: P F Carol

Inventor before: J.W.Chang

Inventor before: A. Dreyer

Inventor before: A.Dai

Inventor before: C. federeggi

Inventor before: R. Garcia

CB03 Change of inventor or designer information