JP6461081B2 - Output management for electronic communication - Google Patents

Description

  Various wired and wireless networks include Push-To-PTT (PTT), including over-cellular PTT (PTT over cellular) that enables two-way wireless services available to mobile communication devices on traditional cellular networks. Talk) has been developed to handle communication sessions. PTT provides a rapid one-to-one or one-to-many communication technology that is particularly useful for many applications. PTT communication connection generally activates a button or some other means of activation for the originating wireless communication device that opens the icon or communication between the originator and each member device of the group Is activated by Existing PTT systems, for example, have faster call setup times, ideally within one second compared to cellular voice channels that can take more than 5 seconds to establish a connection. Has advantages over conventional cellular systems.

  In some mechanisms, one PTT speaker is authorized to speak at a time. Such authority is commonly referred to as “floor grant” and cannot be spoken by other group members of the PTT session until the mobile device with the call permission releases the call right. Various forms of electronic communication have been developed to exchange information even when people are physically far from each other. By using electronic communication devices and communication networks, two or more people can communicate over virtually any distance. Historically, such telecommunications was only associated with telephones, but today, computers and mobile electronic devices are not only acoustic (i.e., voice), but also text, video, data or any combination thereof. Can be used for communication to communicate.

  Modern electronic communication devices are very convenient, but the intended recipient of the transmission of a live communication such as a telephone or video conference call is not ready or suitable for receiving the transmission May not be in the correct environment. For example, the recipient may be in a very quiet environment such as a library or a very noisy environment such as a construction site, and the acoustic output from the recipient communication device is unfavorable or inaudible Sometimes. Also, in some substantially immediate communication schemes, such as a push-to-speak scheme, the recipient device is not provided with a way to delay or manage incoming communications. Similarly, in a very bright environment, such as outdoors on a clear day, or a very dark environment, such as a movie theater, the video output may not be clearly visible or may be inappropriate or undesirable. Also, regardless of the environment surrounding the recipient device, the recipient himself may be busy or may require a short delay before joining the call. Thus, electronic communication often has problems with the user experience because the user cannot manage the output of the electronic communication or only limited management.

  Various embodiments include a method, system, and device for managing electronic communications on a communications device including receiving an incoming call indication from a source device. The methods, systems, and devices receive a first message segment from the originating device following the step of transmitting a delay indication in response to determining that it was authorized to accept the call late, followed by Receiving a second message segment. In addition, these methods, systems, and devices are responsive to determining whether to output at least one of the first message segment and the second message segment. Outputting at least one of the message segments may be included. Further, these methods, systems and devices reflect a progression of updated output of the first message segment at the recipient device in response to determining that the first message segment should be output. Transmitting a playback state including a state display of

  Further embodiments may include a computing device having a processor configured with processor-executable instructions for performing various operations corresponding to the methods discussed above.

  Further embodiments may include a computing device having various means for performing functions corresponding to the operations of the methods discussed above.

  Non-transitory processor-readable storage media that may be included in further embodiments store processor-executable instructions configured to cause a processor to perform various operations corresponding to the operations of the methods discussed above.

  The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate exemplary embodiments of the invention, and together with the summary set forth above and the detailed description set forth below, It serves to explain the features of the present invention.

3 is a call flow diagram according to one embodiment. 4 is another call flow diagram according to one embodiment. FIG. 6 illustrates a push and transmit (PTX) group of wireless communication devices displaying exemplary user interface images of various embodiments. FIG. 4 is a screenshot of a touch screen display of a source PTX device showing a PTX communication session of one embodiment. FIG. 3 is a screenshot of a touch screen display of a recipient PTX device that requested buffering, illustrating a PTX communication session of one embodiment. FIG. 5 is another screenshot of a touch screen display of a recipient PTX device that requested buffering, illustrating an alternative embodiment PTX communication session. FIG. 4 is a screenshot of a touch screen display of a recipient PTX device in a live communication mode showing a PTX communication session of one embodiment. FIG. 6 is a screenshot illustrating details of a playback display on a touch screen display of a source PTX device for use with various embodiments. FIG. 6 is a screenshot showing details of a playback display of another aspect of a touch screen display of a source PTX device for use with various embodiments. FIG. 7 is a screenshot showing details of a playback display similar to FIG. 6 of the touch screen display of the originating PTX device. 1 is a schematic diagram of a wireless communication device suitable for various disclosed embodiments. FIG. 1 is a schematic diagram of a wireless data network with a server suitable for various disclosed embodiments. FIG. 4 is a flow diagram of caller, server, memory buffer and PTX recipient communication according to one embodiment. 4 is a flow diagram of caller, server, memory buffer and PTX recipient communication according to another embodiment. 6 is a flow diagram of caller, server, memory buffer and PTX recipient communication according to yet another embodiment. 3 is a process flow diagram of an embodiment method for managing PTX communications; 6 is a process flow diagram of another embodiment method for managing PTX communications; 6 is a process flow diagram of an additional embodiment method for managing PTX communications. FIG. 6 is a process flow diagram of a further embodiment method for managing PTX communications. FIG. FIG. 6 is a process flow diagram of a further embodiment method for managing PTX communications. FIG. 3 is a flowchart of a video call according to one embodiment. 4 is a flow diagram of another video call according to one embodiment. It is a figure which shows switching of the manual mode during PTX communication by one Embodiment. It is a figure which shows switching of the manual mode during PTX communication by one Embodiment. It is a figure which shows switching of the manual mode during PTX communication by one Embodiment. It is a figure which shows switching of the manual mode during PTX communication by one Embodiment. It is a figure which shows switching of the manual mode during PTX communication by one Embodiment. It is a figure which shows switching of the automatic detection mode in PTX communication by one Embodiment. It is a figure which shows switching of the automatic detection mode in PTX communication by one Embodiment. It is a figure which shows switching of the automatic detection mode in PTX communication by one Embodiment. It is a figure which shows switching of the automatic detection mode in PTX communication by one Embodiment. It is a figure which shows switching of the automatic detection mode in PTX communication by one Embodiment. FIG. 4 is another screenshot of a touch screen display of a source PTX device showing activation from video off mode to video on mode. FIG. 23 is a screenshot similar to that shown in FIG. 22 showing the video on mode. It is a figure which shows the screenshot of the touchscreen display of the transmission origin PTX device which shows the some video stream transmitted / received simultaneously. FIG. 3 is a diagram illustrating a PTX group communicating with multiple simultaneous communication streams according to one embodiment. FIG. 6 is a block diagram of components of a mobile device suitable for use with various embodiments. FIG. 3 is a block diagram of components of a server device suitable for use with various embodiments.

  Various embodiments will be described in detail with reference to the accompanying drawings. Wherever possible, the same reference numbers will be used throughout the drawings to refer to the same or like parts. References to specific examples and implementations are for illustrative purposes, and are not intended to limit the scope of the invention or the claims.

  The word “exemplary” is used herein to mean “serving as an example, example, or illustration”. Any implementation described herein as "exemplary" is not necessarily to be construed as preferred or advantageous over other implementations.

  The terms “press and transmit”, “PTX”, “press and talk” or “PTT” are used herein to refer to a communication that provides a constant connection or at least substantially always on, such as “transceiver”. Used interchangeably to refer to services. Traditional push-to-talk (PTT) may use half-duplex, which means that communication can only take place in one direction at any given time, and only one user at a time can hear the story. is there. The form of the PTT message may also include video exchange. Therefore, the communication is not limited to “calling and transmitting”, that is, “call” called “PTX”. PTX may allow one communication group member to make video images available to the entire group. Also, the expression “press and send” takes into account the fact that modern mobile communication devices often have a touch screen interface that only needs to be “pressed” with a finger rather than pressed. It is. The video image may be a still photo or a moving video and is similarly considered a message segment. PTX communication may include audio or may be silent. Thus, PTX can be considered a generic term for this type of communication that exchanges at least one of voice, video and data messages. Also, PTX as used herein may use full duplex with control where one or more members of the communication group have priority over the channel of communication. A PTX message segment refers to a communication segment that includes the generation of audio and / or video transmissions by a user. A PTX message segment contains more than just signal data and thus contains a substantial portion of the message sent from the user to the other members of the PTX group. The transmitted message may include an audio transmission such as a “talk spurt”, a video transmission or a combination thereof. In the description of various embodiments, messages may be referenced in terms of one or more audio segments, but may such messages include video messages or other media streaming content intended for continuous live broadcasts? It should be understood that or consist entirely of these.

  The term “output” as used herein refers to the operation or processing of generation, distribution, or supply to or from a device such as a communication device such as a PTX device. The outputting step may include generating an audio sound, a video image, a series of images, or other display. Also, user interface (UI) components such as a display, one or more speakers, a print or other means may be used for output. Thus, a communication device such as a PTX device may be said to output audio, video, images, text and other information.

  The terms “mobile device”, “PTX device”, “wireless communication device” and “communication device” are used herein to refer to cellular phones, smartphones, portable multimedia players or mobile multimedia players, personal digital assistants ( PDA), laptop computers, tablet computers, smartbooks, palmtop computers, wireless email receivers, multimedia cellular phones connected to the Internet, wireless game controllers, and programmable processors and storage devices, including groups To be compatible to refer to any or all of similar portable electronic devices that may include software and / or hardware to enable communication, especially push-to-speak functionality It is.

  In one embodiment, modern electronic communications can be extended by allowing callers to request status indications from the intended recipient communications device. This can be done during or after call activation by the caller, for example by sending a call activation signal. The status indication may reflect that the recipient device is ready to receive communications or has requested a delay. The status indication may be automatically generated by the recipient communication device or may be provided in response to user input from the recipient communication device. Alternatively, the recipient communication device may use the microphone to automatically detect a noisy condition or a condition that kills the sound. User input may include the user pressing a designated button to indicate that the recipient is requesting a delay. The caller may be allowed to initiate a call despite the fact that the recipient device has requested a delay. In this way, the caller's message can be stored in a memory buffer for delayed playback. Later, the recipient may choose to either play the buffered message segment or start listening for live transmission from the caller. Also, if the recipient chooses to play the buffered message segment, a playback status may be provided to the caller. In this way, the caller is informed how much of the buffered message is being output from the recipient device. In addition, the caller may send additional signals to cease playback of the delayed message or interrupt playback with additional messages.

  In another embodiment, various wired and wireless networks are pushed-to-talk over mobile phones that allow two-way wireless services available to mobile communication devices on traditional cellular networks. It has been developed to handle PTX communication sessions including cellular). Traditional telephones, including mobile phone networks and devices, allow full-duplex communication that allows customers to call other people on a mobile phone or landline phone and simultaneously speak and listen to other groups. Used. In such communications, the connection must be initiated by dialing a phone number so that the other group answers the call, and the connection remains active until one of the groups ends the call, Or it is interrupted by signal loss or network outage. Traditional telephony protocols require long processing such as dialing, network switching and routing, call setup and waiting for other group responses. In contrast, PTX has a much quicker protocol and provides substantially immediate communication. Existing PTX systems have a call setup time that is ideally less than one second compared to a conventional cellular voice channel, which can take more than 5 seconds to establish a connection. PTX also provides the additional benefit of enabling an active caller to establish its direct connection with a group of recipients as well as a single recipient device.

  In some mechanisms, one PTX speaker at a time is authorized to speak or transmit video images. Such authority is commonly referred to as “call permission” and other group members of the PTX session cannot speak or transmit video until the mobile device with call permission releases the call permission. Once the call permission is granted, the speaker's input is immediately routed to and output from the mobile devices of all other members of the group. Thus, when the PTX group communication is activated, the recipient does not need to answer the call as in traditional telephone services, and immediately listens to and / or talks to the speaker authorized to make the call. Watch the person (or the image sent by the participant). Traditionally, participants physically release or re-reserve the PTX button to release the call right, and then other individual members of the group press their PTX button to request call permission (Ie, “request to send”).

  Various embodiments provide enhanced PTX communication capabilities and methods for managing those communications. In particular, various embodiments allow individual PTX devices that are members of a PTX group to play a delayed PTX message segment (otherwise for delayed playback by one or more requesting members). That is, it is possible to request shunting of a talk spurt or video spurt) (which may be referred to as a buffer request). In an alternative embodiment, the PTX message segments may be buffered in individual receiving PTX devices, communication system servers or connected storage devices, or transmitting PTX devices. Furthermore, the PTX device may automatically determine the need for buffering based on ambient noise and / or local conditions (also referred to as the ambient environment). On the other hand, other members of the same PTX group that do not require buffering may continue to listen to the PTX message segment without delay (ie, “live”), as with conventional PTX. In addition, the originator of the message segment may be notified that one or more PTX group members are requesting buffering or that buffering is taking place (i.e., buffer indication). This means that the PTX devices of those PTX group members are not currently ready to output the message or part thereof. In addition, the caller may be allowed to reject and / or edit the buffer request before rejecting the buffer request or outputting the buffered message segment from any device that did not listen live. . When a caller knows that a buffered message segment should not be heard live, it can decide that it should not be heard or that the message should be reworded . Finally, the buffered message segment may be output from the device of the PTX group member that requested the buffering. Also, the PTX group member devices received with the buffer request may eventually catch up so that subsequent unbuffered message segments can be output.

  Various embodiments may also include further enhancements related to message buffering during a PTX communication session. For example, the playback status of the buffered message segment may be provided to the originating PTX device. The playback status may indicate whether a particular message segment has been played and / or how much has been played. In addition, the playback state may include a speech to text conversion of the PTX message segment or at least a portion thereof. Furthermore, the playback state may indicate a portion of the speech-to-text conversion output at a particular recipient PTX device.

  As mentioned above, buffering may occur with the recipient PTX device requesting the buffer, the originating PTX device, or some other storage resource available in the PTX network. The PTX network server also outputs the segments once the ready signal is received indicating that the recipient PTX device requesting the buffer is ready to output the message segments, or the PTX The buffered message segments may be managed so that the rest of the group can be notified. Thus, depending on whether the originating PTX device indicated that a particular segment should be erased, it is allowed or prevented to output that segment when a ready signal is received. Is done.

  In one embodiment, a method, system or device for managing PTX communications on a PTX device is disclosed. The recipient PTX device may receive a first indication that outgoing permission has been granted to the originating PTX device. It may also be determined whether the PTX message segment from the originating PTX device should be buffered for delayed playback by the recipient PTX device. A buffer signal may be transmitted based on the determination that the PTX message segment should be buffered. The buffer signal may indicate one of a request to buffer the PTX message segment and a recipient storage indication indicating that the PTX message segment is stored in the storage device of the first recipient PTX device. In addition, a ready signal may be transmitted indicating that the recipient PTX device is ready to output a PTX message segment.

  In various embodiments, the PTX message segment may be received at the recipient PTX device. Also, in response to determining that the PTX message segment should be buffered, the PTX message segment may be stored in a storage device of the recipient PTX device, and the buffer signal includes the recipient's storage indication. Further, an erasure signal may be received in connection with the PTX message segment, which may include erasing the PTX message segment from the storage device of the recipient PTX device. As a first instruction that the transmission source PTX device is permitted to transmit, a transmission right occupied signal (floor taken signal) may be received. The transmission right occupied signal may be transmitted separately from the PTX message segment. In addition, a playback status indicating a portion of the PTX message segment output from the recipient PTX device may be transmitted. In addition, the noise level around the recipient PTX device may be measured. Thus, based on ambient noise level measurements, it may be determined whether to buffer the PTX message segment from the originating PTX device. Further, input from the user interface of the recipient PTX device may be received. Accordingly, based on input from the user interface, it may be determined whether to buffer the PTX message segment from the originating PTX device.

  In another embodiment, a method, system, and device for managing PTX communications between PTX devices are disclosed. Specifically, at least one of a buffer request and a buffer state may be received from the recipient PTX device. The buffer request and buffer state may be associated with a PTX message segment from the originating PTX device that is stored for delayed playback. Also, a buffer indication reflecting that at least one of the buffer request and the buffer status is received from the recipient PTX device may be transmitted to the source PTX device.

  In various embodiments, the PTX message segment may be stored in a server storage device. A ready signal may also be received indicating that the recipient PTX device is ready to output a PTX message segment. A PTX message segment may then be sent to the recipient PTX device in response to receiving the ready signal. In addition, a PTX message segment may be stored in the server storage and an erasure signal may be received from the originating PTX device. In this way, the PTX message segment may be erased from the storage device to prevent the PTX message segment from being output from the recipient PTX device. Alternatively, a first erasure signal may be received from the originating PTX device to prevent the PTX message segment from being output from the recipient PTX device. A second erasure signal may also be sent to the recipient PTX device so that the recipient PTX device erases the PTX message segment. Further, a playback state indicating how many PTX message segments have been output from the recipient PTX device may be received at the server. In addition, the playback status may be sent to the originating PTX device. Further, a speech to text conversion of at least a portion of the PTX message segment may be received from the recipient PTX device. In this manner, a playback state output by the recipient PTX device and indicating a portion of the voice-to-text conversion may be transmitted to the originating PTX device. The speech-to-text conversion of at least a portion of the PTX message segment may be generated by a server or associated resource. In this way, a playback state output by the recipient PTX device and indicating a portion of the speech to text conversion may be sent to the originating PTX device.

  In further embodiments, methods, systems and devices for managing PTX communications on PTX devices are disclosed. For example, a source PTX device with active call authorization may receive at least one of a buffer request and a buffer status. The buffer request and buffer state may be associated with a PTX message segment from the originating PTX device that is stored for delayed playback at the recipient PTX device. Further, the source PTX device may output at least one instruction among the buffer request and the buffer status. A ready signal may be received indicating that the recipient PTX device is ready to output a PTX message segment.

  In various embodiments, the PTX message segment may be stored in the storage device of the originating PTX device. Also, the PTX message segment may be transmitted in response to receiving the ready signal. The PTX message segment may be stored in a storage device so that it can be determined whether the PTX message segment should be erased. The PTX message segment is then erased from the storage device in response to determining that the PTX message segment should be erased to prevent the recipient PTX device from outputting at least a portion of the PTX message segment. Good. It may be determined whether at least one portion of the PTX message segment should be output from the recipient PTX device. Further, an erasure signal may be transmitted in response to determining that at least one portion of the PTX message segment should not be output from the recipient PTX device. A playback state indicating how many PTX message segments have been output from the recipient PTX device may be received at the source PTX device. In addition, the playback state may be output by the source PTX device. Alternatively, a playback status indicating how many PTX message segments have been output may be received from the recipient PTX device. The playback state may include a voice-to-text conversion of at least a portion of the PTX message segment. Also, the playback state may be displayed on the display of the source PTX device. In addition, a playback state has been received indicating how much of the PTX message segment has been output from the recipient PTX device and that a speech to text conversion has been generated for at least one part of the PTX message segment. Good. The originating PTX device may display a playback status that includes at least one portion of the voice-to-text conversion.

  In further embodiments, extensions may be provided for video transmissions that are included or transmitted instead of acoustic transmissions of PTX communications. For example, around the area of the source device may be useful for transmission, such as at a construction site or when the user of the source device wants to show the recipient something other than himself. In addition, the user of the originating device may transfer it to his raw image, which in turn allows further expansion. For example, image recognition and facial expression recognition can be used to transmit or convert facial expressions to text or emoticons. Image recognition can also be used to convert a message from a gesture language to a text message or an acoustic message.

  Another embodiment allows the originating device to request information regarding the status of the recipient device, such as the need for buffering. Information regarding the status of the recipient device may be displayed on a user interface (UI) of the originating device such as a display. The recipient device may choose to play the transmitted communication when ready, may convert the acoustic message to a text message, or acoustic to catch up with the live broadcast from the originating device. / You can fast-forward the video message. In addition, the user of the originating device may specify a key frame from the video transmission and the recipient user may choose to view the key frame instead of the entire video.

  Further embodiments include one that allows a user of a PTX device to control the output mode of the device. Such control may be manual (ie, activated by user input) or may be performed automatically by the PTX device. In this way, the user of the recipient device can switch to the sound only mode simply by making a selection on the UI. Also, the user may select acceleration / deceleration of video or audio transmission by making another selection on the UI. These inputs may be made by direct contact with the PTX device by the user or by gesture recognition. Gesture recognition can be performed by a PTX device that can be equipped with a camera and software to detect and analyze a user's physical motion or device orientation. For example, a user swiping a hand in the vertical direction can represent one action, and swiping a hand in the horizontal direction can represent another action. Alternatively, an orientation of the PTX device may be employed, such as pointing the right side of the PTX device down / up. As a further alternative, the recipient device may use a sensor to determine the user's situation and select an appropriate mode. Also, the mode may be switched when the user's environment changes, such as when the bright sun is hidden in the clouds in an outdoor environment or when the noise level around the recipient device changes.

  Further embodiments include a recipient device that can receive several audio and / or video streams simultaneously. The user of the recipient device can select / view / listen to one of the available video or audio, rather than watching and listening to all video and audio segments simultaneously. In this way, the recipient device may display a video stream that includes audio output from one source device, and may display a video stream that does not have audio output from another source device. Also good. Some devices may have the ability to convert sound to text to minimize or eliminate some of the speech emitted from the recipient device. Also, sound to text conversion may provide a subtitle for the video display. Further extensions to this functionality may even include language translation in the form of audio output, translated into the desired language, of subtitle text or video dubbing. Still further, a motion to text conversion function may be provided to convert the physical motion of the originating user into text or sound output. For example, if the originating user is waving as part of a message segment, the recipient device may output “WAVING HAND” as text or provide an equivalent sound output. Good.

  FIG. 1A shows a call flow diagram between a calling party 11 and an intended recipient 12. Both caller 11 and intended recipient 12 are simply represented by a user interface (UI). Caller 11 sends a request for status indication from intended recipient 12. The status indication may reflect that the recipient device is ready to receive communications or has requested a delay. The status indication may be automatically generated by the intended recipient 12 communication device or may be provided in response to user input from the intended recipient 12. Automatic responses may include situations where the intended recipient device user is pre-set to require longer response times, such as additional call rings. In this way, if there is a call, it will allow more than two or three conventional rings before being sent to voicemail. Alternatively, the recipient communication device may use its microphone (not shown) to make a noisy condition where a loud crowd surrounds the device, or a condition where sound such as in a pocket or purse is killed Can be detected automatically. User input may include the user pressing a designated button (eg, a “Delay” button on the UI) to indicate that the recipient is requesting a delay. Alternatively, the intended recipient 12 may simply accept the call (by pressing the “Accept” button), which is rather similar to a traditional telephone call answer, or ( You may simply reject the call completely (by pressing the "Decline" button).

  FIG. 1B shows a further call flow diagram after the call from FIG. 1A is activated and the call becomes active. The caller 11 may be allowed to initiate a call despite the fact that the intended recipient 12 is requesting a delay. In this way, the caller's message can be stored in a memory buffer for delayed playback. Later, the recipient may choose to either play the buffered message segment or skip the buffered message and start listening to the live broadcast from the caller exactly. As shown in FIG. 1B, the intended recipient 12 is in delay mode and the call is actively stored in a buffer. A status indication may be provided to the caller 11, so that the caller 11 has started listening to the buffered message that the intended recipient 12 has not yet "accepted" the call. recognize. In this way, the caller is informed whether a buffered message is being output from the recipient device. In addition, the caller can pause buffering by pressing the “Buffer” button, and then resume transmission of additional message segments by pressing the “Start” button. Alternatively, the caller may simply end the call by pressing an “End” button.

  FIG. 2 shows a PTX communication group 100 that includes five PTX devices in various PTX states. At the time shown in FIG. 2, the first PTX device 102 has outgoing permission, and the second PTX device 104 is playing a buffered message segment from the current PTX session and has three Another PTX device 106 is outputting the raw message segment generated by the first PTX device 102. Through various embodiments, each PTX device 102, 104, 106 may include elements commonly found in conventional PTX devices such as displays 112, 114, 116, and controller 115. Also, some or all of the PTX group devices may have extended functionality as described herein. In other words, not all communication devices of the PTX group need to have the same functionality and / or extended capabilities. In various embodiments, these methods may be implemented using a wireless communication device that does not include dedicated PTX hardware, such as PTX buttons or PTX specific circuitry. Furthermore, although the PTX communication group 100 of the embodiment includes all smartphone-type communication devices, the PTX device may be any device that can communicate with the PTX network by software and / or hardware. Further, the displays 112, 114, 116 are shown as similar touch screen user interfaces, but are shown displaying various features based on their PTX status in accordance with embodiments herein.

  The first PTX device 102 may activate a call right request so that control of the PTX call right is permitted, and may begin sending a message after receiving the call permission. In FIG. 2, the first PTX device 102 is shown with “FLOOR GRANTED”, which means that the first PTX device 102 has received the call permission and is currently maintaining it. It is. The second and third PTX devices 104, 106 are shown with “Floor Taken”, which informs the user of the device that the call right is being used by another member of the group It is. In addition, the second PTX device 104 is shown in “PLAYBACK” mode, which is the novel PTX mode described herein. For the playback mode, the message sent by the first PTX device is stored in the buffer rather than being output immediately. Prior to the time shown in FIG. 2, the second PTX device 106 shown was not ready to output a PTX message and issued a buffer request. When granted, the buffer request stores one or more message segments generated by the PTX device along with the outgoing permission to allow playback by the requesting PTX device after the delay. This delay may be a predetermined time or may be the time required for the requesting device to get ready to begin outputting buffered segments. At the time shown in FIG. 2, playback of the first of the two message segments stored in the buffer of the second PTX device has been activated. The other three PTX devices 106 are also shown outputting the message “LIVE” of the first PTX device, similar to the conventional PTX output mode. Further aspects of various PTX modes are further described below.

  3-7 are screen shot illustrations of an exemplary PTX device touch screen user interface. The wireless communication device illustrated by the screen shot may include a display that provides visual feedback of the participation status of one or more devices in PTX communication. In various embodiments, the wireless communication device may display visual feedback indicating the group's users who can participate in group communications as well as users who are not allowed to participate. In various embodiments, the wireless communication device may display the status of one or more users who are unable to participate in live group communication.

  FIG. 3 shows a screen shot from the first PTX device 102 that is the source PTX device. There are various indicators on the device display 112 that provide visual feedback regarding the current PTX session. The session name field 120 indicates a rather comprehensive “Press-to-transmit Group” as the name of the session. Session name field 120 may also include additional information such as the number of participants in the PTX session. In this illustrative example, a display of “(You + 4)” means that a total of 5 PTX group members are participating in the current active session. Further expanded functionality may be provided, such as a participant add-on icon 128 for adding additional PTX group members. The session roll field 122 provides information regarding whether the participant has permission to make a call. In this example, the indication “FLOOR GRANTED” indicates to the user or the person watching the device that the first PTX device 102 currently has permission to make a call. In addition, an additional status icon 125 is provided in the form of a microphone to give the user of the first PTX device 102 further alert that the remaining members of the PTX group are listening. In this way, the display 112 informs the user of his current role in the PTX session in more than one part of the screen in more than one way, not just in one direction. Alternatively, if it is a full-duplex environment or if multiple PTX group members are allowed to become active in the simultaneous transmission of communications, the session roll field 122 may be changed accordingly. In this way, PTX group members may be given a priority level and multiple group members may be allowed to communicate simultaneously depending on their priority level. For example, it should be noted that in FIGS. 20 and 21 where the session roll field 122 indicates “PRIORITY GRANTED”, it means that the device has priority over the rest of the communication group. The priority may be shared depending on the setting, but the session roll field 122 may be used to indicate the priority status to the user. Below the session roll field 122 are icons and indicators for some or all of the session participants. In this illustrative example, four other participants (members other than the user of the PTX device 102) are represented. The first recipient PTX participant 131 includes a portion of the name (ie Seth R) and an image. Such images need not be live-action to each participant, but rather may be avatars or any type of images as commonly used in online social networks. The second recipient PTX participant 132 also includes a portion of the name (ie Kameron) and an image. Third and fourth recipients PTX participants 133, 134 include a name (i.e. Anne and Jonah), but a generic avatar that means that specialized images for those participants are not available It is included.

  Below the participant image on the display 112 is a session status indicator 142 that describes the status of each participant according to embodiments disclosed herein. Second participant 132, third participant 133, and fourth participant 134 are shown having a session state of "Live", indicating that these members have permission to make calls. Means listening to communication from the originating PTX device without substantial delay. In this case, those participants 132, 133, 134 are listening to live communications from the first PTX device 102. In contrast, the first participant 131 is labeled “Delayed”. A session state of “Delayed” means that the participant was inconvenient and requested buffering. Buffer requests are very convenient when a particular user is unable to listen to or be interested in the source participant for the time being. Alternatively, the participant's device may automatically send a buffer request under certain circumstances. For example, when a PTX device recognizes that it is in a very noisy environment based on the ambient noise level measured with the device's microphone, such as when the noise level exceeds a threshold that the user cannot hear well, You may send a request. As another example, a PTX device may be in a very quiet environment such as a library where normal output from the device may not be desirable based on ambient measured noise levels, also sensed by the device's microphone. When it is recognized that it exists, a buffer request may be transmitted. Whether the user or device initiated the buffer request, once created, the buffer request may be granted, rejected, or even canceled after it was first granted. Further, such permission or denial may be automatically performed based on a predetermined setting, or may be manually performed by a user. In the example shown, Seth R's buffer request was allowed. An authorized buffer request means that communications sent from the originating PTX device are stored in a memory buffer for later playback. The communication may be divided into segments so that participants can cancel (ie, delete) the selected segment and listen to others with a delay. Segment decomposition may be based on predetermined time intervals, caller story breaks, or some other measure.

  Again, the memory buffer used to store the PTX message segment may be part of the originating PTX device, the recipient device requesting the buffer, the server, or some other element associated with the PTX group. In the example shown in FIG. 3, playback states 151 and 152 are shown for each of the two stored PTX message segments. The first of the two PTX message segments 151 is shown in the course of playback processing on the first participant's device 104. Each of the message segments may include additional status indicators such as numbered label field 161, played time field 162, and unplayed time field 168. A progress bar 165 may also be provided to show a direct visual indicator of the played and unplayed portions of the PTX message segment. A selection button 170 may also be provided to access additional information about the segment or simply to delete the message segment from the memory buffer. In this way, the user of the first PTX device 102 may touch one of the selection buttons 170 on the screen (shown as a circle surrounding the minus sign) to erase that message segment. . Alternatively, touching the select button 170 may expand the display for that message segment, as described further below with respect to FIGS. 5B-7.

  The screen shot from the second PTX device 104 shown in FIG. 4A is, in this example, operated by the first participant user 131 on the recipient PTX device. The device display 114 of the second PTX device 104 includes various visual indicators of the state of the current active PTX session, similar to the first PTX device 102 described above. That is, a session name field 120, an icon 128 for adding a participant, a session roll field 122, and a participant icon. This screen shot shows a slightly different group of PTX participants since the user of the second PTX device 104 need not be represented on its own display. In this example, user 135 (ie Mark L.) of the originating PTX device is shown. Arranged in a prominent position, such as the leftmost, may include not only the originating PTX device, but also a “FLOOR” session status indicator 142 to reflect that Mark L. has permission to originate. In this example, session status indicator 142 includes a further indication that the message segment of the originating PTX device is “Buffering”. By doing this, other users understand that the originating user is enabling buffering. Some form of media player may be provided when the second PTX device 104 is playing a buffered message segment rather than the microphone session state icon used for the originating PTX device. In this example, a caption 123 of “PLAYBACK” is also included, so that the user of the second PTX device 104 is aware of his current PTX state. Like the smaller media player that represents the playback state used on the originating PTX device, the larger version here is a numbered label field 161, a played time field 162 and an unplayed time field 168 and a progress A bar 165 may be included. Preferably, the playback state shown on the device playing the buffer should be the same as that shown on the originating PTX device, although there may be a slight delay. In contrast to the select button 170 on the originating PTX device, the select button 170 on the recipient device is “pause” (represented by two vertical lines), “play” (represented by a triangle) ), Or “stop” (represented by a square and stops playback).

  The alternative screenshot from the second PTX device 104 shown in FIG. 4B is that operated in this example by the first participant user 131 on the recipient PTX device. In this embodiment, multiple message segments are buffered, but those segments were generated on different PTX devices (ie, sent from different group members). Similar to the illustrative example of FIG. 4A, in FIG. 4B, the second PTX device 104 is playing a buffered message segment 151 (the first of the two). A “PLAYBACK” caption 123 is also included, so the user of the second PTX device 104 is aware of his current PTX state. The difference in this embodiment is from the current caller (Mark L.), where the second message segment 152 holds the call permission, and the first message segment 151 is the second participant user 132 (Kameron ). Since each message segment represents a different originator, as shown, the label field 161 includes a small version of the PTX group member image or icon and may further include their name or similar distinguishing indicator. It should be understood that each of those callers may have an indication on their display that their messages are buffered or buffering, as described with respect to FIG. .

  FIG. 5A shows a screenshot from a third PTX device 106 that is identical to the fourth and fifth PTX devices of this example. The third, fourth and fifth PTX devices 106 represent recipient PTX devices that receive live communications from the originating PTX device. The device display 116 of the third PTX device 106 includes various visual indicators of the state of the current active PTX session, similar to the first PTX device 102 and the second PTX device 104 described above. That is, a session name field 120, an icon 128 for adding a participant, a session roll field 122, and a participant icon. This screenshot shows yet another group of expected PTX participants. Here, caption 123 indicates “LIVE” reflecting that this participant is receiving live communication from a source PTX device that is authorized to transmit. In this example, the only selection button 170 prepared is for making a buffer request.

  The screen shot shown in FIG. 5B is similar to that of FIG. 3 and similar to that of the first PTX device 102, but slightly modified. In FIG. 5B, display 112 includes an expanded playback state to show one particular participant, Seth R, in more detail. As shown, the playback state includes a voice-to-text conversion of the two message segments 151,152. Although a portion of the second message segment 152 has been cut, a similar element may be provided in the right scroll bar to allow the user to scrutinize the buffered message segment. The expanded playback status display 166 additionally shown represents the played and unplayed portions of the message segment. The text shown in bold underlined represents a conversation that has not yet been played on the delayed participant's device. Alternatively, color coding or some other distinction may be used to distinguish and reveal the playback state.

  The further screenshots shown in FIGS. 6 and 7 are similar to those of FIG. 3 and similar to those of the first PTX device 102, but are also slightly modified from FIG. 5B. The embodiment of FIGS. 6 and 7 includes an extended playback status display 167 that simply erases any text that has already been played. However, as in the previous screenshot, the select button 170 continues to be effective in deleting the selected message segment 151,152. In FIG. 6, the user 5 is about to press the selection button 170 for the first message segment 151. As a result, the remaining part of the first message segment 151 is erased, and it looks like the illustration of FIG. Thereafter, user 5 may optionally press another select button 170 to erase other portions of the buffered message segment.

  FIG. 8 shows a schematic diagram of a PTX device 101 that can be used in various embodiments. The various PTX devices 102, 104, 106 described above may be substantially similar to the PTX device 101. In various embodiments, individual members of the PTX group communication provided by each PTX device 101 utilize software and native device hardware such as touch screen display 110 and / or other user interfaces such as a keypad interface. . In various embodiments, the PTX method described herein can be implemented on a wide range of devices, including most Internet-connected mobile phones and other wireless communication devices. The method may also be implemented using a wireless communication device that does not include dedicated PTX hardware such as PTX buttons or PTX specific circuitry. A processor platform 180 that such a PTX device 101 may include may handle voice and data packets, execute software applications, and transmit information across a wireless network. The processor platform 180 includes, among other components, a RISC processor such as a processor 185 such as an application specific integrated circuit ("ASIC") or one that implements an ARM architecture, among others. The processor 185 is typically installed when manufacturing the wireless communication device 101 and is not normally upgradeable. An application programming interface (“API”) layer 183 that is executed by the processor 185 or other processor includes a resident application environment and may include an operating system loaded on the processor 185. The resident application environment interfaces with any resident program on storage device 181 such as the computer readable storage medium of wireless communication device 101, for example.

  As shown in FIG. 8, the wireless communication device 101 may be a wireless communication telephone having a graphics display 110, but may be any wireless device having a processor platform 180 known in the art, such as a personal digital assistant (PDA). It may be a pager with a graphics display 110, or a separate processor platform 180 with a wireless communication portal, otherwise it may have a wired connection to a network or the Internet. Further, the storage device 181 may comprise read only memory or random access memory (RAM and ROM), EPROM, EEPROM, flash card, or any storage device common to processor platforms. The processor platform 180 may also include a local database 189 for storing software applications that are not actively used on the storage device 181. The local database 189 is generally composed of one or more flash memory cells, but any secondary known in the art, such as magnetic media, EPROM, EEPROM, optical media, tape, or soft disk or hard disk. It may be a storage device or a tertiary storage device. Graphics display 110 may show information regarding ongoing group PTX calls, as described more fully herein.

  The processor platform 180 may also include a direct communication interface 187 configured to open a direct communication channel. The direct communication interface 187 may be part of a standard communication interface for the wireless communication device 101 that carries audio, video and / or data that is normally transmitted to and received from the wireless communication device 101. The direct communication interface 187 may comprise hardware known in the art. In various embodiments, software for enabling the PTX function of the wireless communication device 101 may be preinstalled on the device during device manufacture. In addition or alternatively, at least a portion of the software that provides the PTX function may be downloaded to the device 101, or otherwise installed on the device. As a further alternative, the PTX software or a portion thereof may be a downloadable application such as a mobile application.

  FIG. 9 illustrates one or more wireless communication devices such as one or more cellular phones, smart phones, tablet computers, smart pagers, personal computers and personal digital assistants (PDAs) in PTX Group 201 and other 1 illustrates an exemplary embodiment of a system 200 for managing PTX communication with a wireless communication device. In the system 200, each wireless communication device 201 may be capable of selectively communicating with a target set of one or more other wireless communication devices throughout the wireless communication network 203. The target set of wireless communication devices may be all devices in communication group 201 or a subset thereof.

  In one embodiment, the wireless communication device may communicate individually with the server 204 residing in the server-side network, such as through the router 210, throughout the wireless data network 203. Server 204 may share information with a set of target wireless communication devices specified by the originating PTX device, either anywhere in wireless network 203 or another network, or any other accessible to server 204. Information may be shared with a computing device. Server 204 may have an attached or accessible database 206 for storing group data and / or buffered message segments. Also, some or all of the functions of server 204, router 205 and database 206 may be combined in a single device.

  In various embodiments, server 204 may establish a PTX channel between all devices in group 201 or selected devices. Such communication may occur by the server 204, i.e., under control of the server 204. However, although not all data packets of a device need necessarily travel through the server 204 itself, in various embodiments, preferably the server 204 can ultimately control the communication. The reason is that, generally, the server 204 knows and / or can obtain the identity of a member of the communication group, or the identity of the member of the communication group 201 is another computer device. This is because it is the only or primary server-side component that can be directed to. A PTX channel may be established between communicating wireless communication devices via a (true or virtual) half-duplex channel. Server 204 may notify some devices in group 201 when other members of the group cannot be reached or request a delay. Furthermore, although the server 204 is shown here as having an attached database 206, it may have its own data storage and database functions.

  In various embodiments, the server 204 performs an arbitration function between competing requests of various wireless communication devices 201 for use with or controlling the PTX communication channel across the wireless network 203. You can do it. For example, in response to a request from one wireless communication device of PTX group 201 with one or more other target devices, server 204 requests the requesting (originating) device and group 201. A PTX channel may be established between all or a portion of the target devices. Accordingly, the server 204 may allow the source wireless communication device to control the “call right”. When there are competing requests for control of “send rights” among the devices of group 201, server 204 may arbitrate between competing requests based on a predetermined priority criterion. Priority criteria may be established when forming the PTX group 201, such as by a group administrator, and stored in the server 204 and / or database 206.

  In various embodiments, the server 204 receives a data package from a wireless communication device (such as wireless communication devices 101, 102, 104, 106) and a member of the communication group 201 stores the stored data package in a wireless communication network. It may be configured to selectively allow reception throughout 203. In one embodiment, the data package includes, but is not limited to: photos such as JPEG, TIF, flash video, video files such as AVI, MOV, MP4, MPG, WMV, 3GP, audio files such as MP3, MP4, WAV, Documents and / or presentations may be included. The data package may further include streaming media such as a multimedia application (powerpoint, MOV file, etc.). The data package may also include a half-duplex video conference between members of the communication group where the speaker's image may be broadcast to other group members in substantial real-time or delayed.

  System 200 may use server 204 or database 206 to store data packets containing buffered message segments for reception by the target PTX device. Alternatively, such data packets may be stored locally at the originating PTX device and / or one or more recipient devices. When individual members of the PTX group 201 receive the data package, the server 204 may be notified, and then the server 204 will see the PTX group member who received such data package for the originating PTX device. And / or an acknowledgment that informs the state when those PTX group members have not yet received or are receiving communications.

  A series of communication servers that can be included in the wireless data network 203 controls communication between wireless communication devices of PTX group members in the PTX system. Depending on any system that a wireless data network may include, between two remote modules and between three or more remote modules, and / or two of the wireless network, including, but not limited to, wireless network carriers and / or servers Wireless communication is performed between components and between three or more components. A series of communication servers may be connected to a group communication server. Such a server may be connected to a wireless service. The wireless network may control messages sent (typically in the form of data packets) to the message service controller.

  10-12 are communication flow diagrams illustrating exemplary interactions between PTX devices, network servers, and storage devices. FIG. 10 shows a communication flow starting from a source PTX device 1002 (shown as “sender”). Communication is a group communication server 1004 (also referred to simply as “server”), a memory buffer 1006, a first recipient PTX device 1008 (shown as “receiver 1”), and the abbreviation “receive” Also flows to a further group of recipients 1010, designated “N” (the group of recipient PTX devices is numbered from 1 to N, where N is the last recipient PTX device). In this embodiment, a transmission right request 1020 is transmitted from the caller 1002 to the server 1004. The server 1004 may transmit the transmission right occupied signal 1024 to the recipients 1008 to 1010 after confirming the provisional PTX group members. In addition, the server 1004 may return a call permission signal 1023 to the caller 1002. A buffer request 1028 may then be sent from the recipient 1 1008 to the server 1004. On the other hand, the caller 1002 may transmit the first communication segment 1032 (“communication segment 1”) for distribution to the receiver to the server 1004. When server 1004 receives buffer request 1028, server 1004 may send buffer instruction 1043 (“R1 buffering”) to caller 1002. The server 1004 may also ensure that the first communication segment 1032 is transmitted to the memory buffer 1006 for at least temporary storage. Since other recipients 1010 have not sent buffer requests, the first communication segment 1032 may also be sent to other recipients 1010. In this exemplary embodiment, the ready signal 1063 transmitted by the first recipient 1008 to the server 1004 is then transmitted by the server 1004 to the originator 1002. The server may then access the memory buffer 1006 or send a ready signal 1063 to send the first communication segment 1032 to the first recipient 1008 ready to receive. . When the first recipient 1008 plays the first communication segment, a playback state 1073 (“Communication 1 state”) may be sent to the server 1004 and then sent to the caller 1002. At the end of this example, caller 1002 sends a second communication segment 1070 (“communication segment 2”) to server 1004 for distribution. Since there is no active buffer request, the server sends a second communication segment 1070 to all recipients 1008-1010. In this example, the second communication segment 1070 generated next to the first communication segment 1032 is transmitted to the recipient after playing back the buffered first communication segment 1032.

  FIG. 11 shows a communication flow starting from the source PTX device 1102 (shown as “sender”). Communications include a group communication server 1104 (also referred to simply as “server”), a memory buffer 1106, a first recipient PTX device 1108 (shown as “receiver 1”), and the abbreviation “receive Also flows to a group of additional recipients 1110 denoted “N” (the group of recipient PTX devices is numbered from 1 to N, where N is the last recipient PTX device). In this embodiment, the transmission right request 1122 is transmitted from the caller 1102 to the server 1104. The server 1104 may transmit the transmission right occupied signal 1124 to the recipients 1108 to 1110 after confirming the provisional PTX group members. Further, the server 1104 may return a call permission signal 1123 to the caller 1102. A buffer request 1128 may then be sent from the recipient 1 1108 to the server 1104. On the other hand, the caller 1102 may send a first communication segment 1132 (“communication segment 1”) for distribution to the recipient to the server 1104. When server 1104 receives buffer request 1128, server 1104 may send buffer instruction 1143 (“R1 buffering”) to caller 1102. The server 1104 may also ensure that the first communication segment 1132 is sent to the memory buffer 1106 for at least temporary storage. Since other recipients 1110 have not sent buffer requests, the first communication segment 1132 may also be sent to other recipients 1110. In this embodiment, caller 1102 sends second communication segment 1152 (“communication segment 2”) to server 1104 before the ready signal is sent from first recipient 1108. Since the buffer request from the first recipient 1108 continues to be active, the server 1104 may ensure that the second communication segment 1152 is sent to the memory buffer 1106 for at least temporary storage. Since the other recipient 1110 has not sent a buffer request, the second communication segment 1152 may also be sent to the other recipient 1110. A ready signal 1163 that can then be transmitted by the first recipient 1108 to the server 1104 is then transmitted by the server 1104 to the originator 1102. The server may then access the memory buffer 1106 or send a ready signal 1163 to send the first communication segment 1132 to the first recipient 1108 ready to receive. . When the first recipient 1108 plays the first communication segment 1132, it may send a playback state 1173 (“communication state 1”) to the server, which then sends the playback state 1173 to the caller 1102. May be sent to The server may also later send a second communication segment 1152 to the first recipient 1108 ready to receive. When the first recipient 1108 plays the second communication segment 1152, it may send a second playback state 1183 ("communication state 2") to the server, and then the server will send the playback state 1183 to it. It may be sent to the caller 1102. In this example, the second communication segment 1152 generated next to the first communication segment 1132 is transmitted to the recipient prior to playing the buffered first communication segment 1132.

  FIG. 12 shows a communication flow starting from the source PTX device 1202 (shown as “sender”). The communication includes a group communication server 1204 (also referred to simply as “server”), a memory buffer 1206, a first recipient PTX device 1208 (shown as “receiver 1”), and the abbreviation “receive”. Also flows to a group of further recipients 1210, designated as “N” (the groups of recipient PTX devices are numbered from 1 to N, where N is the last recipient PTX device). In this embodiment, the transmission right request 1222 is transmitted from the caller 1202 to the server 1204. The server 1204 may transmit the transmission right occupied signal 1224 to the recipients 1208 to 1210 after confirming the provisional PTX group member. In addition, the server 1204 may return a call permission signal 1223 to the caller 1202. A buffer request 1228 may then be sent from the recipient 1 1208 to the server 1204. On the other hand, the caller 1202 may transmit the first communication segment 1232 (“communication segment 1”) for distribution to the receiver to the server 1204. When server 1204 receives buffer request 1228, server 1204 may send buffer indication 1243 (“R1 buffering”) to caller 1202. The server 1204 may also ensure that the first communication segment 1232 is sent to the memory buffer 1206 for at least temporary storage. Since other recipients 1210 have not sent buffer requests, the first communication segment 1232 may also be sent to other recipients 1210. In this embodiment, before the first communication segment 1232 is played, the caller 1202 sends an erasure signal 1272 to the server 1204 indicating that the buffered first communication segment 1232 should be erased. You can do it. An erase signal 1272 may then be sent to the memory buffer 1206 to erase the first communication segment 1232. A ready signal 1263 that can be sent by the first recipient 1208 to the server 1204 after becoming ready is then sent by the server 1204 to the caller 1202. At the end of this example, caller 1202 sends a second communication segment 1270 (“communication segment 2”) to server 1204 for distribution. Since there is no active buffer request, the server sends a second communication segment 1270 to all recipients 1008-1010. In this example, after the first communication segment 1032 is erased from the memory buffer 1206, the second communication segment 1270 generated next to the first communication segment 1032 is transmitted to the recipient.

  FIG. 13 is a process flow diagram illustrating an embodiment method 1300 for managing PTX communication between PTX devices using a server. Method 1300 relates to initially establishing or attempting to establish a communication session for a PTX group. At block 1310, the server may receive a request to issue a PTX session from the originating PTX device, as described above. At block 1320, the server may initiate PTX group verification, which includes verifying that all members of the PTX group are reachable and / or any provisioning necessary to establish PTX communication. Performing may be included. At block 1330, the server may determine whether the desired PTX group is ready to communicate. If the PTX group is confirmed (i.e., decision block 1330 = Yes), at block 1360, the server may send a call permission indication to the originating PTX device, and similarly, other members of the PTX group may be granted the right to call. An occupation instruction may be transmitted. If the call right is granted, the server may send an audio and / or visual feature indicating that to a member of the PTX group that includes the originating PTX device. As discussed above, these visual features may include an indication of a word or symbol representing this state. It will be appreciated that in addition to the visual feedback cues described herein, other feedback cues such as audio cues or haptic cues may be employed to indicate the status of group communication. If the PTX group is not verified (ie, decision block 1330 = No), the server may determine whether a portion of the desired PTX group is ready to communicate. If the partial PTX group is confirmed (i.e., decision block 1340 = Yes), at block 1370, the server may send a partial call permission indication to the originating PTX device, as well as the convenience of the PTX group. A transmission right occupied instruction may be transmitted to other members. Partial call permissions also include audio and / or visual features, and can be somewhat different from full call permissions, so that all group members are aware of who they are. After block 1380 indicates full call permission or partial call permission, the server sends a PTX such as one or more message segments from the originating PTX device and / or buffer requests from other members of the PTX group. May leave the group communication. If even the partial PTX group cannot be confirmed (ie, decision block 1340 = No), the server may send a call permission reject indicating that the call right cannot be granted.

  FIG. 14 is a process flow diagram illustrating an embodiment method 1400 for managing PTX communication between PTX devices using a server. Method 1400 relates to handling of PTX group communications, where the originating PTX device has been assigned outgoing permission or partial outgoing permission once it is not convenient for all PTX group members. At block 1410, the server may receive a PTX message segment. The server may determine at decision block 1420 whether a buffer request or buffer status has been received and whether the received is still active. The buffer request and / or buffer status may have been received from one or more of the PTX group recipient PTX devices. The buffer request represents an indication from the recipient PTX device that the PTX message segment needs to be stored for delayed playback. In contrast, the buffer status represents an indication that a PTX message segment is being stored, is about to be stored, and will be stored. Thus, if no buffer request or buffer status has been received (ie, decision block 1420 = No), the server may send the received PTX message segment to all PTX group devices at block 1422. Similarly, even if a buffer request has been received previously, the buffer request may no longer be active. For example, the recipient device may be ready to receive communications after first sending a buffer request, and may be able to catch up on the playback delay of previously buffered message segments (may be caught-up in terms of playing back). . The recipient device may then indicate this ready state to the playback delay, deactivating the previous buffer request at this point. The recipient device will then be included in the delivery of the PTX message segment at block 1422. If at least one buffer request has been received and remains active (ie, decision block 1420 = Yes), the server may determine at block 1424 whether all PTX group members are requesting buffering. If at least one member is not requesting buffering or not actively requesting buffering (also referred to as a “raw recipient”), the server at block 1425 The received PTX message segment may be sent to the raw recipient.

  Recipients must receive raw PTX message segments directly without significant delay if they do not require buffering, and buffered or delayed receivers with active buffer requests. Since there are two categories, the process flow may be separated from block 1425. In order to further manage the PTX communication for raw recipients, the process may return to block 1410 to receive additional PTX message segments for the group. Also, to manage PTX communications for delayed recipients, the process may send a buffer indication to the originating PTX device at block 1430 simultaneously or simultaneously. Alternatively, if it is determined that all group members are requesting buffering (i.e., decision block 1424 = Yes), the server similarly sends a buffer indication to the originating PTX device at block 1430. Good. The buffer indication may reflect the fact that the buffer request was received and from which recipient PTX device the buffer request was received. In conjunction with sending the buffer request, at block 1435, the PTX message segment may be stored by the server. The message segment may be stored locally in a server or a remote network memory storage element available to the server. Alternatively, the PTX message segment may be stored at either the originating PTX device or the device requesting buffering of the PTX group member, as further described below.

  Once the one or more PTX message segments are stored, the server may receive a ready signal at block 1440. The ready signal may indicate that the recipient PTX device that previously sent the buffer request is ready to output the PTX message segment. Alternatively, the server may not receive a ready signal from the recipient device prior to the end of the PTX communication session, in which case processing does not need to proceed to block 1440. However, if a ready signal is received, the server may determine at block 1450 whether an erasure signal has been received from the originating PTX device or may have determined in advance. The erasure signal represents the fact that the originating PTX device does not want delayed playback of the buffered one or more PTX message segments. If an erasure signal has been received (i.e., decision block 1450 = Yes), one or more PTX message segments may be erased at block 1454, or the server may erase the stored PTX message segments. An erase signal may be transmitted. At block 1470, the server may wait for further PTX group communications. For example, at block 1470 from block 1454, processing may return to block 1410 and be ready to receive additional PTX message segments, or alternatively (from returning to method 1300 above) from another PTX device. A call request may be received. However, if an erasure signal has not been received (i.e., decision block 1450 = No), at block 1452, the server stores in advance for one or more recipient PTX devices for which a ready signal was received at block 1440. Sent PTX message segments.

  In various embodiments, when one or more buffered (ie, stored) PTX message segments are being played on the recipient PTX device, the playback status is the server as well as the PTX device that originated the message. May be reported to In this manner, the server may determine, at decision block 1460, whether a playback state has been received from a member in the ready state that received the PTX message segment at block 1452. If a playback state has been received (ie, decision block 1460 = Yes), at block 1464, the playback state may be transmitted to the originating PTX device and optionally other members of the PTX group. Alternatively, if the playback state has not been received (ie, decision block 1460 = No), the server may wait for further PTX group communication at block 1470. As described above with respect to block 1470, the process may then return to block 1410 to be ready to receive additional PTX message segments, or alternatively to receive an origination request from another PTX device. Good.

  FIG. 15 is a process flow diagram illustrating an embodiment method 1500 for managing PTX communication between PTX devices from the perspective of a source PTX device. At block 1510, the originating PTX device may initiate a call origination request, similar to the latest PTX method. If the call permission is not received (i.e., decision block 1520 = No) and the call right occupancy indication is not received (indicating that someone else has the call right), then the originating PTX device receives the call right at block 1510. The request may be triggered again. Once a call permission is received (ie, decision block 1520 = Yes), the originating PTX device may send one or more PTX message segments at block 1530. In one embodiment, the buffered message segments may be stored locally at the originating PTX device. Thus, to achieve this alternative, a PTX message segment is stored at block 1535. If no buffer request has been received, PTX communication may proceed in the same way as in modern PTX systems. However, according to various embodiments, a buffer request may be received at block 1540. Note that any device in the PTX group that does not wish to buffer may simply receive the transmitted PTX message segment as a live broadcast. The originating PTX device may also be notified as to which device is receiving the message segment live. Alternatively, the originating PTX device may assume that any device that did not request buffering has received the message segment unless otherwise notified by the server.

  In one embodiment, the buffer request may be rejected (i.e., decision block 1550 = Yes), which indicates that the originating sender does not want the recipient to play the buffered message segment. That means. Therefore, an erasure signal may be sent to the server to prevent such playback. When the originating PTX device stores the message locally at block 1535, the stored message segment may then be erased, and a notification about the deletion is sent to the server at block 1554. At block 1580, the originating PTX device may abandon the call and wait for PTX group communication from others, or return to block 1530 and continue to send one or more PTX message segments.

  If the buffer request is not rejected (ie, decision block 1550 = No), the recipient PTX device may play a previously stored PTX message segment once ready. Thus, at block 1552, the originating PTX device receives a ready signal indicating that one or more recipient PTX devices are ready to output one or more stored PTX message segments. It's okay. Once the message segment is stored locally at block 1535, the stored message segment may be sent again at block 1555. In this way, the delayed recipient PTX device may play back one or more missed message segments.

  In one embodiment, playback status may be reported to the originating PTX device and / or a member of the PTX group. Thus, when a delayed recipient PTX device plays a previously stored PTX message segment, a playback state can be provided to the originating PTX device. If a playback status is received (ie, decision block 1560 = Yes), at block 1570, the playback status may be displayed on the originating PTX device. As discussed above, the playback state may be a simple progress bar, an audible display, or a speech-to-text conversion of at least a portion of the played PTX message segment. Also, the voice-to-text conversion displayed on the originating PTX device reflects the updated playback state of the portion of the first PTX message segment that has not been output from the first recipient PTX device. Also good. In one embodiment, as described above with respect to block 1554, the originating PTX device may interrupt or terminate playback by sending an erasure signal. Regardless, once playback is terminated or terminated, at block 1580, the originating PTX device either abandons the call and waits for PTX group communication from others, or returns to block 1530 One or more PTX message segments may continue to be transmitted.

  FIG. 16 is a process flow diagram illustrating an embodiment method 1600 for managing PTX communication between PTX devices from the perspective of a recipient PTX device. At block 1610, the recipient PTX device may receive an outgoing right occupied signal. The originating right occupied signal may indicate that the originating PTX device is authorized to at least temporarily control the main communication of the PTX session. Such control may include being the only one allowed to speak during the control period. Alternatively, such control may include transmission of video or other data.

  In an alternative embodiment, the recipient PTX device may attempt to “know” the ambient noise level to compensate for the ambient noise level. The recipient PTX device may be configured to detect a noise level around the device in response to receiving the outgoing right occupied signal, or even the actual PTX message segment. In addition, it may be determined whether the ambient noise level is below a quiet threshold. If the ambient noise level is less than the quiet threshold, the first adjusted output volume may be set. The first adjusted output volume may include a volume level that is lower than a preselected volume level of the device. Alternatively, the first adjusted output volume may include the output of the PTX message segment through the earphone paired with the device. As a further alternative, the device may be configured to use a screen text display and / or a preselected “silence mode” setting. In contrast, when a device detects a “noisy” environment, such as a concert or a noisy crowd (i.e., a relatively high level of ambient noise), the device It may be configured to further determine whether the threshold is exceeded. If the ambient noise level exceeds the noisy threshold, a second adjusted output volume may be set. The second adjusted output volume may include a volume level that is higher than a preselected volume level of the device. In addition or alternatively, the second adjusted output volume is output via a PTX message through the earphone paired with the device, a text display on the screen, a pre-selected “loud mode” for the device The use of settings may be included and / or a buffer request may be sent. Other device behaviors can be used to enhance or enhance the experience, such as reproducing the sound at an appropriate volume, or using vibration when it seems inconvenient to produce a sound. . Accordingly, at block 1615, the ambient noise level may be measured, or the user may respond to the input message by entering a buffer request, or a predetermined setting for the device automatically responds to such a request. It may be generated. As a further alternative, processing may skip from block 1615 to block 1645 without requiring buffering, in which case only the PTX message segment output will be adjusted.

  At block 1620, it may be determined whether message buffering should be initiated from the intended recipient PTX device. Such buffering may be activated after receipt of an outgoing occupancy signal, after receipt of a PTX message segment, or after other options of block 1615 relating to ambient noise measurements or received user input. If it is determined that buffering should be activated (ie, decision block 1620 = Yes), at block 1625, a buffer request and / or buffer status may be sent to the server. When the buffer status is transmitted, the received PTX message segment may be stored in local memory. In this way, the buffer status reflects the storage condition of the PTX message segment or confirms the storage of the PTX message segment. After a buffer request or buffer status has been sent, or if the buffer request has not been initiated (ie, decision block 1620 = No), it may be determined whether a PTX message segment has been received by the recipient PTX device. If no PTX message segment has been received (ie, decision block 1630 = No), this can be interpreted as an acknowledgment of the buffer request. Accordingly, at block 1652, the recipient PTX device may wait for further PTX communication or may initiate its own PTX communication, such as by sending a ready signal as discussed below.

  When a PTX message segment is received by the recipient PTX device (i.e., decision block 1630 = Yes), this means that there was no buffer request and there is no currently active buffer request from the intended recipient PTX device (i.e. May mean that the previous buffer request was canceled to become inactive now), or possibly the buffer request was rejected / invalidated. For example, receiving a PTX message segment after a buffer request or buffer status was first indicated may mean that the buffer request has been rejected or invalidated in any case. Thus, if the buffer request is rejected, either no buffer request has been created or there is no active buffer request (i.e., decision block 1640 = Yes), and at block 1660 the received PTX message segment is Can be output at the recipient PTX device. Alternatively, the output of the recipient PTX device, such as the output volume or method, may be adjusted at block 1645 as described above with respect to ambient noise detection. However, if the PTX message segment is output at block 1660 without substantial delay from when it was buffered or transmitted, the output of the recipient PTX device is considered “raw”. Accordingly, at block 1670, a raw output state may be transmitted from the recipient device indicating that the recipient device is a raw participant in the PTX session.

  In one embodiment, when a PTX message segment is received by the recipient PTX device (i.e., decision block 1630 = Yes), this is instead that the PTX message segment is stored locally on the recipient PTX device. Can mean. Thus, if the created buffer request is active and not rejected (ie, decision block 1640 = No), this may mean that the PTX message segment is stored locally. If the recipient PTX device is not able to store the message segment locally (i.e., decision block 1650 = No), at block 1652, the recipient PTX device waits for further PTX communication or has its own PTX communication May be activated. For example, the recipient PTX device may transmit a signal indicating at block 1652 that the PTX message segment cannot be stored locally. If the recipient PTX device can store the message segment locally (i.e., decision block 1650 = Yes), at block 1654, the PTX message segment may be stored in the storage of the recipient PTX device, and then Proceeding to block 1652, a further PTX communication may be awaited or its own PTX communication may be initiated.

  FIG. 17 is a process flow diagram illustrating an embodiment method 1700 for managing PTX communication between PTX devices from the perspective of the recipient PTX device once the recipient PTX device is ready to receive the message segment. is there. For example, the noise level around the recipient PTX device may have changed, and once that level is appropriate, the recipient device may be ready to play or receive a PTX message segment . Thus, it may be determined whether the ambient noise level is above / below a silence threshold or a noise threshold. Alternatively, the user of the recipient device may manually enter a ready status indication for receiving the message. Accordingly, at block 1710, the recipient PTX device may send a ready signal to the server indicating that the recipient PTX device is ready to output the PTX message segment. Thereafter, one or more message segments of the originating PTX device may be played on the recipient PTX device. Playback may be activated prior to the originating PTX device stopping sending raw messages.

  If one or more PTX message segments are stored locally on the recipient PTX device (i.e. decision block 1720 = Yes), an erasure signal is received from the originating PTX device before playing the PTX message segment. It may be judged whether or not. The receipt of the erasure signal is an indication that the originating PTX device does not want the stored message segment or segments to be played back. If an erasure signal has been received (ie, decision block 1730 = Yes), then at block 1735, the locally stored PTX message segment may be erased. If one or more PTX message segments are not stored locally (ie, decision block 1720 = No), it may be determined whether the PTX message segment is stored remotely. For example, one or more PTX message segments may be stored on the originating PTX device, server or some other resource. If one or more PTX message segments are not stored remotely (i.e., decision block 1740 = No), at block 1742, the recipient PTX device waits for further PTX communication or initiates its own PTX communication You can do it. If one or more PTX message segments are stored remotely (ie, decision block 1740 = Yes), it is determined whether a stored PTX message segment has been received. If no message has been received from the remote storage (ie, decision block 1744 = No), the process may proceed to block 1742 to wait for further PTX communication or to activate its own PTX communication. If one or more message segments have been received (ie, decision block 1744 = Yes), at block 1750, the PTX message segment may be output at the recipient PTX device. Similarly, if the PTX message segment is stored locally and no erasure signal has been received (ie, decision block 1730 = No), at block 1750, the PTX message segment may be output at the recipient PTX device.

  In one embodiment, as described above, the playback status of buffered message segments may be reported to the server, the originating PTX device, and optionally other members of the PTX group. Accordingly, at block 1750, in response to the output of the PTX message segment, the playback status may be transmitted from the outputting recipient PTX device to at least the server. If the output of the PTX message segment has not ended (ie, decision block 1770 = No), the recipient PTX device may continue to output the PTX message segment at block 1750 and report the playback status at block 1760. Once the output of the PTX message segment is complete (ie, decision block 1770 = Yes), a playback completion signal may be sent at block 1775. Thereafter, processing may proceed to block 1742, waiting for further PTX communications or initiating its own PTX communications.

  Various embodiments are directed to extending PTX communications as compared to traditional PTX services, systems and hardware. It should be understood that the current embodiment may be implemented in software such that the user does not have to “press” a physical button to speak. In various embodiments, the user interface allows the user to activate an icon or to make easier multitasking and using the device's speakerphone mode and / or earphone mode in a more natural way. An interaction with the PTX network may be possible with the hands free mode enabled.

  Various embodiments can provide a more visual experience than previously employed and can enhance existing voice cues for users to control and arbitrate calling rights Or a visual feedback cue that may be replaced. The visual feedback cue may be provided by graphic elements and colors on the display screen of the wireless device. In various embodiments, the feedback queue may indicate the status of the group communication session and the personal message segment, for example, may indicate outgoing rights control for group communication. In various embodiments, the calling right is open, the calling right is requested by the user's device, the calling right is controlled by the user's device or another device, and buffering is required. Visual features may be displayed on the display screen that informs the user that the message is being buffered, buffered, and the playback status of the buffered message segment. In some embodiments, the display screen may also display a visual display of group members in the group communication and may further display one or more of the participation status of each member of the group. For example, the device may display the members of the group currently participating in the group's communication session, as well as members of the group delayed who may play the message segment late. Software that can implement these enhancements is executed on the user's wireless device and may be partially implemented on the server.

  FIG. 18 shows a call flow during PTX communication including video. Source device 11 is shown requesting a state that reflects the state of recipient device 12. The recipient device 12 may then send a status indication. The status indication may indicate that the recipient device is ready, requires buffering, or is simply unavailable. The status indication transmitted from the recipient device 12 to the source device 11 may be displayed on the source device 11 (ie, on the UI display). As shown in Figure 18, the "Buffering" status display is shown, which reflects that the video message sent from the originating device 11 is stored in the memory buffer It is. Similarly, recipient device 12 may display a status indicating “Incoming Video Buffering” to inform the recipient user that the video message is stored. The source device may choose not to point the source device's camera at the otherwise desired image if no one is watching, or choose to turn off the broadcast video completely. May be.

  FIG. 19 shows the recipient device 12 in the process of playing back video in the memory buffer. The recipient device may choose to play the transmitted communication when ready, may convert the acoustic message to a text message, or acoustic to catch up with the live broadcast from the originating device. / You can fast-forward the video message. Once again, the originating device is presented with a status indication reflecting that the recipient is outputting a buffered message segment and is currently 30 seconds behind the live broadcast that is being buffered. The source device 11 can also play the video communication generated by the source device 11 to delete the segment or to select a key frame from the video for viewing by the recipient device. FIG. 19 shows that the originating user may use a “Key Frame (s)” button to specify a key frame from the transmission of the video, and the recipient user may use the key frame instead of the entire video. It is further shown that you can choose to browse.

  In the embodiment shown in FIGS. 20A-20E, the output from the PTX device can be controlled using a manual function. FIG. 20A shows the recipient device UI receiving the incoming video. FIG. 20B shows how the user can apply a swipe perpendicular to the UI to convert the acoustic component of the communication into text. In this aspect, the converted story is displayed on the UI using a speech-to-text recognition algorithm. Similarly, exercise activated activation can be used as a switch. For example, if the PTX device is shaken or moved sufficiently, the sensor may detect this and switch the operating mode. This aspect includes an automatic function, but input may be received by a user action (ie, user input) such as shaking the device. FIG. 20C illustrates that the user of the recipient device can apply a vertical swipe in the opposite direction to switch back to the acoustic mode. FIGS. 20D and 20E illustrate how the user can select acceleration / deceleration of video or audio transmission by making another selection on the UI. FIG. 20D shows how a horizontal swipe accelerates the transmission of audio and / or video. FIG. 20E shows that another horizontal swipe in the opposite direction slows down the transmission of audio and / or video. Alternatively, an input may be provided from a user or sensor that allows an audio only mode or a video only mode. These inputs may be made by direct contact with the PTX device by the user or by gesture recognition. Gesture recognition can be performed by a PTX device that can be equipped with a camera and software to detect and analyze a user's physical motion or device orientation. For example, a user swiping a hand in the vertical direction can represent one action, and swiping a hand in the horizontal direction can represent another action. Alternatively, the orientation of the recipient device may be employed, such as facing the right side of the recipient device down / up. Additional gesture controls may be provided to allow the user to change where to view / listen to audio / video supplements. For example, a cell phone with a wrist-worn paired device with a screen may be used as an alternative output for that message segment or portion.

  As a further alternative, the recipient device may use a sensor to determine the user's situation and select an appropriate mode. Also, the mode may be switched when the user's environment changes, such as when the bright sun is hidden in the clouds in an outdoor environment or when the noise level around the recipient device changes. In the embodiment shown in FIGS. 21A-21E, the output from the PTX device can be controlled using an automatic function. FIG. 21A shows a recipient device UI receiving incoming video. FIG. 21B is similar to that of FIG. 20A, and a sensor such as a microphone detects that the noise around the recipient device is too loud and switches to a text output mode that converts the acoustic component of the communication into text. It shows that. This aspect may also use a speech to text recognition algorithm to display the converted story on the UI. FIG. 21C shows how a further sensor, such as an optical sensor, is used to detect ambient brightness. In this way, the device may detect that the environment is too bright and the display may not be visible and may switch to the acoustic mode. Such switching can also save energy for the device. FIG. 21D and FIG. 21E show how the automatic detection mode can accelerate / decelerate the transmission of sound or video. FIG. 21D shows an example of how the microphone may detect a pause in the user's voice so that the recipient device automatically accelerates the transmission of sound and / or video. Alternatively, the recipient device may automatically accelerate to catch up with the live broadcast if the delay is too great, such as after a predetermined period of time from the live broadcast. FIG. 21E illustrates how the transmission of audio and / or video is decelerated when the device detects a delay in the communication stream.

  FIG. 22 shows how the user 5 of the originating device can supplement an active PTX group communication session from an audio-only session in order to add video to the session. The user can turn on / off the video transmission option by pressing the “add video” icon 153 that acts like a button. FIG. 22 shows video transmission in the off mode. FIG. 23 shows how the UI is switched when the user 5 once presses the “add video” icon 153. Specifically, a video display window 154 appears and shows a video image acquired by a camera on the transmission source device. In FIG. 23, an “add video” icon 153 is shown with a cross, indicating that when the icon is pressed again, the device switches to the video transmission off mode. In this example, it is assumed that the camera of the device is facing the user, but if a camera facing away from the user is enabled, a “switch on camera” button may also be provided. Alternatively, the camera switching may be achieved by a voice operation command from the user. In addition, this further alternative aspect may allow other group members to activate camera switching from their side. For example, if the originating device uses a back camera (also known as a “see what I see” session) and a group member wants to see the originating device user face, during the session The group member may be allowed to switch to the source device's camera. Such a remote camera switching function may also be activated by the UI for the voice command function, if enabled. Switching to a camera facing away from the user may be useful in this scenario, such as a construction site, where the user wants to show others the situation or problem or progress. Any face that is clear in the display with the user facing or facing the active camera can be used with image recognition and facial expression recognition algorithms. In this way, objects or objects that can be recognized by the image recognition algorithm can be communicated to the PTX group. Also, facial expression recognition algorithms can be used to convert facial expressions into text or emoticons that can be communicated to PTX groups as well. In addition, more sophisticated image recognition algorithms can be used to convert the gesture language represented by the user into a text message or an acoustic message. Such conversion and / or translation may be performed from at least one still image or video image.

  FIG. 24 shows a screenshot of a PTX device with a touch screen display, according to a further embodiment, where the originating device or recipient device receives several input streams (including audio, video and / or data). Can be received simultaneously. Accordingly, the plurality of input stream display portions 2441, 2442, 2443 that the UI shown in FIG. 24 includes below the group member display portion represents each received input stream. In this embodiment, the PTX device is receiving input streams from three PTX group members 132, 134, 135. In an illustrative example, the first display portion 2441 includes a video segment 2431 included as part of the input stream received from the PTX group member Mark L. 135. The device shown may output an audio segment corresponding to a video segment corresponding to actual audio from member 135 in this environment. The UI may include optional buttons such as a voice to text conversion function 2451 (such as a button labeled “T”) or a video off function 2461 (such as a button with a video icon). The first display portion 2441 indicates that the voice-to-text conversion function 2451 is in the off mode and the video switch 2461 is in the on mode. Second display portion 2442 also includes a video segment 2432 from PTX group member Kameron 132. However, the acoustic segment of Kameron's speech has been switched to speech-to-text conversion mode. Thus, the words spoken by Kameron are displayed in the caption area 2433 below the corresponding video segment 2432. Also, a voice-to-text conversion function 2453 is shown in the on mode. Additional options included in this example include motion-to-text conversion, which employs motion recognition and is recognized as Kameron waving. The transformation from motion to text produces an interpretation of the analyzed image or images. The interpreted motion can be further processed by a text-to-speech algorithm and then output as text (referred to as “text interpretation”) or spoken language (referred to as “acoustic interpretation”). In general, motion-to-text conversion recognizes body part movements and facial gestures when they are aligned with preprogrammed recognizable movements. The converted action-to-text output is an interpretation of the action and is output either by text or by a verbal description of the text output form of the text. Therefore, the caption area 2433 is reinforced with a gesture instruction 2434 indicating “waving an arm”.

  The third display portion 2443 in FIG. 24 does not include a video segment, reflecting that the video on / off switch 2465 is disabled. Nevertheless, the acoustic segment from user Jonah 134 is being output from the PTX device simultaneously with the acoustic supplement from user Mark L. 135. In this way, the user of the PTX device can select and view / listen to one of the valid video or audio, rather than watching and listening to all video and audio segments simultaneously. In this way, the recipient device may display a video stream that includes audio output from one source device, and may selectively select a video stream from another source device that has no audio output. May be displayed.

  FIG. 25 illustrates multiple simultaneous communication streams being received by a single user device 102. The first source device 2504 includes a microphone and camera that can capture the audio and video images of the first user 2502. Both audio and video images may be sent to server 254 over communication line 2506. Similarly, a second source device 2512, which also includes a microphone and camera, can capture and transmit the voice and video images of the second user 2510. Audio / video communications from the second source device 2512 may be sent to the server 254 through a further communication line 2514. In this manner, various message segments from the first source device 2504 and the second source device 2512 may be communicated to the single user device 102 by the streams 2520, 2522. Thus, the first display portion 2541 of the user device may output the communication stream 2520 from the first source device 2504 and the second display portion 2542 may output the communication stream 2522 from the second source device 2512. You may output. In an illustrative example, the acoustic component of the first communication stream 2520 is turned off, as indicated by the silence symbol 2531 in the upper left corner. However, the video component 2530 may be displayed on the first display portion 2541 while being active. The caption 2532 may also be displayed with the sound output turned off and the speech to text recognition function activated. Also, as indicated by the speaker symbol 2533, while the acoustic component of the second communication stream 2522 is active, the video supplement of that communication is shown to be erased. This embodiment shows an example of image recognition in which the algorithm recognizes that the second user 2510 is waving his arm. This recognition is reflected in the instruction of the second display portion 2542 indicating that “Kameron is waving his arm”. Gestures conveyed by body part movements can generally be recognized in this way. Further extensions to this functionality may even include language conversion in the form of audio components, translated into the desired language, of subtitle text or video dubbing. Still further, motion-to-text conversion functionality may be provided to convert the physical motion of the originating user in the video portion of the message segment into text or audio output.

  These embodiments may be implemented in various mobile wireless communication devices, particularly mobile computing devices. An example of a wireless communication device that implements these various embodiments is the smartphone 2600 shown in FIG. A wireless communication device such as a smartphone 2600 may include a processor 2601 coupled to a memory 2602 and a radio frequency data modem 2605. Modem 2605 may be coupled to an antenna 2604 for receiving and transmitting radio frequency signals. Smartphone 2600 may also include a display 2603 such as a touch screen display. Mobile device 2600 may also include a user input device such as a button 2606 for receiving user input or a microphone 2610 for measuring ambient noise levels or receiving audio. In various embodiments, the haptic output surface that the smartphone 2600 includes may be disposed on the display 2603 (e.g., using E-Sense ™ technology), may be disposed on the back surface 2612, or the mobile device It may be placed on another surface of 2600. The mobile device processor 2601 may be any programmable micro that may be configured with instructions (i.e., applications, more generally software) to perform various functions, including the functions of the various embodiments described herein. It may be a processor, a microcomputer or a plurality of processor chips. In general, software applications may be stored in internal memory 2602 and then accessed and loaded into processor 2601. In some mobile computing devices, additional memory chips (eg, SD (Secure Digital) cards) may be plugged into the mobile device and coupled to the processor 2601. Internal memory 2602 may be volatile memory or non-volatile memory such as flash memory, or a combination of both. For purposes of this description, a general reference to memory refers to all memory accessible by the processor 2601, including internal memory 2602, removable memory connected to the mobile device, and the memory of the processor 2601 itself. Is.

  The various embodiments described above may be implemented on any of a variety of commercially available server devices, such as server 2700 shown in FIG. Such a server 2700 generally includes a processor 2701 coupled to volatile memory 2702 and a large capacity non-volatile storage device such as a disk drive 2703. Server 2700 may also include a floppy disk drive, compact disk (CD) drive or DVD disk drive 2706 coupled to processor 2701. Server 2700 may also include a network access port 2704 coupled to processor 2701, such as a local area network coupled to other broadcast system computers and servers to establish a data connection with network 2707. The processors 2601, 2701 may be any programmable microprocessor, microcomputer or multiple processor chips that may be configured with instructions (applications or software) to perform various functions, including those of the various embodiments described above. Some devices may be provided with multiple processors 2601, 2701, such as one processor dedicated to wireless communication functions and one processor dedicated to running other applications. In general, software applications may be stored in internal memory 2602, 2702, and 2703 and then accessed and loaded into processors 2601, 2701.

  The processors 2601 and 2701 may include sufficient internal memory to store application instructions. In many devices, the internal memory may be volatile memory or non-volatile memory such as flash memory, or a combination of both. For the purposes of this description, a general reference to memory refers to memory accessible by the processor 2601, 2701, including internal memory or removable memory connected to the device and the memory of the processor 2601, 2701 itself. It is.

  The foregoing method descriptions and process flow diagrams are provided merely as illustrative examples so that the steps of the various embodiments may be required or implied to be performed in the order shown. Not intended. Those skilled in the art will appreciate that the order of the steps of the foregoing embodiments can be performed in any order. The words “after”, “next”, “next”, etc. are not intended to limit the order of the steps, and these words are only used to guide the reader through the description of the method. ing. In addition, any reference to an element in a claim using the singular article such as “a”, “an”, or “the” limits the element to the singular. Should not be interpreted as.

  The various illustrative logic blocks, modules, circuits, and algorithm steps described in connection with the aspects disclosed herein may be implemented as electronic hardware, computer software, or a combination of both. To clearly illustrate this interchangeability of hardware and software, various illustrative components, blocks, modules, circuits, and steps have been described above generally in terms of their functionality. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the overall system. Those skilled in the art can implement the described functions for each particular application in a variety of ways, but such implementation decisions should not be construed to depart from the scope of the present invention. .

  The hardware used to implement the various exemplary logic, logic blocks, modules, and circuits described in connection with the aspects disclosed herein includes general purpose processors, digital signal processors (DSPs), Application-specific integrated circuits (ASICs), field programmable gate arrays (FPGAs) or other programmable logic devices designed to perform the functions described herein, discrete gate or transistor logic, discrete hardware It may be implemented or performed using components, or any combination thereof. A general purpose processor may be a microprocessor, but in the alternative, the processor may be any conventional processor, controller, microcontroller, or state machine. The processor may also be implemented as a combination of computing devices, for example, a combination of DSP and microprocessor, multiple microprocessors, one or more microprocessors associated with a DSP core, or any other such configuration. it can. Alternatively, some steps or methods may be performed by circuitry that is limited to a given function.

  In one or more exemplary aspects, the functions described may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functions may be stored on a non-transitory computer-readable storage medium as one or more instructions or code. The steps of the methods or algorithms disclosed herein may be present in or stored on a non-transitory computer readable medium or processor readable medium and may be embodied in a processor executable software module. Non-transitory computer readable storage media and processor readable storage media may be any available media that can be accessed by a computer or processor. By way of example, and not limitation, such non-transitory computer readable media can be RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage device, or in the form of instructions or data structures Any other medium that can be used to store the desired program code and that can be accessed by a computer. Discs and discs used in this specification are compact discs (CDs), laser discs (discs), optical discs (discs), digital versatile discs (DVDs), floppy discs (discs). Including a registered trademark disk and a Blu-ray disc, the disk normally reproducing data magnetically, and the disk optically reproducing data with a laser. Combinations of the above should also be included within the scope of non-transitory computer readable media. In addition, the operations of the method or algorithm may exist as one or any combination or set of non-transitory processor readable media and / or code and / or instructions stored on a computer readable media that may be incorporated into a computer program product. You can do it.

  The previous description of the disclosed aspects is provided to enable any person skilled in the art to make or use the present invention. Various modifications to these aspects should be readily apparent to those skilled in the art, and the generic principles defined herein may be changed to other aspects without departing from the spirit or scope of the invention. Can be applied. Accordingly, the present invention is not intended to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein. It has become.

5 users
100 PTX communication group
102 First PTX device
104 Second PTX device
106 Third PTX device
112 display
114 display
115 Control unit
116 display
120 Session name field
122 Session Role Field
123 Caption
125 Status icon
128 Participant add-on icons
131 First recipient PTX participant
132 Second recipient PTX participant
133 Third recipient PTX participant
134 4th recipient PTX participant
135 Source PTX device user
142 Session status indicator
151 Playback status
152 Playback status
161 Numbered label field
162 Played Time Field
165 progress bar
166 Extended playback status display
168 Unplayed time field
170 Select button
180 processor platform
181 storage device
183 Application Programming Interface (API)
185 processor
187 Direct communication link
189 Local database
System for managing 200 PTX communication
201 wireless communication devices
203 wireless data network
204 servers
205 router
206 Database
1002 Caller
1004 group communication server
1006 Memory buffer
1008 Recipient 1
1010 Recipient N
1020 Call right request
1023 Call permission
1024 Calling rights occupied
1028 Buffer request
1032 First communication segment
1043 Buffer instruction
1063 Ready signal
1070 Second communication segment
1073 Communication 1 status
1102 Caller
1104 Group communication server
1106 Memory buffer
1108 Recipient 1
1110 Recipient N
1122 Request to send
1123 Call permission
1124 Calling rights occupied
1128 Buffer request
1132 First communication segment
1143 Buffer instructions
1152 Second communication segment
1163 Ready signal
1173 Communication 1 status
1183 Communication 2 status
1202 Caller
1204 Group communication server
1206 Memory buffer
1208 Recipient 1
1210 Recipient N
1222 Request to send
1223 Call permission
1224
1228 Buffer request
1232 First communication segment
1243 Buffer instruction
1252 Second communications segment
1263 Ready signal
1272 Erase signal for first communication segment
2600 smartphone
2601 processor
2602 memory
2603 touchscreen display
2605 modem
2606 button
2610 microphone
2612 reverse side
2700 server
2701 processor
2702 internal memory
2703 internal memory
2704 Network access port
2706 floppy disk drive, compact disk (CD) drive or DVD disk drive
2707 Network

Claims (15)

A method for managing electronic communication on a communication device, comprising:
Receiving an incoming call instruction from a source device at the communication device;
Determining whether delayed call admission is guaranteed;
Transmitting a delay indication from the communication device to the source device in response to determining that the delayed call authorization is guaranteed;
Transmitting from the communication device a ready signal indicating that the communication device is ready to receive a first message segment;
In the communication device, in response to the ready signal, receiving the first message segment and subsequently receiving a second message segment from the source device;
Determining at the communication device whether to output at least one of the first message segment and the second message segment;
In response to determining whether to output at least one of the first message segment and the second message segment, at least one of the first message segment and the second message segment Outputting from the communication device;
Transmitting a playback status from the communication device to the source device, wherein the playback status includes a series of status indications, each status indication being an output of the first message segment from the communication device Reflecting the progress of the updated output, wherein the playback state is transmitted in response to determining that the first message segment should be output. In the communication device, receiving from the source device a playback interruption that stops playback of the first message segment, or
The communication device stops playback of the first message segment in response to receiving the second message segment prior to the end of output of the first message segment, and the second message The method of claim 1, further comprising: outputting a segment. Outputting at least one of the first message segment and the second message segment;
A speech-to-text conversion of spoken language contained in at least one of the first message segment and the second message segment;
Language translation of spoken language contained in at least one of the first message segment and the second message segment;
A motion-to-text conversion that interprets body part motion detected in at least one of the first message segment and the second message segment, or
At least one of accelerating, decelerating, pausing, and skipping a portion of at least one of the first message segment and the second message segment;
The method of claim 1, comprising at least one of: The communication device is
An audio portion and a video portion that simultaneously output a plurality of message segments, each from a different source device, wherein at least one of the plurality of message segments is converted into at least one of text and verbal explanation At least one of, or
The communication device is a video and audio stream from a first source device;
A video stream from a second source device, and the video stream from the second source device is converted from audio to text in the audio portion of the message segment received from the second source device. Highlighted with a subtitle representing the transformation,
The method of claim 1. A communication device,
Means for receiving an incoming call instruction from a source device at the communication device;
Means for determining whether delayed communication is guaranteed at the communication device;
Means for transmitting a delay indication to the source device in response to determining from the communication device that the delayed call authorization is guaranteed;
Means for transmitting from the communication device a ready signal indicating that the communication device is ready to receive a first message segment;
Means for receiving at the communication device the first message segment from the originating device followed by a second message segment;
Means for determining at the communication device whether to output at least one of the first message segment and the second message segment;
In response to determining at the communication device whether to output at least one of the first message segment and the second message segment, the first message segment and the second message segment Means for outputting at least one of them,
In response to a determination from the communication device that the first message segment should be output, a series of status indications reflecting the progress of the updated output of the first message segment at the communication device. A communication device comprising: means for transmitting the playback state to the source device.   A non-transitory computer-readable storage medium storing processor-executable instructions configured to cause a processor of a communication device to perform the method of any one of claims 1 to 4. A method of managing electronic communications on a server,
Receiving a delay indication associated with a communication device at the server in response to the server sending a call initiation signal;
Transmitting the delay instruction from the server to a source device;
Receiving a first message segment from the source device at the server;
Sending from the server the first message segment to memory for delayed output;
Receiving from the communication device a ready signal indicating that the communication device is ready to receive a first message segment;
Transmitting the ready signal to a memory to transmit the first message segment to the communication device;
Receiving at the server a playback state including a series of state indications reflecting the progress of the updated output of the first message segment at the communication device;
Transmitting the playback state to the source device;
Including a method. Receiving a playback interruption from the originating device and transmitting the playback interruption to the communication device to stop playback of the first message segment, or
Prior to termination of output of the first message segment, receiving a second message segment from the source device at the communication device;
The method of claim 7, further comprising: transmitting the second message segment to the communication device to stop outputting the first message segment at the communication device. A server in a communication system,
Means for receiving at the server a delay indication associated with a communication device in response to the server sending a call initiation signal;
Means for transmitting the delay indication from the server to a source device;
Means for receiving a first message segment from the source device at the server;
Means for transmitting from the server the first message segment to memory for delayed output;
Means for receiving from said communication device a ready signal indicating that said communication device is ready to receive a first message segment;
Means for transmitting the ready signal to a memory to transmit the first message segment to the communication device;
Means for receiving at the server a playback state including a series of state indications reflecting the progress of the updated output of the first message segment at the communication device;
Means for transmitting the playback status from the server to the source device;
Comprising a server.   A non-transitory computer-readable storage medium storing processor-executable instructions configured to cause a server processor to perform the method of claim 7 or 8. A method of managing electronic communication on a source device,
Receiving a delay indication associated with a communication device at the source device in response to the source device sending a call initiation signal;
Outputting the delay indication at the source device;
Transmitting the first message segment from the source device for storage in memory;
Receiving at the source device a ready signal indicating that the communication device is ready to receive a first message segment;
Receiving a playback state indicating the progress of output of the first message segment retrieved from the memory at the source device and at least partially receiving output from the communication device, the playback state being a series of Each status indication reflects the progress of the updated output of the output of the first message segment from the communication device; and
Outputting the playback state at the source device;
Including a method. Sending a playback interruption to stop playback of the first message segment at the source device; or
12. The method of claim 11, further comprising: transmitting a second message segment for stopping playback of the first message segment at the source device. Receiving user input designating a key frame from a video image included in the first message segment;
Transmitting from the source device the user input including a designated key frame;
The method of claim 11 further comprising: The originating device,
Means for receiving at the source device a delay indication associated with a communication device in response to the source device sending a call initiation signal;
Means for outputting the delay indication at the source device;
Means for transmitting the first message segment from the source device for storage in memory;
Means for receiving at the source device a ready signal indicating that the communication device is ready to receive a first message segment;
Means for receiving at the source device a playback state indicating progress of output of the first message segment retrieved from the memory and at least partially receiving output from the communication device, the playback state Comprises a series of status indications, each status indication reflecting an updated output progress of the output of the first message segment from the communication device;
Means for outputting the playback state at the source device;
A source device comprising:   A non-transitory computer-readable storage medium storing processor-executable instructions configured to cause a processor of a communication device to perform the operations of the method of any one of claims 11-13.