WO2011029080A2

WO2011029080A2 - Communicating in a computer environment

Info

Publication number: WO2011029080A2
Application number: PCT/US2010/047929
Authority: WO
Inventors: Gregory Dorso; Uri Raz; Eric Setton
Original assignee: Sgiggle Incorporated
Priority date: 2009-09-06
Filing date: 2010-09-05
Publication date: 2011-03-10
Also published as: WO2011029080A3

Abstract

Communicating in a peer-to-peer computer environment. A tracker peer is contacted and a list of peer nodes are obtained at a user device. A subset of peer nodes are selected from the list of peer nodes to act as relay nodes for a communication. An appropriate compression scheme is selected for the communication at the user device. Data is transmitted to at least one relay node for the at least one relay node to forward to at least one other user device, wherein the relay node is one of the subset of the peer nodes. Data is received at the user device from the at least one relay node forwarded from the at least one other user.

Description

COMMUNICATING IN A COMPUTER ENVIRONMENT

RELATED APPLICATIONS

[001] This application claims priority to the copending U.S. Provisional Patent Application 61/240,223, Attorney Docket No. SGIG-001.PRO, entitled "System for Multi-Way Communication" with the filing date of September 6, 2009, which is herein incorporated by reference in its entirety, and which is hereto attached as Appendix A. In addition, this application claims priority to the pending U.S. Non- Provisional Patent Applications 12/876,153, Attorney Docket No. SGIG-005, entitled "Communicating with a User Device" with the filing date of September 5, 2010, U.S. Non-Provisional Patent Applications 12/876,154, Attorney Docket No. SGIG-006, entitled "Communicating with a User Device in a Computer

Environment" with the filing date of September 5, 2010, and U.S. Non-Provisional Patent Application 12/876,152, Attorney Docket No. SGIG-001, entitled

"Communicating in a Computer Environment" with the filing date of September 5, 2010, which are all herein incorporated by reference in their entirety.

BACKGROUND

[002] Modern technology provides for a variety of communication means and methods. For example, there are many types of communications including voice calls, video calls, text messages, photo messages, etc. Additionally, communication may be one-to-one or multi-party communications. Typically multi-party communications are accomplished by a plurality of devices communicating with a central infrastructure. For example, a central infrastructure may be a central computer server or a group of computer servers maintained in a server farm. A central infrastructure may be expensive to establish, maintain and operate.

[003] Many advances have been made in mobile handheld technology.

Specifically smart phones have increasingly more features. There is an increased demand to develop the functionality of the smart phones and their components. For example, some smart phones now have two cameras, one mounted on the front and one on the back. Additionally, smart phones may have more than one method of connecting to a network to transmit data. These features can be used to provide a more comprehensive communication experience.

SUMMARY

[004] Broadly, this writing discloses the following: communicating in a peer-to- peer computer environment. A tracker peer is contacted and a list of peer nodes are obtained at a user device. A subset of peer nodes are selected from the list of peer nodes to act as relay nodes for a communication. An appropriate compression scheme is selected for the communication at the user device. Data is transmitted to at least one relay node for the at least one relay node to forward to at least one other user device, wherein the relay node is one of the subset of the peer nodes. Data is received at the user device from the at least one relay node forwarded from the at least one other user. BRIEF DESCRIPTION OF THE DRAWINGS

[005] Figure 1 illustrates a block diagram of an example environment for communicating in a peer-to-peer computer environment in accordance with embodiments of the present technology.

[006] Figure 2 illustrates a flowchart of an example method for

communicating in a peer-to-peer computer environment in accordance with embodiments of the present technology.

[007] Figure 3 illustrates a diagram of an example computer system upon which embodiments of the present technology may be implemented.

[008] Figure 4 illustrates a flowchart of an example method for

[009] Figure 5 illustrates a block diagram of an example user device in accordance with embodiments of the present technology.

[0010] Figure 6 illustrates a block diagram of an example environment for communicating with a user device in accordance with embodiments of the present technology. [0011] Figure 7 illustrates a block diagram of example user devices in accordance with embodiments of the present technology.

[0012] Figure 8 illustrates a flowchart of an example method for manipulating images on a display of a communication device in accordance with embodiments of the present technology.

[0013] Figure 9 illustrates a flowchart of an example method for

communicating in accordance with embodiments of the present technology.

[0014] Figure 10 illustrates a flowchart of an example method for automatic contact filtering in accordance with embodiments of the present technology.

[0015] Figure 1 1 illustrates a flowchart of an example method for

communicating in accordance with embodiments of the present technology.

[0016] The drawings referred to in this description of embodiments should be understood as not being drawn to scale except if specifically noted.

DESCRIPTION OF EMBODIMENTS

[0017] Reference will now be made in detail to embodiments of the present technology, examples of which are illustrated in the accompanying drawings.

While the technology will be described in conjunction with various embodiment(s), it will be understood that they are not intended to limit the present technology to these embodiments. On the contrary, the present technology is intended to cover alternatives, modifications and equivalents, which may be included within the spirit and scope of the various embodiments as defined by the appended claims.

[0018] Furthermore, in the following description of embodiments, numerous specific details are set forth in order to provide a thorough understanding of the present technology. However, the present technology may be practiced without these specific details. In other instances, well known methods, procedures, components, and circuits have not been described in detail as not to unnecessarily obscure aspects of the present embodiments.

[0019] Unless specifically stated otherwise as apparent from the following discussions, it is appreciated that throughout the present description of

embodiments, discussions utilizing terms such as "contacting," "selecting," "employing," "receiving," "providing," "sending," "terminating," "transmitting," "communicating," "transcoding," "displaying," "enabling," "initiating,"

"normalizing," "establishing," "reporting," or the like, refer to the actions and processes of a computer system, or similar electronic computing device. The computer system or similar electronic computing device, such as a telephone, smartphone, or handheld mobile device, manipulates and transforms data represented as physical (electronic) quantities within the computer system's registers and memories into other data similarly represented as physical quantities within the computer system memories or registers or other such information storage, transmission, or display devices. Embodiments of the present technology are also well suited to the use of other computer systems such as, for example, optical and mechanical computers.

Overview of Communicating in a Computer Environment

[0020] Embodiments of the present technology are for communicating in a computer environment. A single user may be capable of communicating one-to-one with another user via user devices using no relays. However, such a connection without a relay may not be possible or may be poor quality, unreliable and lack effective scalability to extend a call to a multiparty call.

[0021] Embodiments of the present technology allow a user to communicate to at least one other user via user devices associated with a peer-to-peer computer environment. In one embodiment, a user obtains a list of potential relay peers from a tracker peer. The user device then selects at least one relay peer based on the availability and capabilities of the relay peers. An optimization scheme may be employed to make such a selection. A user device may employ one or several relay peers to communicate with a second user device.

[0022] In one embodiment, pluralities of relay peers are employed to transmit the communication to a second user device. The user device may divide the communication into portions and each portion is sent via a different relay peer. For example, the audio portion of a communication is divided from the video portion of the communication and sent over different relay peers. In one embodiment, the communication is relayed using a first relay peer and then switches to a second relay peer seamlessly during the communication.

[0023] In one embodiment, the relay peers each receive a portion of the communication and replicate the portion. The replicated portions may be sent to a plurality of user devices each being employed by a user. Thus, scalability is achieved and a multi-party communication may be achieved without loss of quality or reliability. Additionally, operations traditionally performed by central servers and infrastructure may be performed by the idle computers of end users.

[0024] In one embodiment, the user devices that are communicating have disparate capabilities. For example, the first user device may be a personal computer with a high speed Internet connection and the second user device may be a smart phone connected to a mobile network. In one embodiment, a relay peer may transcode the communication so that the communication comprises a base layer and an enhanced layer. The base layer may be sent to devices with lower capabilities and the base layer with the enhanced layer may be sent to the devices with higher capabilities.

[0025] A communication as used in the present disclosure may refer to several types of communications, including but not limited to, real-time audio and/or video communication, text messages, photo messages, video messages, error correction codes etc. wherein the communication and messages are sent and received via electronic communication devices. User devices may be, but are not limited to, telephones, computer systems, netbooks, notebooks, televisions, hand held devices, mobile devices, smart phones, cellular phones, personal digital assistants, connected televisions, etc. In the present disclosure the terms relay peer(s) or storage peer(s) refer to electronic devices that are nodes or peers in a peer-to-peer computer environment. The term tracker peer may refer to either a peer in a peer-to-peer computer environment or a server computer system.

[0026] It should be appreciated that a peer-to-peer computer environment is well known in the art and is also known as a peer-to-peer network and is often abbreviated as P2P. It should be understood that a peer-to-peer computer environment may comprise multiple computer systems, and may include routers and switches, of varying types that communicate with each other using designated protocols. In one embodiment, a peer-to-peer computer environment is a distributed network architecture that is composed of participants that make a portion of their resources (such as processing power, disk storage, and network bandwidth) available directly to their peers without intermediary network hosts or servers. Participants may be known as nodes or peers. In one embodiment, peer-to-peer technology is used to manage a cluster of nodes.

[0027] The following discussion will demonstrate various hardware, software, and firmware components that are used with and in computer systems for communicating in a peer-to-peer computer environment using various embodiments of the present technology. Furthermore, the systems and methods may include some, all, or none of the hardware, software, and firmware components discussed below.

Embodiments of Communicating in a Computer Environment

[0028] With reference now to Figure 1, a block diagram of an environment 100 for communicating in a peer-to-peer computer environment. Environment 100 includes first user device 105, tracker peer 1 10, first relay peer 120, second relay peer 125, third relay peer 130, central server 135, second user device 140, third user device 145, connection line 150, and communication lines 160, 165, 170, 175, 180, and 185. Environment 100 comprises components that may or may not be used with different embodiments of the present technology and should not be construed to limit the present technology. [0029] In one embodiment, environment 100 comprises a peer-to-peer computer environment that includes, but is not limited to, first user device 105, tracker peer 110, first relay peer 120, second relay peer 125, third relay peer 130, central server 135, second user device 140, and third user device 145. In one embodiment, each of the components is able to connect with and communicate with any other device or peer in the peer-to-peer computer environment. In one embodiment, every device or peer is not able to connect with or communicate with every other device or peer, but is at least able to connect with and

communicate with one other device or peer.

[0030] In one embodiment, first user device 105 is an electronic device used for communicating in a peer-to-peer computer environment. First user device 105 may be a computer system, a netbook, a notebook, a hand held device, a mobile device, a smart phone, a cellular phone, a personal digital assistant, etc. First user device 105 must have some type of connectivity to a network such as Ethernet, WiFi, WiMax, a cellular network, 3G etc. In one embodiment, first user device 105 is required to install an application for purposes of the present technology. In one embodiment, first user device 105 is not required to install any additional application and instead may practice the present technology using technology already installed on first user device 105. For example, first user device 105 may use a web browser application to practice the present technology. In one embodiment, second user device 140 and third user device 145 comprise the same features as first user device 105. [0031] In one embodiment, the peer-to-peer computer environment comprises a tracker peer computer system such as tracker peer 110. The purpose of tracker peer 110 is to receive a request from a user device such as first user device 105. In one embodiment, tracker peer 110 provides potential relay peers to first user device 105. For example, tracker peer 110 may provide data identifying first relay peer 120, second relay peer 125, and third relay peer 130 as peers or nodes associated with the peer-to-peer computer environment. Tracker peer 1 10 may be combined with other components of environment 100 to provide data to users. Tracker peer 1 10 also has the ability to be updated by the peer-to-peer computer environment with data regarding potential peers and central servers. In one embodiment, the peer-to-peer computer environment comprises more than one tracker peer. In one embodiment, tracker peer 1 10 may also operate as a relay peer or as a central server.

[0032] In one embodiment, first user device 105 identifies potential peers associated with the peer-to-peer computer environment using the list of potential peers obtained from tracker peer 1 10. First user device 105 then contacts each of the potential peers and assesses the potential peers' availability and capabilities. In one embodiment, an optimization scheme is employed to select relay peers from the list of potential peers. [0033] First user device 105 may make a determination, based on user requirements, as to which potential peers may be utilized as relays peers. Such a determination may be made using an algorithm based optimization schemed factors related to the capabilities and availability of the potential peers. The capabilities of the potential peer may include the potential peer's available downlink throughput and uplink throughput as well as processing power and speed. It should be appreciated that the factors for the optimization scheme may include, but are not limited to an estimated geographical length of a path taken by a communication through the peer-to-peer computer environment, the number of hops taken by a communication through the peer-to-peer computer environment, the round trip time of a communication through the peer-to-peer computer environment, the compatibility of a network address translator (NAT) or firewall between a relay peer and the user devices, amount of available resource of the relay peer, the historic reliability of the relay peer, and the length of time the relay peer has been running the protocol.

[0034] In one embodiment, a peer may provide first user device with data identifying potential relays. First user device 105 may then contact the potential relay peers identified by the peer that were not identified by tracker peer 110 to determine if such potential relay peers may be utilized by first user device 105 as a relay peer for communications. This process may be repeated with a subset or with all of the potential relay peers identified by the peers that were not identified by tracker peer 110. In this manner first user device 105 may increase the number of potential peers that may be utilized as relay peers. In one embodiment, a list of potential relay peers is stored on user device 105 between sessions and is used to avoid having to contact tracker peer 110 for a new list every session. In one embodiment, the process of identifying additional potential relay peers is performed during idle times when first user device 105 is not performing a communication.

[0035] In one embodiment, first user device 105 may determine that first relay peer 120 may be utilized as a relay peer for communication. First relay peer 120 may be a computer system or other device associated with the peer-to-peer computer environment. In one embodiment, first relay peer 120 receives a communication from first user device 105 over communication line 160 and relays the communication to second user device 140 over communication line 180 in a one-to-one communication. In one embodiment, first relay peer 120 may receive a communication from first user device 105 and replicate the

communication. The replicated communication may also be used to send the communication to both second user device 140 and third user device 145 in a multi-party communication. Thus replicating and relaying a communication to both second user device 140 and third user device 145 comprise relaying or sending a communication to a plurality of devices. It should be appreciated that a relay may require verification from a user device for each action it takes. In other words, a relay peer may require a communication sent to a relay peer from a user device to be signed by an encryption key. The encryption key is then verified using a second key. Such a step may take place each time data is received at a relay peer.

[0036] First user device 105 may send information regarding the presence status of a user to a relay peer. This information may in turn be relayed to the contacts of a user. The information may also be updated each time a user changes status.

[0037] In one embodiment, first relay peer 120 receives only a portion of a communication, second relay peer 125 receives a different portion of the communication, and third relay peer 130 receives yet a different portion of the communication from first user device 105. First relay peer 120, second relay peer 125, and third relay peer 130 then send or relay their respective portions of the communication to second user device 140. Thus a communication is relayed using a plurality of relay peers. It should be appreciated that a communication may be relayed using any number of peers and is not limited to first relay peer 120, second relay peer 125, and third relay peer 130 of Figure 1. Additionally, first relay peer 120, second relay peer 125, and third relay peer 130 may each replicate their respective portions of the communication. In one embodiment, the replicated portions are sent to third user device 145 and the original portions are sent to second user device 140 for a multi-party communication. [0038] In one embodiment, a communication may be divided such that one portion is audio and one portion is video. An audio or video portion may also be divided into multiple portions to be sent over multiple relay peers. For example, a communication may be divided such that the audio is divided into three portions where one portion is considered the base portion and is given priority or protection over the other portions. The two other audio portions provide additional details to the base portion such that if only the base portion is relayed to second user device 140, the audio communication will be accomplished, but each additional portion that is relayed to second user device 140 will increase the quality of the audio communication. Additionally, an audio portion may be given priority or protection over a video portion. A video portion divided into to sub- portions may similarly provide a low-resolution portion and a hi-resolution portion.

[0039] In one embodiment, the communication also includes error correcting codes. For example, forward error correction (FEC) may be employed to protect the communication data stream in the event where at least of the relay peers used for the communication is terminated. Such a termination may occur to an error, the relay peer being disconnected from the network, the relay peer being shut down, or other reasons. In one embodiment, layered coding is used to in combination with multiple path transmission for the communication. Thus, different levels of communication may be sent to different devices with different capabilities. [0040] In one embodiment, the relay peer has the ability to transcode the communication stream before it relay the communication stream to another user device. In one embodiment, the transcoding comprises separating the

communication stream into a base layer and an enhanced layer. The relay peer may send only the base layer or the base layer and the enhanced layer to a user device. In one embodiment, the relay peer has the ability to selectively determine which layers to send to which user devices based on the capabilities of the user devices. In one embodiment, the base layer comprises enough data for both a video and audio transmission at a lower resolution or data rate than the original communication. In one embodiment, the base layer comprises only audio and no video. In one embodiment, the enhanced layer is used in conjunction with the base layer at an end user device to recreate the original communication. For example, a multi-way call may include three devices two personal computers and one smart phone. In such an example, the relay peer may send the base layer and the enhanced layers to the personal computers and only the base layer to the smart phone. Thus, the relay peer may act as an effective relay for at least two heterogeneous devices that have disparate resources and capabilities.

[0041] In one embodiment, the transmitting user device may select an appropriate compression scheme for the communication. It should be appreciated that such compression scheme for digital media are well known in the art. A communication stream may comprises a combination of text, audio, video, and error correction code.

[0042] It should be appreciated that the described relay processes performed by first relay peer 120, second relay peer 125, and third relay peer 130 may be reversed where the relay peers are receiving communications or portions of communications from second user device 140 or third user device 145 and then relayed to other user devices such as first user device 105. Therefore a given relay peer in the peer-to-peer computer environment may receive a

communication from a first user device then relay the communication to a second user device and also receive a communication from the second user device and relay it to the first user device. Thus a multi-party communication may include two-way communications for each user device involved. Additionally, the number of user devices in a multi-party call is not limited to first user device 105, second user device 140 and third user device 145, but may be performed using any number of user devices.

[0043] In one embodiment, relay peers may be added or removed during the communication. For example, a communication may begin utilizing two relay peers and subsequently removes a relay peer. This may be useful if second user device 140 is receiving an audio portion over first relay peer 120 and a video portion over second relay peer 120 and the video portion is terminated during the communication. Then second relay peer 125 may be removed from relaying or 29 sending the communication. Another example may be a communication using both first relay peer 120 and second relay peer 125 to communicate to both

second user device 140 and third user device 145 in a multi-party communication. In this example, if either second user device 140 or third user device 145

terminates the communication, the other user devices may continue the

communication using only first relay peer 120. Conversely, more relay peers may be added to a communication as additional portions of communication are added or additional user devices are added to a multi-party communication during a communication. Adding a relay peer to replace a failed relay peer may be

considered a failover relay. Thus the ability to add relay peers to a communication in progress provides scalability and reliability to a communication sent over a peer-to-peer computer environment.

[0044] In one embodiment, a relay peer sending or relaying a given portion of a communication may terminate the relaying during a communication and a second relay peer may be selected to continue relaying the remaining portion of the

communication. Thus reliability for the communication is maintained during the regular churn of a peer-to-peer computer environment as peers join and leave the peer-to-peer computer environment.

[0045] In one embodiment, a relay peer may receive feedback from a user device that is receiving a communication from the relay peer. The feedback may comprise information such as the quality of the call or an amount of the communication not being received at the receiving user device. This feedback may be relayed to the sending user device. First user device 105 may decide to change relays or the number of relay peers may be adjusted based on the feedback. For example, first user device 105 may utilize first relay peer 120 and second relay peer 125 to relay a communication divided into two portions to second user device 1 0. In this example first relay peer 120 may receive feedback from second user device 140 that the communication quality is below a threshold. At this point in the communication, first user device 105 divides the remainder of the communication into three portions and send the three portions over first relay peer 120, second relay peer 125, and third relay peer 130 respectively. Thus third relay peer 130 was added to increase the diversity of paths used for communicating or the redundancy in the data relayed across the peer-to-peer computer environment in an attempt to increase the quality of the communication. Any number of relay peers may be added or removed in this manner during a communication.

[0046] Additionally, user device may adjust a parameter of the communication based on a feedback received from another user device via a relay peer. In one embodiment, adjusting a parameter may be adjusting an encoding rate of the communication. In one embodiment, adjusting a parameter may be adjusting a resolution communication In one embodiment, adjusting a parameter may be changing a codec used for the communication. In one embodiment, adjusting a parameter may be changing the error protection scheme of the communication.

[0047] In one embodiment, first user device 105 may communicate with second user device 140 in a one-to-one communication. For such a one-to-one communication, the present technology may be employed with or without relay peers. If a relay peer is not used, there must be compatibility of NATs and firewalls between the user devices. A protocol such as Interactive Connectivity Establishment (ICE), Traversal Using Relay NAT (TURN) or Session Traversal Utilities for NAT (STUN) may be used to accomplish a one-to-one

communication without a relay peer. Layered coding and unequal error protection for the different layers is used to ensure sufficient protection for the lower layers in case of data loss. Encoding rates may also be increased or reduced during a communication.

[0048] In one embodiment, environment 100 includes ^"central server 135.

Central server 135 represents a server computer system or a group of server computer systems such as a server farm. In one embodiment, central server 135 is not employed. In one embodiment, central server 135 is utilized as a back-up system for tracker peer 1 10, storage peer 1 15, first relay peer 120, second relay peer 125, and/or third relay peer 130. Thus central server 135 may perform any of the tasks functions or services of tracker peer 1 10, first relay peer 120, second relay peer 125, and/or third relay peer 130. Therefore if one of the components of the peer-to-peer computer environment fails to perform its task, service or operation, central server 135 may be utilized as a failover to perform the task, service or operation. Central server 135 provides an additional layer of reliability in a peer-to-peer computer environment where peers and nodes join and leave the environment due to regular churn. In one embodiment, central server 135 is utilized as a component of routine operations in the course of a communication.

Operations of Communicating in Computer Environment

[0049] Figure 2 is a flowchart illustrating process 200 for communicating in a peer-to-peer computer environment, in accordance with one embodiment of the present invention. In one embodiment, process 200 is a computer implemented method that is carried out by processors and electrical components under the control of computer usable and computer executable instructions. The computer usable and computer executable instructions reside, for example, in data storage features such as computer usable volatile and non- volatile memory. However, the computer usable and computer executable instructions may reside in any type of computer usable storage medium. In one embodiment, process 200 is performed by tracker peer 1 10, and first relay 120 of Figure 1. In one embodiment, the methods may reside in a computer usable storage medium having instructions embodied therein that when executed cause a computer system to perform the method. [0050] At 202, a tracker peer is contacted and a list of peer nodes are obtained at a user device. In one embodiment, the user device may be first user device 105 and the tracker peer may be tracker peer 110 of Figure 1. In one embodiment, the tracker peer may be a central server.

[0051] At 204, an optimization procedure based on factors relating to the peer nodes of the list of peer nodes is employed. The optimization procedure may include an algorithm and use factors related to the capabilities and resources, such as CPU and bandwidth, of the relay peers as described above in reference to Figure 1.

[0052] At 206, a subset of peer nodes are selected from the list of peer nodes to act as relay nodes for a communication. At 208, an appropriate compression scheme is selected for the communication at the user device.

[0053] At 210, a media format is selected for the communication. The media formats used may be those well known in the art for digital communications comprising speech, audio, video, images, text, error correction code and other digital media. For example h.264 video format may be employed, speex for audio and parity codes for error correction.

[0054] At 212, a type of encoding is selected for the communication. Encoding and decoding procedures and protocols may those that are well known in the art or may based on unique algorithms. At 214, employing error correcting code. In one embodiment, the error correction code is Forward Error Correction (FEC).

[0055] At 214, data is transmitted to at least one relay node for the at least one relay node to forward to at least one other user device, wherein the relay node is one of the subset of the peer nodes. In one embodiment, the at least one relay node and the user devices may be the user devices and relay peers described in Figure 1.

[0056] At 216, data is received at the user device from the at least one relay node forwarded from the at least one other user.

[0057] At 218, data is transmitted to a second relay node for the second relay node to forward to the at least one other user device, wherein the second relay node is one of the subset of the peer nodes. In such a step, the first relay node may have been terminated either at the request of the user device or at the request of the relay peer at which point the second relay peer acts as a failover.

[0058] At 220, data is received from the second relay node forwarded from the at least one other user.

[0059] Figure 4 is a flowchart illustrating process 400 for communicating in a peer-to-peer computer environment, in accordance with one embodiment of the present invention. In one embodiment, process 400 is a computer implemented method that is carried out by processors and electrical components under the control of computer usable and computer executable instructions. The computer usable and computer executable instructions reside, for example, in data storage features such as computer usable volatile and non- volatile memory. However, the computer usable and computer executable instructions may reside in any type of computer usable storage medium. In one embodiment, process 200 is performed by tracker peer 1 10, and first relay 120 of Figure 1. In one embodiment, the methods may reside in a computer usable storage medium having instructions embodied therein that when executed cause a computer system to perform the method.

[0060] At 402, a portion of a communication is received from a first user device at a relay peer, wherein the relay peer is one of a list of potential peers and wherein the first user device and a second user device have disparate CPU power and bandwidth capabilities.

[0061] At 404, the portion of the communication is transcoded to comprise a base layer and an enhanced layer. In one embodiment, transcoding encompasses changing the resolution of the communication.

[0062] At 406, the base layer of the portion of the communication is sent to the second user device from the relay peer. [0063] At 408, the enhanced layer of the portion of the communication is selectively sent to the second user device depending upon a set of capabilities of the second user device.

[0064] At 410, the receiving the portion of the communication from the first user device is terminated at the relay peer during the communication.

[0065] At 412, the portion of the communication from the first user device is received at a second relay peer.

[0066] At 414, the portion of the communication is sent to the second user from the second relay peer.

[0067] At 416, the base layer and the enhanced layer are sent to a third user device.

[0068] It should be appreciated that processes 200 and 400 need not carry out each of the described steps to complete its operation. Nor do the steps need to be carried out in the order described. For example, these process may be for a live communication scenario where the steps take place continuously for portion after portion of the communication. In another example, in the case of video communication: one picture is encoded and transmitted by a first user device then it is separated into two layers by a relay peer. The relay peer transmits the base layer to a second user device and the base and enhancement layers to a third user device. This all takes place in no particular order. During this time the first user device is already encoding the next picture and so on.

Example Computer System Environment

[0069] With reference now to Figure 3, portions of the technology for providing a communication composed of computer-readable and computer-executable instructions that reside, for example, in computer-usable media of a computer system. That is, Figure 3 illustrates one example of a type of computer that can be used to implement embodiments of the present technology.

[0070] Figure 3 illustrates an example computer system 300 used in accordance with embodiments of the present technology. It is appreciated that system 300 of Figure 3 is an example only and that the present technology can operate on or within a number of different computer systems including general purpose networked computer systems, embedded computer systems, routers, switches, server devices, user devices, various intermediate devices/artifacts, stand alone computer systems, mobile phones, personal data assistants, televisions and the like. As shown in Figure 3, computer system 300 of Figure 3 is well adapted to having peripheral computer readable media 302 such as, for example, a floppy disk, a compact disc, and the like coupled thereto. [0071] System 300 of Figure 3 includes an address/data bus 304 for

communicating information, and a processor 306 A coupled to bus 304 for processing information and instructions. As depicted in Figure 3, system 300 is also well suited to a multi-processor environment in which a plurality of processors 306A, 306B, and 306C are present. Conversely, system 300 is also well suited to having a single processor such as, for example, processor 306A. Processors 306A, 306B, and 306C may be any of various types of microprocessors. System 300 also includes data storage features such as a computer usable volatile memory 308, e.g. random access memory (RAM), coupled to bus 304 for storing information and instructions for processors 306A, 306B, and 306C.

[0072] System 300 also includes computer usable non- volatile memory 310, e.g. read only memory (ROM), coupled to bus 304 for storing static information and instructions for processors 306A, 306B, and 306C. Also present in system 300 is a data storage unit 312 (e.g., a magnetic or optical disk and disk drive) coupled to bus 304 for storing information and instructions. System 300 also includes an optional alpha-numeric input device 314 including alphanumeric and function keys coupled to bus 304 for communicating information and command selections to processor 306A or processors 306 A, 306B, and 306C. System 300 also includes an optional cursor control device 316 coupled to bus 304 for communicating user input information and command selections to processor 306A or processors 306A, 306B, and 306C. System 300 of the present embodiment also includes an optional display device 318 coupled to bus 304 for displaying information. [0073] Referring still to Figure 3 , optional display device 318 of Figure 3 may be a liquid crystal device, cathode ray tube, plasma display device or other display device suitable for creating graphic images and alpha-numeric characters recognizable to a user. Optional cursor control device 316 allows the computer user to dynamically signal the movement of a visible symbol (cursor) on a display screen of display device 3 8. Many implementations of cursor control device 316 are known in the art including a trackball, mouse, touch pad, joystick or special keys on alpha-numeric input device 314 capable of signaling movement of a given direction or manner of displacement. Alternatively, it will be appreciated that a cursor can be directed and/or activated via input from alpha-numeric input device 314 using special keys and key sequence commands.

[0074] System 300 is alsoi well suited to having a cursor directed by other means such as, for example, voice commands. System 300 also includes an I/O device 320 for coupling system 300 with external entities. For example, in one embodiment, I/O device 320 is a modem for enabling wired or wireless communications between system 300 and an external network such as, but not limited to, the Internet. A more detailed discussion of the present technology is found below.

[0075] Referring still to Figure 3, various other components are depicted for system 300. Specifically, when present, an operating system 322, applications 324, modules 326, and data 328 are shown as typically residing in one or some combination of computer usable volatile memory 308, e.g. random access memory (RAM), and data storage unit 312. However, it is appreciated that in some embodiments, operating system 322 may be stored in other locations such as on a network or on a flash drive; and that further, operating system 322 may be accessed from a remote location via, for example, a coupling to the internet. In one embodiment, the present technology, for example, is stored as an application 324 or module 326 in memory locations within RAM 308 and memory areas within data storage unit 312. The present technology may be applied to one or more elements of described system 300. For example, a method of modifying user interface 225 A of device 1 15A may be applied to operating system 322, applications 324, modules 326, and/or data 328.

[0076] System 300 also includes one or more signal generating and receiving device(s) 330 coupled with bus 304 for enabling system 300 to interface with other electronic devices and computer systems. Signal generating and receiving device(s) 330 of the present embodiment may include wired serial adaptors, modems, and network adaptors, wireless modems, and wireless network adaptors, and other such communication technology. The signal generating and receiving device(s) 330 may work in conjunction with one or more communication interface(s) 332 for coupling information to and/or from system 300. Communication interface 332 may include a serial port, parallel port, Universal Serial Bus (USB), Ethernet port, antenna, or other input/output interface. Communication interface 332 may physically, 29 electrically, optically, or wirelessly (e.g. via radio frequency) couple system 300 with another device, such as a cellular telephone, radio, or computer system.

[0077] The computing system 300 is only one example of a suitable computing environment and is not intended to suggest any limitation as to the scope of use or functionality of the present technology. Neither should the computing environment 300 be interpreted as having any dependency or requirement relating to any one or combination of components illustrated in the example computing system 300.

[0078] The present technology may be described in the general context of computer-executable instructions, such as program modules, being executed by a computer. Generally, program modules include routines, programs, objects,

components, data structures, etc., that perform particular tasks or implement particular abstract data types. The present technology may also be practiced in distributed computing environments where tasks are performed by remote

processing devices that are linked through a communications network. In a

distributed computing environment, program modules may be located in both local and remote computer-storage media including memory-storage devices.

[0079] Embodiments of the present technology may be described in the general context of computer-executable instructions, such as program modules, being executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform particular tasks or 047929 implement particular abstract data types. Embodiments of the present technology may also be practiced in distributed computing environments where tasks are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, program modules may be located in both local and remote computer-storage media including memory-storage

devices.

[0080] Although the subject matter is described in a language specific to

structural features and/or methodological acts, it is to be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are disclosed as example forms of implementing the claims.

Overview of Communicating with a User Device

[0081] Embodiments of the present technology are for with a user device.

Modern technology provides a variety of mobile handheld device. Smart phones, for example, may have more than one camera and more than one method of

connecting to a network. Additionally the bandwidth and display capabilities of such devices are increasing.

[0082] Many users of computer systems have the ability to easily send real-time audio and video communication streams using their computer systems over a network. Such activities may be considered video messaging, video conferencing or the like. Embodiments of the present technology are for sending and receiving multiple communication streams using a handheld device over a network.

Embodiments of the present technology may comprise both the use of handheld technology manipulating communication streams and the use of peer-to-peer computer environments for communicating.

[0083] Such communications streams may include video, audio, text and error correction code. Embodiments of the present technology allow a user to easily add or remove communications streams to an existing communication stream. For example, a user may employ a handheld device to communicate a second handheld device operated by a second user. The initial communication may start with only an audio communication stream. However, either user may initiate a second form of communication, e.g. a video stream, enabling each user to view the other user without ending the audio communication stream.

[0084] Embodiments of the present technology allow for flexibility in adding and removing communication streams to an existing communication. During an audio communication, a video communication may be initiated and terminated several times while maintaining an uninterrupted audio communication stream.

Additionally, only one video stream may be sent so that a first user may view a second user without the second user viewing the first. Additionally, an audio stream may be terminated while maintaining a video communication. [0085] Embodiments of the present technology also allow for flexibility in selecting which camera to use for a video communication stream. For example, a communication may begin with an audio stream and a video stream where the video is captured from a camera located on the front of a handheld device. During the communication the video stream may be seamlessly swapped to video that is captured from a camera located on the back of a handheld device without disrupting the audio stream. Additionally, a first user may send two video streams to a second user comprising video from both the front camera and back camera of the handheld device.

[0086] Embodiments of the present technology also allow for a seamless transition between networks. For example a user may employ a smart phone to initiate a communication with a second smart phone using a wireless card in the first smart phone. The user may then move out of range to effectively use the wireless card during the communication. At that point the smart phone may switch to a cellular network to seamlessly continue the communication. In one embodiment, the communication may change the level of resolution or level of quality to communicate more or less data depending on the difference in capabilities between the two networks.

[0087] In one embodiment, a handheld device may display a video stream that is being transmitted and simultaneously display a second video display that is being received from another handheld device. In one embodiment, the hand held device has the ability to swap the position of the two video streams being displayed on the handheld device.

[0088] In one embodiment, a handheld device is able to initiate a communication with a second handheld device by initiating contact with a data center. After the initial contact with the data center, in one embodiment, the two handheld devices are able to communicate with each other without requiring the use of the data center or another central server, but rather send and receive the communication over a network.

[0089] In one embodiment, a handheld device may provide seamless contact matching. Most smart phones employ a contact list or address book. Upon executing an application for the present technology on a smart phone, the smart phone may seamlessly filter the contacts in the contact list of smart phone to match the smart phone with other smart phones which also have installed an application for the present technology in an automatic fashion requiring no intervention from a user. Such a technique may require the use of a data center or server computer system.

[0090] These techniques may all be used in combination to allow an endless variety of initiating and terminating communication streams all during the course of one communication. Embodiments of Communicating; with a User Device

[0091] With reference now to Figure 5, a block diagram of a front and back view of device 500 used for communicating with a user device. Device 500 includes display 505, image receiving device 510, first display region 515, second display region 520 and image receiving device 525. Device 500 comprises components that may or may not be used with different embodiments of the present technology and should not be construed to limit the present technology.

[0092] In one embodiment, device 500 also comprises at least one means for connecting to a network. It should be appreciated that device 500 is a user device that may be, but is not limited to, a smart phone, a pda, a handheld computer, a cell phone, an iPhone®, or a smart phone with the Android® operating system by Google®.

[0093] In one embodiment, device 500 has two cameras, image receiving device 510 on the front of device 500 such that images of a user may be captured while a user is viewing display 505 and image receiving device 525 on the back of device 500. It should be appreciated that an image receiving device may be a digital camera capable of capturing either still images or video.

[0094] In one embodiment, display 505 is integrated with device 500 and is capable of displaying digital data including text, images and video. Device 500 929 may be a liquid crystal display or an equivalent type display. In one embodiment, display 505 is also a touch screen that is able to receive input data from a user and to display regions that allow a user to input different data into the device. In one embodiment, display 505 has a first display region 515 and a second display region 520 used for displaying video. In one embodiment, video that is being displayed in first display region 515 may be swapped with video being displayed in second display region 520. It should be appreciated that either first display region 515 or second display region 520 may display video that is captured from an image receiving device associated with device 500 or video that is captured from an image receiving device not associated with device 500.

[0095] With reference now to Figure 6, a block diagram of environment 600 used for communicating with a user device. Environment 600 includes data

center 605, network path 610, central server 615, device 620, user 625, device

630, user 635, image 640, image 645, image 650, and image 655. Environment 600 comprises components that may or may not be used with different

embodiments of the present technology and should not be construed to limit the present technology.

[0096] In one embodiment, device 620 and device 630 are the same as device 500 of Figure 5 have all of the same features and capabilities. In one

embodiment, device 620 initiates a communication with device 630 by first

contacting data center 605. Data center 605 then contacts device 630 and a communication is started involving at least one communication stream. It should be appreciated that the communication stream may be audio, images, text, pictures or error correction code relating to other communication streams. In one embodiment, after data center 605 initiates the communication between the devices, data center 605 no longer plays any role in the communication. In such a technique, device 620 and device 630 only use data center 605 to initiate the communication and then continue the communication over an established network without requiring the communication to be routed through a particular server. Thus communications can be hosted using very little central infrastructure.

[0097] In one embodiment, device 620 and device 630 are unable to use network path 610 for communicating with each other. This may be due to firewalls, which ports are open in a particular network, etc. In one embodiment, device 620 and device 630 may then use central server 615 which is a server computer system designed to relay the communication between the two devices. In one embodiment, central server 615 may be replaced by a peer-to-peer computer network that is described above. In one embodiment, device 620 and device 630 may not require the use of central server 615 for a length of time during a communication and may require the use of central server 615 for a different length of time during the communication. In such an example, the communication may be seamlessly routed between network path 610 and central server 615 without terminating the communication. [0098] Figure 6 depicts a typical arrangement of two users each using a device to communicate video and audio streams. For example, user 625 uses an image receiving device to capture video of user 625 's head. This video is then displayed as image 640 on device 620 and is transmitted to device 630 where it is displayed as image 645. In turn, user 635 uses an image receiving device to capture video of user 635's head. This video is then displayed as image 655 on device 630 and is transmitted to device 620 where it is displayed as image 650. In one embodiment, device 620 is able to swap the position of image 640 and image 650. Such a swap may be performed upon receiving a request from a user. Such a request may be obtained by displaying a region superimposed over image 650. When a user touches the region, then the images will be swapped.

[0099] Figure 6 shows device 620 which displays image 650 as a large image which comprises most of the available display and image 640 as being

superimposed over image 640 and obscuring a portion of image 650. Such an arraignment is sometimes known as picture in picture. It should be appreciated that the present technology is not limited to such an arraignment but may instead display the two images such that one is not overlapping or superimposed over the other. The images may be the same size and shape or different from each other.

[00100] With reference now to Figure 7, a series of block diagrams illustrating example devices in accordance with the present technology. The images are examples and should not be construed to limit the present technology. [00101] In one embodiment, device 705 and 710 illustrate an example of a oneway video communication where device 705 captures images or a video, displays the video on device 705, and transmits the video to device 710 and device 710 displays the video. A one-way video communication may also include an audio communication that is either one-way or two ways. Additionally, one-way video may include more than one video stream. For example, device 705 may capture images or video from two distinct image receiving devices and transmit both video streams to device 710. It should be appreciated that a one-way video communication make take place as only a portion of a communication. In other words, during the duration of a communication, portions of the communication may freely switch between two-way video, one-way video, and no video and may or may not include two-way audio.

[00102] In one embodiment, device 715 and device 720 illustrate other possible video communications. For example, device 715 may capture two video streams and transmit them to device 720, one video stream of a rectangle and one of a triangle. Device 715 is shown as displaying the rectangle in the smaller display region on the lower left portion of the display, but does not display the video of the triangle. Device 720 displays both video streams which it receives from device 715, one portion of the display shows the triangle and another portion of the display shows the rectangle. Device 720, in turns captures a video stream of a circle which it does not display but transmits to device 715 where it is shown in a large portion of the display associated with device 715. Moreover, devices 715 and 720 may also be communicating using other forms of communication, such as audio or text, simultaneous to the described video communications. Such additional communications may be one-way or two way and may start and stop during the overall communication and may take place simultaneous to a video communication or may take place during a time period in between two video communications.

[00103] Therefore it is demonstrated that a communication may employ a plurality of video streams, a plurality of audio streams, and/or a plurality of other communication streams. Additionally, the streams may start or stop periodically throughout the communication and the streams may be one-way or two-way. Additionally a communication is not limited to only two device but may include more than two devices in a mutli-party communication where an endless possibility communication streams may be employed during a single

communication.

[00104] In one example, a communication may begin with only audio streams. A video stream or streams may then be added. Later the video stream or streams may be terminated while maintaining the audio streams. Video streams may then be added again later. [00105] In one embodiment, a touchscreen may be used as part of a device. Portions of the display on the touchscreen may be display regions designated to receive input from a user using techniques well known in the art of touchscreens. For example, three regions may be employed at the bottom of the display and a fourth region the top right portion of the display that may be used with the touchscreen device to control to manipulate which communications streams are displayed, received and transmitted and also the position of the displayed images. This may include regions which act as buttons to mute or terminate an audio stream or video stream. It should be appreciated that a touchscreen is not required to use the present technology. In one embodiment, hardware buttons are used to manipulate and control the communication streams. Additionally a combination of a touchscreen and hardware buttons may be used.

Operations of Communicating with a User Device

[00106] Figure 8 is a flowchart illustrating process 800 for communicating with a user device, in accordance with one embodiment of the present invention. In one embodiment, process 800 is a computer implemented method that is carried out by processors and electrical components under the control of computer usable and computer executable instructions. The computer usable and computer executable instructions reside, for example, in data storage features such as computer usable volatile and non-volatile memory. However, the computer usable and computer executable instructions may reside in any type of computer usable storage medium. In various embodiments, process 800 may be performed by the devices and components depicted in Figures 1, 3, 5, 6 and 7. In one embodiment, the methods may reside in a computer usable storage medium having instructions embodied therein that when executed cause a computer system to perform the method.

[00107] At 802, data captured from a first image receiving device is displayed in a first display region disposed in a first position on a display of the

communication device. For example, the first image receiving device may be a camera associated with the communication device which may be a smart phone and the data captured and displayed is a video.

[00108] At 804, data captured from a second image receiving device is displayed in a first display region disposed in a second position on the display of the communication device. For example, the second image receiving device may be a camera not associated with the communication device which may be a smart phone and the data captured and displayed is a video.

[00109] At 806, selective swapping of said first position of said first display region with said second position of said second display region, upon receiving a selection for said swapping from a user, is enabled. This feature may be useful if a user is transmitting a video while simultaneously receiving a video. The user may be more concerned with what the image being transmitted looks like and may therefore desire to have the image being transmitted displayed as the larger image.

[00110] At 808, selective swapping of the data captured from the first image receiving device with data captured from a third image receiving device, upon receiving a selection for the swapping from the user, is enabled. In one embodiment, this feature allows a device to either display two sets of data both captured with image receiving devices associated with the communication device or display data from two image receiving devices not associated with the communication device. In one embodiment, this feature allows a communication device to terminate data being captured and displayed by an image receiving device with another image receiving device.

[00111] At 810, terminating a first portion of a communication on the communication device while maintaining a second portion of the communication upon receiving a selection for the terminating from the user, is enabled.

[00112] At 812, receiving and transmitting an additional communication on the communication device without terminating an existing communication upon receiving a selection for the additional communication from the user, is enabled.

[00113] Figure 9 is a flowchart illustrating process 900 for communicating, in accordance with one embodiment of the present invention. In one embodiment, process 900 is a computer implemented method that is carried out by processors and electrical components under the control of computer usable and computer executable instructions. The computer usable and computer executable instructions reside, for example, in data storage features such as computer usable volatile and non-volatile memory. However, the computer usable and computer executable instructions may reside in any type of computer usable storage medium. In various embodiments, process 900 may be performed by the devices and components depicted in Figures 1 , 3, 5, 6 and 7. In one embodiment, the methods may reside in a computer usable storage medium having instructions embodied therein that when executed cause a computer system to perform the method.

[00114] At 902, a request is received at a first device to communicate with a second device.

[00115] At 904, contact is initiated with the second device via a data center.

[00116] At 906, a first form of communication is transmitted to the second device without requiring the transmitting to be routed through a central server.

[00117] At 908, the first form of communication is received from the second device without requiring the receiving to be routed through a central server. [00118] At 910, a second form of communication is transmitted to the second device without requiring the transmitting to be routed through a central server.

[00119] At 912, the second form of communication is received from the second device.

[00120] At 914, the transmitted second form of communication is displayed on the first device.

[00121] At 916, the received second form of communication is displayed on the first device.

[00122] At 918, the transmitted second form of communication is terminated while maintaining the transmitted first form of communication.

[00123] Figure 10 is a flowchart illustrating process 1000 for automatic contact filtering, in accordance with one embodiment of the present invention. In one embodiment, process 1000 is a computer implemented method that is carried out by processors and electrical components under the control of computer usable and computer executable instructions. The computer usable and computer executable instructions reside, for example, in data storage features such as computer usable volatile and non-volatile memory. However, the computer usable and computer executable instructions may reside in any type of computer usable storage medium. In various embodiments, process 1000 may be performed by the devices and components depicted in Figures 1, 3, 5, 6 and 7. In one embodiment, the methods may reside in a computer usable storage medium having instructions embodied therein that when executed cause a computer system to perform the method.

[00124] At 1002, data associated with a contact list is received at a computer system from a handheld device, wherein the data was sent automatically from the handheld device without requiring actions from a user. In one embodiment, an application in accordance with the present technology is installed on a device. The application may access a contact list or address book associated with the device. The contact list may have information regarding names, phone numbers, email addresses, etc. This data is then sent to a computer system. For example the computer system could be data center 605 of Figure 6. The data may be sent automatically without require a user to request or send the data manually. In one embodiment, the application will seek and receive permission from the user before accessing the contact list.

[00125] At 1004, the data associated with the contact list is normalized using an algorithm. In one embodiment, this takes place in a computer system. The normalizing may include reorganizing the data associated with the contact list such that it can be easily compared to other contact data. For example, a contact list may have phone numbers that include local phone numbers and international phone number that include a prefix. The normalizing can identify items such as area codes and international prefixes and then properly classify or organize this data to be compared with other data.

[00126] At 1006, the normalized data is compared with data in a database. The other data in the database may be information regarding other devices that also have installed an application in accordance with embodiments of the present technology.

[00127] At 1008, contact matches are established between the data associated with the contact list and contacts in the database. It should be appreciated that the present technology is not limited to a database and may also use some other type of electronic records. For example, a first user may install the application on a first device at which point the name of the user and phone number for the device are sent to the database. Then a second user may input the name and phone number of the first user in a second device. Once the second user installs the application, the contact data regarding the first user, which is stored in the second device, is normalized and compared with the data base. A match to the first user will be found. Then, at 1010, the matches are reported to the handheld device. Thus the second user will be able to contact the first user via the installed application and the components of the device. 2010/047929

[00128] Figure 1 1 is a flowchart illustrating process 1 100 for communicating, in accordance with one embodiment of the present invention. In one embodiment, process 1 100 is a computer implemented method that is carried out by processors and electrical components under the control of computer usable and computer executable instructions. The computer usable and computer executable

instructions reside, for example, in data storage features such as computer usable volatile and non-volatile memory. However, the computer usable and computer executable instructions may reside in any type of computer usable storage

medium. In various embodiments, process 1 100 may be performed by the devices and components depicted in Figures 1, 3, 5, 6 and 7. In one embodiment, the methods may reside in a computer usable storage medium having instructions embodied therein that when executed cause a computer system to perform the method.

[00129] At 1 102, a tracker peer is contacted and obtaining a list of peer nodes at a user device.

[00130] At 1 104, a subset of peer nodes are selected from the list of peer nodes to act as relay nodes for a communication.

[00131] At 1 106, a first form of communication is transmitted to at least one relay node for the at least one relay node to forward to at least one other user device, wherein the relay node is one of the subset of the peer nodes. [00132] At 1 108, the first form of communication is received at the user device from the at least one relay node forwarded from the at least one other user.

[00133] At 1 1 10, a second form of communication is transmitted to the at least one relay node for the at least one relay node to forward to the at least one other user device, wherein the relay node is one of the subset of the peer nodes.

[00134] It should be appreciated that processes 800, 900, 1000, and 1 100 need not carry out each of the described steps to complete its operation. Nor do the steps need to be carried out in the order described. All elements, parts and steps described herein are preferably included. It is to be understood that any of these elements, parts and steps may be replaced by other elements, parts and steps or deleted altogether as will be obvious to those skilled in the art.

This writing has disclosed at least the following concepts.

CONCEPTS

Concept 1. A computer implemented method for communicating in a peer-to-peer computer environment, said method comprising:

contacting a tracker peer and obtaining a list of peer nodes at a user device; selecting a subset of peer nodes from said list of peer nodes to act as relay nodes for a communication;

selecting an appropriate compression scheme for said communication at said user device; transmitting data to at least one relay node for said at least one relay node to forward to at least one other user device, wherein said relay node is one of said subset of said peer nodes; and

receiving data at said user device from said at least one relay node forwarded from said at least one other user.

Concept 2. The computer-implemented method of Concept 1 wherein said selecting a subset of peer nodes to act as relay nodes further comprises:

employing an optimization procedure based on factors relating to said peer nodes of said list of peer nodes.

Concept 3. The computer-implemented method of Concept 2 wherein said factors relating to said peer nodes comprises an amount of available resources of said peer nodes, wherein said resources are CPU and bandwidth.

Concept 4. The computer-implemented method of Concept 2 wherein said factors relating to said peer nodes comprises a type of Internet connection of said peer nodes.

Concept 5. The computer-implemented method of Concept 2 wherein said factors relating to said peer nodes comprises a distance between a user and said peer nodes, wherein said distance comprises round trip time, hops, and forward trip time. Concept 6. The computer-implemented method of Concept 2 wherein said factors relating to said peer nodes comprises a length of time a relay protocol has been running on one of said peer nodes.

Concept 7. The computer-implemented method of Concept 2 wherein said factors relating to said peer nodes comprises a historic reliability of one of said peer nodes.

Concept 8. The computer- implemented method of Concept 1 wherein said selecting appropriate compression scheme for said communication further comprises:

selecting a media format for said communication.

Concept 9. The computer-implemented method of Concept 1 wherein said selecting appropriate compression scheme for said communication further comprises:

selecting a type of encoding for said communication.

Concept 10. The computer-implemented method of Concept 1 wherein said selecting appropriate compression scheme for said communication further comprises:

employing error correcting code.

Concept 1 1. The computer-implemented method of Concept 10 wherein said error correcting code is packet level Forward Error Correction (FEC).

Concept 12. The computer-implemented method of Concept 1, further comprising: transmitting data to a second relay node for said second relay node to forward to said at least one other user device, wherein said second relay node is one of said subset of said peer nodes; and

receiving data from said second relay node forwarded from said at least one other user.

Concept 13. A computer-usable storage medium having instructions embodied therein for causing a computer system to perform the steps of communicating in a peer-to-peer computer environment:

contacting a tracker peer and obtaining a list of peer nodes at a user device;

selecting a subset of peer nodes from said list of peer nodes to act as relay nodes for a communication;

selecting an appropriate compression scheme for said communication at said user device;

transmitting data to at least one relay node for said at least one relay node to forward to at least one other user device, wherein said relay node is one of said subset of said peer nodes; and

Concept 14. The computer-usable storage medium of Concept 13 wherein said selecting a subset of peer nodes to act as relay nodes further comprises: employing an optimization procedure based on factors relating to said peer nodes of said list of peer nodes.

Concept 15. The computer-usable storage medium of Concept 14 wherein said factors relating to said peer nodes comprises an amount of available resources of said peer nodes, wherein said resources are CPU and bandwidth.

Concept 16. The computer-usable storage medium of Concept 14 wherein said factors relating to said peer nodes comprises a type of Internet connection of said peer nodes.

Concept 17. The computer-usable storage medium of Concept 14 wherein said factors relating to said peer nodes comprises a distance between a user and said peer nodes, wherein said distance comprises round trip time, hops, and forward trip time.

Concept 18. The computer-usable storage medium of Concept 14 wherein said factors relating to said peer nodes comprises a length of time a relay protocol has been running on one of said peer nodes.

Concept 19. The computer-usable storage medium of Concept 14 wherein said factors relating to said peer nodes comprises a historic reliability of one of said peer nodes.

Concept 20. The computer-usable storage medium of Concept 13 wherein said selecting appropriate compression scheme for said communication further comprises: selecting a media format for said communication.

Concept 21. The computer-usable storage medium of Concept 13 wherein said selecting appropriate compression scheme for said communication further comprises: selecting a type of encoding for said communication.

Concept 22. The computer-usable storage medium of Concept 3 wherein said selecting appropriate compression scheme for said communication further comprises: employing error correcting code.

Concept 23. The computer-usable storage medium of Concept 22 wherein said error correcting code is packet level Forward Error Correction (FEC).

Concept 24. The computer-usable storage medium of Concept 13, further comprising: transmitting data to a second relay node for said second relay node to forward to said at least one other user device, wherein said second relay node is one of said subset of said peer nodes; and

Concept 25. A computer implemented method for communicating in a peer-to-peer computer environment, said method comprising: receiving a portion of a communication from a first user device at a relay peer, wherein said relay peer is one of a list of potential peers and wherein said first user device and a second user device have disparate CPU power and bandwidth capabilities; transcoding said portion of said communication to comprise a base layer and an enhanced layer;

sending said base layer of said portion of said communication to said second user device from said relay peer; and

selectively sending said enhanced layer of said portion of said communication to said second user device depending upon a set of capabilities of said second user device.

Concept 26. The computer implemented method as recited in Concept 25, further comprising:

terminating said receiving said portion of said communication from said first user device at said relay peer during said communication;

receiving said portion of said communication from said first user device at a second relay peer; and

sending said portion of said communication to said second user from said second relay peer.

Concept 27. The computer implemented method as recited in Concept 25 wherein said relay peer replicates said portion of said communication and sends said replicated said portion to a plurality of other user devices. Concept 28. The computer implemented method as recited in Concept 25 wherein said first user device is a user computer system and said second user device is a handheld mobile device.

Concept 29. The computer implemented method as recited in Concept 25 wherein said receiving said portion of said communication from said first user further comprises: transcoding said portion of said communication to change a resolution of said communication.

Concept 30. The computer implemented method as recited in Concept 25 wherein said sending said portion of said communication further comprises:

sending only said base layer to said second user device; and

sending said base layer and said enhanced layer to a third user device.

Concept 31. A computer-usable storage medium having instructions embodied therein for causing a computer system to perform the steps of communicating in a peer-to-peer computer environment- receiving a portion of a communication from a first user device at a relay peer, wherein said relay peer is one of a list of potential peers and wherein said first user device and a second user device have disparate CPU power and bandwidth capabilities; transcoding said portion of said communication to comprise a base layer and an enhanced layer; sending said base layer of said portion of said communication to said second user device from said relay peer; and

Concept 32. The computer-usable storage medium as recited in Concept 31 , further comprising:

Concept 33. The computer-usable storage medium as recited in Concept 31 wherein said relay peer replicates said portion of said communication and sends said replicated said portion to a plurality of other user devices.

Concept 34. The computer-usable storage medium as recited in Concept 31 wherein said first user device is a user computer system and said second user device is a handheld mobile device. Concept 35. The computer-usable storage medium as recited in Concept 31 wherein said receiving said portion of said communication from said first user further comprises: transcoding said portion of said communication to change a resolution of said communication.

Concept 36. The computer-usable storage medium as recited in Concept 31 wherein said sending said portion of said communication further comprises:

sending only said base layer to said second user device; and

sending said base layer and said enhanced layer to a third user device.

Concept 37. A system for communicating in a peer-to-peer computer environment, said system comprising:

a tracker peer configured to send a list of potential relay peers upon a request from a user device;

a first user device configured to contact said tracker peer to obtain said list of potential relay peer and further configured to send and receive a communication to at least one relay peer selected from said list of potential relay peers;

a second user device configured to send and receive said communication from said at least one relay peer, wherein said second user device has capabilities disparate to the capabilities of said first user device;

said at least one relay peer configured to receive said communication from said first user device and transcode said communication into a base layer and an enhanced layer and sending base layer to said second user device and sending said enhanced layer to said second user device upon a confirmation of said second user device that second user device meets a minimum capability requirement.

Concept 38. The system of Concept 37, further comprising:

a central server configured to replace said tracker upon a failure of said tracker peer and configured to replace said at least one relay peer upon failure of said at least one relay peer.

Concept 39. The system of Concept 37, further comprising:

a second relay peer configured to replace said at least one relay peer upon failure of said at least one relay peer.

Concept 40. A method for manipulating images on a display of a communication device, said method comprising:

displaying data captured from a first image receiving device in a first display region disposed in a first position on a display of said communication device;

displaying data captured from a second image receiving device in a first display region disposed in a second position on said display of said communication device; enabling selective swapping of said first position of said first display region with said second position of said second display region, upon receiving a selection for said swapping from a user;

enabling selective swapping of said data captured from said first image receiving device with data captured from a third image receiving device, upon receiving a selection for said swapping from said user; and enabling terminating a first portion of a communication on said communication device while maintaining a second portion of said communication upon receiving a selection for said terminating from said user.

Concept 41. The method of Concept 40, further comprising:

enabling receiving and transmitting an additional communication on said communication device without terminating an existing communication upon receiving a selection for said additional communication from said user.

Concept 42. The method of Concept 40 wherein said data captured from said first image receiving device, said second image receiving device, and said third image receiving device are video streams.

Concept 43. The method of Concept 40 wherein both said first and second image receiving devices are associated with a second communication device.

Concept 44. The method of Concept 40 wherein said first image receiving device is associated with said communication device and said second image receiving device is associated with a second communication device.

Concept 45. The method of Concept 40 wherein said first display region on said display is smaller than said second display region on said display. Concept 46. The method of Concept 40 wherein said second display region in said second position on said display overlaps and obscures a portion of said first display region in said first position on said display.

Concept 47. A computer-usable storage medium having instructions embodied therein for causing a computer system to perform the steps of manipulating images on a display of a communication device:

displaying data captured from a second image receiving device in a first display region disposed in a second position on said display of said communication device; enabling selective swapping of said first display region with said second position and vice versa, upon receiving a selection for said swapping from a user;

enabling selective swapping of said data captured from said first image receiving device with data captured from a third image receiving device, upon receiving a selection for said swapping from said user; and

enabling terminating a first portion of a communication on said communication device while maintaining a second portion of said communication upon receiving a selection for said terminating from said user.

Concept 48. The computer-usable storage medium of Concept 47, further comprising: enabling receiving and transmitting an additional communication on said communication device without terminating an existing communication upon receiving a selection for said additional communication from said user.

Concept 49. The computer-usable storage medium of Concept 47, wherein said first display region on said display is smaller than said second display region on said display.

Concept 50. A method for communicating, said method comprising:

receiving a request, at a first device, to communicate with a second device;

initiating contact with said second device via a data center;

transmitting a first form of communication to said second device without requiring said transmitting to be routed through a central server;

receiving said first form of communication from said second device without requiring said receiving to be routed through a central server; and

transmitting a second form of communication to said second device without requiring said transmitting to be routed through a central server.

Concept 51. The method as recited in Concept 50, further comprising:

receiving said second form of communication from said second device.

Concept 52. The method as recited in Concept 50 wherein said transmitting said first and second form of communication and receiving said first form of communication occur simultaneously. Concept 53. The method as recited in Concept 50, further comprising: displaying said transmitting said second form of communication on said first device.

Concept 54. The method as recited in Concept 51, further comprising:

displaying said receiving said second form of communication on said first device.

Concept 55. The method as recited in Concept 50 wherein said first form of communication is an audio communication and wherein said second form of

communication is a video communication.

Concept 56. The method as recited in Concept 55 wherein said audio communication is transmitted and received using speex protocol.

Concept 57. The method as recited in Concept 50, further comprising:

terminating said transmitting said second form of communication while maintaining said transmitting said first form of communication.

Concept 58. The method as recited in Concept 50 wherein said transmitting and receiving said first form of communication occurs for a period of time before said transmitting said second form of communication occurs. Concept 59. The method as recited in Concept 50 wherein said transmitting and said receiving further comprises transmitting and receiving error correction code.

Concept 60. A computer-usable storage medium having instructions embodied therein for causing a computer system to perform the steps of communicating:

receiving a request, at a first device, to communicate with a second device;

initiating contact with said second device via a data center;

Concept 61. The computer-usable storage medium of Concept 60, further comprising: receiving said second form of communication from said second device.

Concept 62. The computer-usable storage medium of Concept 60, further comprising: displaying said transmitting said second form of communication on said first device.

Concept 63. The computer-usable storage medium of Concept 61, further comprising: displaying said receiving said second form of communication on said first device. Concept 64. The computer-usable storage medium of Concept 60, further comprising: terminating said transmitting said second form of communication while maintaining said transmitting said first form of communication.

Concept 65. A method for automatic contact filtering, said method comprising:

receiving data associated with a contact list at a computer system from a handheld device, wherein said data was sent automatically from said handheld device without requiring actions from a user;

normalizing said data associated with said contact list;

comparing said normalized data with data in a database;

establishing contact matches between said data associated with said contact list and contacts in said database; and

reporting said matches to said handheld device.

Concept 66. A device for communicating, said device comprising:

a handheld device configured to communicate using a first form of

communication and a second form of communication wherein said second form of communication occurs simultaneous to said first form of communication and can be terminated without disrupting said first from of communication;

at least one image receiving device associated with said handheld device for capturing digital images to be used as a form of communication on said handheld device; a display associated with said handheld device configured to display data captured from said at least one image receiving device and configured to display data captured from an image receiving device not associated with said handheld device and further configured to swap the position of the data captured from the at least one image receiving device with data captured from the image receiving device not associated with said handheld device upon a request from a user.

Concept 67. A method for communicating, said method comprising:

contacting a tracker peer and obtaining a list of peer nodes at a user device;

transmitting a first form of communication to at least one relay node for said at least one relay node to forward to at least one other user device, wherein said relay node is one of said subset of said peer nodes;

receiving said first form of communication at said user device from said at least one relay node forwarded from said at least one other user; and

transmitting a second form of communication to said at least one relay node for said at least one relay node to forward to said at least one other user device, wherein said relay node is one of said subset of said peer nodes.

Concept 68. The method as recited Concept 67 wherein said transmitting said second form of communication is to a second relay node second from said subset of peer nodes. Concept 69. The method as recited in Concept 67, further comprising: receiving said second form of communication from said at least one relay device.

Concept 70. The method as recited in Concept 67 wherein said user device selects said at least one relay node based on factors relating to a distance between a user and said peer nodes, wherein said distance comprises round trip time, hops, and forward trip time.

Concept 71. The method as recited in Concept 67, further comprising:

displaying said receiving said first and second form of communication on said user device.

Concept 72. The method as recited in Concept 67 wherein said first form of communication is an audio communication and wherein said second form of

communication is a video communication.

Concept 73. The method as recited in Concept 67, further comprising:

Concept 74. A computer-usable storage medium having instructions embodied therein for causing a computer system to perform the steps of communicating:

Concept 75. The computer-usable storage medium as recited Concept 74 wherein said transmitting said second form of communication is to a second relay node second from said subset of peer nodes.

Concept 76. The computer-usable storage medium as recited in Concept 74, further comprising:

receiving said second form of communication from said at least one relay device.

Concept 77. The computer-usable storage medium as recited in Concept 74 wherein said user device selects said at least one relay node based on factors relating to a distance between a user and said peer nodes, wherein said distance comprises round trip time, hops, and forward trip time. Concept 78. The computer-usable storage medium as recited in Concept 74, further comprising:

Concept 79. The computer-usable storage medium as recited in Concept 74 wherein said first form of communication is an audio communication and wherein said second form of communication is a video communication.

Concept 80. The computer-usable storage medium as recited in Claim 74, further comprising:

Concept 81. A system for communicating in a computer environment, said system comprising:

a first user device configured to contact said tracker peer to obtain said list of potential relay peer and further configured to communicate to at least one relay peer selected from said list of potential peers using a first form of communication and a second form of communication wherein said second form of communication occurs simultaneous to said first from of communication and can be terminated without disrupting said first form of communication;

a display associated with said first user device configured to display data captured from at least one image receiving device and configured to display data captured from an image receiving device not associated with said first user device and further configured to swap the position of the data captured from the at least one image receiving device with data captured from the image receiving device not associated with said first user device upon a request from a user;

a second user device configured to send and receive said communication from said at least one relay peer; and

Claims

1. A computer implemented method for communicating in a peer-to-peer computer environment, said method comprising:

contacting a tracker peer and obtaining a list of peer nodes at a user device;

2. The computer-implemented method of Claim 1 wherein said selecting a subset of peer nodes to act as relay nodes further comprises:

3. The computer-implemented method of Claim 2 wherein said factors relating to said peer nodes comprises an amount of available resources of said peer nodes, wherein said resources are CPU and bandwidth.

4. The computer-implemented method of Claim 2 wherein said factors relating to said peer nodes comprises a type of Internet connection of said peer nodes.

5. The computer-implemented method of Claim 2 wherein said factors relating to said peer nodes comprises a distance between a user and said peer nodes, wherein said distance comprises round trip time, hops, and forward trip time.

6. The computer-implemented method of Claim 2 wherein said factors relating to said peer nodes comprises a length of time a relay protocol has been running on one of said peer nodes.

7. The computer-implemented method of Claim 2 wherein said factors relating to said peer nodes comprises a historic reliability of one of said peer nodes.

8. The computer-implemented method of Claim 1 wherein said selecting appropriate compression scheme for said communication further comprises:

selecting a media format for said communication.

9. The computer-implemented method of Claim 1 wherein said selecting appropriate compression scheme for said communication further comprises:

selecting a type of encoding for said communication.

10. The computer-implemented method of Claim 1 wherein said selecting appropriate compression scheme for said communication further comprises: employing error correcting code.

11. The computer-implemented method of Claim 10 wherein said error correcting code is packet level Forward Error Correction (FEC).

12. The computer-implemented method of Claim 1, further comprising:

transmitting data to a second relay node for said second relay node to forward to said at least one other user device, wherein said second relay node is one of said subset of said peer nodes; and

13. A computer-usable storage medium having instructions embodied therein for causing a computer system to perform the steps of communicating in a peer-to-peer computer environment:

contacting a tracker peer and obtaining a list of peer nodes at a user device;

transmitting data to at least one relay node for said at least one relay node to forward to at least one other user device, wherein said relay node is one of said subset of said peer nodes; and receiving data at said user device from said at least one relay node forwarded from said at least one other user.

14. The computer-usable storage medium of Claim 13 wherein said selecting a subset of peer nodes to act as relay nodes further comprises:

15. The computer-usable storage medium of Claim 14 wherein said factors relating to said peer nodes comprises an amount of available resources of said peer nodes, wherein said resources are CPU and bandwidth.

16. The computer-usable storage medium of Claim 14 wherein said factors relating to said peer nodes comprises a type of Internet connection of said peer nodes.

17. The computer-usable storage medium of Claim 14 wherein said factors relating to said peer nodes comprises a distance between a user and said peer nodes, wherein said distance comprises round trip time, hops, and forward trip time.

18. The computer-usable storage medium of Claim 14 wherein said factors relating to said peer nodes comprises a length of time a relay protocol has been running on one of said peer nodes.

19. The computer-usable storage medium of Claim 14 wherein said factors relating to said peer nodes comprises a historic reliability of one of said peer nodes.

20. The computer-usable storage medium of Claim 13 wherein said selecting appropriate compression scheme for said communication further comprises:

selecting a media format for said communication.

21. The computer-usable storage medium of Claim 13 wherein said selecting appropriate compression scheme for said communication further comprises:

selecting a type of encoding for said communication.

22. The computer-usable storage medium of Claim 13 wherein said selecting appropriate compression scheme for said communication further comprises:

employing error correcting code.

23. The computer-usable storage medium of Claim 22 wherein said error correcting code is packet level Forward Error Correction (FEC).

24. The computer-usable storage medium of Claim 13, further comprising:

transmitting data to a second relay node for said second relay node to forward to said at least one other user device, wherein said second relay node is one of said subset of said peer nodes; and receiving data from said second relay node forwarded from said at least one other user.

25. A computer implemented method for communicating in a peer-to-peer computer environment, said method comprising:

receiving a portion of a communication from a first user device at a relay peer, wherein said relay peer is one of a list of potential peers and wherein said first user device and a second user device have disparate CPU power and bandwidth capabilities; transcoding said portion of said communication to comprise a base layer and an enhanced layer;

26. The computer implemented method as recited in Claim 25, further comprising: terminating said receiving said portion of said communication from said first user device at said relay peer during said communication;

27. The computer implemented method as recited in Claim 25 wherein said relay peer replicates said portion of said communication and sends said replicated said portion to a plurality of other user devices.

28. The computer implemented method as recited in Claim 25 wherein said first user device is a user computer system and said second user device is a handheld mobile device.

29. The computer implemented method as recited in Claim 25 wherein said receiving said portion of said communication from said first user further comprises:

transcoding said portion of said communication to change a resolution of said communication.

30. The computer implemented method as recited in Claim 25 wherein said sending said portion of said communication further comprises:

sending only said base layer to said second user device; and

sending said base layer and said enhanced layer to a third user device.

31. A computer-usable storage medium having instructions embodied therein for causing a computer system to perform the steps of communicating in a peer-to-peer computer environment: receiving a portion of a communication from a first user device at a relay peer, wherein said relay peer is one of a list of potential peers and wherein said first user device and a second user device have disparate CPU power and bandwidth capabilities; transcoding said portion of said communication to comprise a base layer and an enhanced layer;

32. The computer-usable storage medium as recited in Claim 31 , further comprising: terminating said receiving said portion of said communication from said first user device at said relay peer during said communication;

33. The computer-usable storage medium as recited in Claim 31 wherein said relay peer replicates said portion of said communication and sends said replicated said portion to a plurality of other user devices.

34. The computer-usable storage medium as recited in Claim 31 wherein said first user device is a user computer system and said second user device is a handheld mobile device.

35. The computer-usable storage medium as recited in Claim 31 wherein said receiving

said portion of said communication from said first user further comprises:

36. The computer-usable storage medium as recited in Claim 31 wherein said sending said portion of said communication further comprises:

sending only said base layer to said second user device; and

sending said base layer and said enhanced layer to a third user device.

37. A system for communicating in a peer-to-peer computer environment, said system comprising:

a first user device configured to contact said tracker peer to obtain said list of potential relay peer and further configured to send and receive a communication to at least one relay peer selected from said list of potential relay peers; a second user device configured to send and receive said communication from said at least one relay peer, wherein said second user device has capabilities disparate to the capabilities of said first user device;

38. The system of Claim 37, further comprising:

39. The system of Claim 37, further comprising:

40. A method for manipulating images on a display of a communication device, said method comprising:

41. The method of Claim 40, further comprising:

42. The method of Claim 40 wherein said data captured from said first image receiving device, said second image receiving device, and said third image receiving device are video streams.

43. The method of Claim 40 wherein both said first and second image receiving devices are associated with a second communication device.

44. The method of Claim 40 wherein said first image receiving device is associated with said communication device and said second image receiving device is associated with a second communication device.

45. The method of Claim 40 wherein said first display region on said display is smaller than said second display region on said display.

46. The method of Claim 40 wherein said second display region in said second position on said display overlaps and obscures a portion of said first display region in said first position on said display.

47. A computer-usable storage medium having instructions embodied therein for causing a computer system to perform the steps of manipulating images on a display of a communication device:

48. The computer-usable storage medium of Claim 47, further comprising:

49. The computer-usable storage medium of Claim 47, wherein said first display region on said display is smaller than said second display region on said display.

50. A method for communicating, said method comprising:

receiving a request, at a first device, to communicate with a second device; initiating contact with said second device via a data center;

51. The method as recited in Claim 50, further comprising:

receiving said second form of communication from said second device.

52. The method as recited in Claim 50 wherein said transmitting said first and second form of communication and receiving said first form of communication occur simultaneously.

53. The method as recited in Claim 50, further comprising:

displaying said transmitting said second form of communication on said first device.

54. The method as recited in Claim 51 , further comprising:

55. The method as recited in Claim 50 wherein said first form of communication is an audio communication and wherein said second form of communication is a video communication.

56. The method as recited in Claim 55 wherein said audio communication is transmitted and received using speex protocol.

57. The method as recited in Claim 50, further comprising: terminating said transmitting said second form of communication while maintaining said transmitting said first form of communication.

58. The method as recited in Claim 50 wherein said transmitting and receiving said first form of communication occurs for a period of time before said transmitting said second form of communication occurs.

59. The method as recited in Claim 50 wherein said transmitting and said receiving further comprises transmitting and receiving error correction code.

60. A computer-usable storage medium having instructions embodied therein for causing a computer system to perform the steps of communicating:

6 . The computer-usable storage medium of Claim 60, further comprising:

receiving said second form of communication from said second device.

62. The computer-usable storage medium of Claim 60, further comprising:

63. The computer-usable storage medium of Claim 61 , further comprising:

64. The computer-usable storage medium of Claim 60, further comprising:

65. A method for automatic contact filtering, said method comprising:

normalizing said data associated with said contact list;

comparing said normalized data with data in a database;

reporting said matches to said handheld device.

A device for communicating, said device comprising: a handheld device configured to communicate using a first form of communication and a second form of communication wherein said second form of communication occurs simultaneous to said first form of communication and can be terminated without disrupting said first from of communication;

67. A method for communicating, said method comprising:

contacting a tracker peer and obtaining a list of peer nodes at a user device;

receiving said first form of communication at said user device from said at least one relay node forwarded from said at least one other user; and transmitting a second form of communication to said at least one relay node for said at least one relay node to forward to said at least one other user device, wherein said relay node is one of said subset of said peer nodes.

68. The method as recited Claim 67 wherein said transmitting said second form of communication is to a second relay node second from said subset of peer nodes.

69. The method as recited in Claim 67, further comprising:

70. The method as recited in Claim 67 wherein said user device selects said at least one relay node based on factors relating to a distance between a user and said peer nodes, wherein said distance comprises round trip time, hops, and forward trip time.

71. The method as recited in Claim 67, further comprising:

72. The method as recited in Claim 67 wherein said first form of communication is an audio communication and wherein said second form of communication is a video communication.

73. The method as recited in Claim 67, further comprising: terminating said transmitting said second form of communication while maintaining said transmitting said first form of communication.

74. A computer-usable storage medium having instructions embodied therein for causing a computer system to perform the steps of communicating:

contacting a tracker peer and obtaining a list of peer nodes at a user device;

75. The computer-usable storage medium as recited Claim 74 wherein said transmitting said second form of communication is to a second relay node second from said subset of peer nodes.

76. The computer-usable storage medium as recited in Claim 74, further comprising: receiving said second form of communication from said at least one relay device.

77. The computer-usable storage medium as recited in Claim 74 wherein said user device selects said at least one relay node based on factors relating to a distance between a user and said peer nodes, wherein said distance comprises round trip time, hops, and forward trip time.

78. The computer-usable storage medium as recited in Claim 74, further comprising: displaying said receiving said first and second form of communication on said user device.

79. The computer-usable storage medium as recited in Claim 74 wherein said first form of communication is an audio communication and wherein said second form of communication is a video communication.

80. The computer-usable storage medium as recited in Claim 74, further comprising: terminating said transmitting said second form of communication while maintaining said transmitting said first form of communication.

81. A system for communicating in a computer environment, said system comprising: a tracker peer configured to send a list of potential relay peers upon a request from a user device;

APPENDIX A

U.S. Provisional Patent Application 61/240,223 "System for Multi-Way Communication" Filing date: September 6, 2009

PROVISIONAL PATENT

Part 1 : Inventor(s)

Name(s). Email Addressees) and telephone

Eric Setton, setton.eric(5>amail.com, 452 Addison Avenue, CA94301 Palo Alto

Part 2: invention

Title: System for multi-way communication

Problem the Invention Addresses

Most communications system typically make use of a significant amount of infrastructure for their operations. In particular the bandwidth and computing power of one or several datacenters comprised of many servers usually serves to relay data streams between users. This increases the cost of providing communication services for the users, since service providers need to pay for the cost of bandwidth, and the purchase and maintenance of the servers.

Solution to Problem:

This invention is a new way of providing communication services while greatly alleviating the need for infrastructure. It shows how communication services can be provided with hardly any infrastructure, provided a large user base is available to contribute to the operations of the system. By relying on the CPU and bandwidth of end-users which are part of the overall system, most of the operations traditionally provided by a datacenter can be performed by the idle computers of end users.

When the user base is large enough many computers which have a significant amount of bandwidth and CPU are available to the system to act as real-time relays, replication points, mixers, or transcoders for the data. The robustness o† the system can be obtained by combining media compression techniques and multipart! routing for the media streams. Constant monitoring of the network conditions help mitigate interruption due to relays disconnecting form the network. Finally, server-based failovers help provide an acceptable quality of service.

Description of the Invention:

This invention is a novel way of delivering 1-to-1 and many-to-many communications services with minimal infrastructure requirements. The idea is described in Figure 1 and refined in the subsequent figures laid out in this document.

In Figurel , we consider the case where User 1 and User 2 wish to communicate though personal computers connected to a network such as the Internet. In many cases, User 1 and User 2 can establish a connection directly and exchange data streams (such as text, voice or video) directly without requiring the help of any infrastructure for this exchange, this is the well-known principle of point-to-point communications. In a significant number of cases, due to the nature of their connections to the Internet, User 1 and User 2 cannot however establish a direct communication link and require a third party or a relay to act as an intermediary node to route the communications streams. In this invention, rather than relying on some infrastructure such as servers maintained by the communication service providers, we use the idle resources (CPU, bandwidth) of other users as a cheap alternative to server-based relays. Some infrastructure, e.g. Server 1 , might still be used in the discovery of the peer relays (User 3, User 6) and in the connection establishment to these relays, for example Server 1 could be a STUN server which helps nodes discover what kind of connection they are behind, or a rendez-vous server (see TURN and ICE protocols), which helps connect certain types of nodes, or a tracker which holds a list of IP addresses of end-points which are running the protocol and that can be connected by users looking to route their data. However, by maintaining the data streams (text, voice, video) on end-users personal computers, the cost savings are significant. In Figure 1 , we describe the data flows when the connections are established by the protocol. User 3, 4, 5, 6 are several end points, which run the protocol and may have spare resources, User 3 and User 6 are those chosen by User 1 and by User 2 to route the data. This choice may be determined by an optimization procedure which considers the following metrics:

The amount of bandwidth available on the relays

The amount of GPU available on the relays

The type of connection to the Internet of the relays

The distance (hops, round-trip time, forward trip time) between the relays and User 1 , and/or User 2

The length of time the relays have been running the protocol

The historic reliability of relays

Server 1

Figure 1 Distributed connection for 1-1 communications with 1 reiay per stream

In addition to the case described in Figure 1 more reliable and optimized communication paths can be setup notably to preempt disruption in the event of relay chum. This is to be expected since relays run on personal computers which can be switched off at any time. This is what is shown in Figure 2, where User 1 splits the data stream destined to user 2 and sends part of the data stream through User 3 and part of the data stream through User 4. Please note that this path diversity arrangement is only relevant in the case where User 1 and User 2 cannot be connected directly. In the converse, since both Users wish to participate in the communication, user churn is not relevant.

Figure 2: Distributed connection for 1-1 communication with data splitting for resilience against churn

In the communication network depicted in Figure 2, User 1 and/or User 2 obtain a list of Peers from a Tracker (for example Server 1 ), this list in this example would hold the addresses of User 3-6. A subset of relay peers is selected using metrics as described above, e.g., User 3 and 4.

Based on its knowledge of the data communication paths employed to reach User 2, User 1 determines an appropriate compression scheme for the data stream (text, voice or video) to be sent out.

In particular:

Different types of media can be sent over different paths

The data can be encoded using multiple description coding and each description can be sent via a separate relay so that even when a relay is disconnected the media is decodable albeit at a reduced resolution or quality

Error-correcting codes, such as packet level Forward Error Correction (FEC) can be employed to protect the data stream in the event where M of the N relays used simultaneously are disconnected Layered coding can be used in combination with multiple path transmission to combine the advantages of the two methods previously mentioned.

In addition to the case described in Figure 1 and 2, different arrangements can be setup in the case of many-to-many communications. In this case, in particular, each of the N participant should ideally receive N- 1 media streams reflecting the text, voice and video emanating from the other N-1 participants in the conversation. As mentioned above, not all personal computers can be connected together directly, depending on the type of their internet connection. In addition, not all participants may have enough bandwidth to send a good quality stream to the N-1 participants.

In Figure 3, we show how to address this problem by employing relays to replicate the data so that each participant only send one stream over their uplink, but receive N-1 stream from the other participants. This arrangement is well suited to the asymmetric structure of most Internet connections. It also well suited to a system which is run on a large user population, where statistically there is a large number of idle nodes with a lot of available bandwidth. This is usually the case in any large system as many studies have shown.

In the communication network depicted in Figure 3, User 1 , 2, and 7 participate in a conversation together involving text, voice and video data from each one. User 1 obtain a list of Peers from a Tracker (for example Server 1 ), this list in this example would hold the addresses of User 3-6. A subset of relay peers is selected using metrics as described above, e.g., User 3. User 3 is selected among other things because it has sufficient bandwidth to replicate the stream from User 1 and send it to both User 2 and 7. For illustrative purposes, the data communications frorii User 2 and from User 7 is also shown. The algorithms shown in Figure 2 and 3 are shown separately but can also be combined, where sub- streams originating from a user are replicated at different relay nodes.

Many-to-many communication with end-user replication

Figure 3: Distributed connection for many-to-many communication with replication on peer relays

In addition to what has been explained above, the invention also encompasses a communication service system to handle heterogeneous devices which may not have the same capabilities, such as in the case depicted in Figure 4 where 3 users (User 1 , 2, and 8) wish to communicate together using text, voice, and/or video. User 1 and 2 are using personal computers and User 8 is using a cellphone which has for example less CPU and bandwidth than the others. For illustrative purposes, User 8 can support encoding/decoding and the transmission of the base layer of a video stream, whereas User 1 and User 2 can support the transmission and encoding/decoding of a base layer and of one or several enhancement layers.

Many-to-many communication with transcoding

In such a case, the communication system described in this provisional patent, encompasses trans-coding of the media streams by the relays as described in the Figure. In this particular example, User 3 and User 6 receive a data stream from User 1 and User 2 which includes a base layer and an enhancement layer. Depending on the peer they are communicating to, the full stream is transmitted (e.g., the full stream is transmitted from User 3 to User 2. In some other case, the stream is transcoded down to a base layer stream only in order to reach a receiver which has less capability (such as User 8's cellphone). This is the way to use end-user nodes to enable adapting the stream of the communication system to the heterogeneous capabilities of the end users.

The case described above should be envisioned of course in its full generality. The example depicted the case of a data stream composed of 2 layers produced by a specific video encoder. However the concept is more general and encompasses:

A stream of data produced using a single layer codec, multi-layer codec, or a composite stream with several streams (e.g. audio/video/text)

Transcoding might be decoding and re-encoding at another rate/resolution (temporal or spatial), using a transcoder which operates directly on the compressed stream, or an a compressed stream that has been encrypted, a transcoder which sheds substream of packets, and so on.

N receivers which can be grouped in M sets with each set having different capabilities

The use of K different transcoders for the stream of one end-point

In addition, this idea of transcoding on intermediate relays may be used in conjunction with the ideas developed in the above.

In the rest of this document we explain how the invention deals with user disconnections which can disrupt the underlying fabric of the transmission network. As described above, when end-users are used as relays for the communication between users, churn is a concern since these relays may be taken offline at any time.

Figures 4 and 5 describe two different ways of how relay disconnections are handled in the case of a single stream transmitted between two users. In Figure 4, once the disconnection of User 3 is detected or signaled, User 4 becomes a failover relay for the stream sent from User 1 . Figure 5 also illustrates that in a similar scenario, when User 6 disappears, a server can be used as a failover relay too.

Figure 4: User 3 and User 6 are used as Figure 5: When relays disappear, other peers real time relays or servers act as failovers

After the failover has taken place, the relays may continue transmitting the stream or the stream may be moved again based on a selecting a more appropriate relay. For example, in order to minimize server expenditure, the stream transmitted by User 2 may use a peer relay rather than a server relay.

In order to mitigate any disruption to the stream during the failover, User 1 and User 2 maintain a list of peers and servers that can act as failover relays as soon as the disconnection of the current relays is detected or when the disconnection is gracefully signaled by the relays when they are about to disconnect from the network.

Figure 4 and Figure 5 illustrate the case of a single communication stream from User 1 to User 2, however this failover mechanism should be understood in its full general sense and applies just as well in the case of multi-stream transmission where the data is split over multiple paths as described above. This case is illustrated in figures 6, 7, and 8, where the data between User 1 and User 2 is split between 2 streams sent over 2 different paths. In this case when one of the relays (in this example User 3) disconnects, the stream affected by the disconnection is either relayed by another peer (e.g. User 5, in Figure 8), or it can also be relayed by a peer that is already being used as a relay by User 1 , in this example User 4.

These failover situations may be maintained for a short while if it is advantageous to change the routing of the stream once better rela s are discovered or more permanently.