EP2174476A2 - Communication system and method for displaying contact user content - Google Patents

Communication system and method for displaying contact user content

Info

Publication number
EP2174476A2
EP2174476A2 EP08735558A EP08735558A EP2174476A2 EP 2174476 A2 EP2174476 A2 EP 2174476A2 EP 08735558 A EP08735558 A EP 08735558A EP 08735558 A EP08735558 A EP 08735558A EP 2174476 A2 EP2174476 A2 EP 2174476A2
Authority
EP
European Patent Office
Prior art keywords
client
user
user terminal
content
communication
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
EP08735558A
Other languages
German (de)
French (fr)
Other versions
EP2174476B1 (en
Inventor
Eric Partaker
Richard Cole
Natasha Sopieva
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Skype Ltd Ireland
Original Assignee
Skype Ltd Ireland
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Skype Ltd Ireland filed Critical Skype Ltd Ireland
Publication of EP2174476A2 publication Critical patent/EP2174476A2/en
Application granted granted Critical
Publication of EP2174476B1 publication Critical patent/EP2174476B1/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L65/00Network arrangements, protocols or services for supporting real-time applications in data packet communication
    • H04L65/1066Session management
    • H04L65/1096Supplementary features, e.g. call forwarding or call holding
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04MTELEPHONIC COMMUNICATION
    • H04M3/00Automatic or semi-automatic exchanges
    • H04M3/42Systems providing special services or facilities to subscribers
    • H04M3/42025Calling or Called party identification service
    • H04M3/42034Calling party identification service
    • H04M3/42059Making use of the calling party identifier
    • H04M3/42076Making use of the calling party identifier where the identifier is a Uniform Resource Locator
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L61/00Network arrangements, protocols or services for addressing or naming
    • H04L61/45Network directories; Name-to-address mapping
    • H04L61/4547Network directories; Name-to-address mapping for personal communications, i.e. using a personal identifier
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L65/00Network arrangements, protocols or services for supporting real-time applications in data packet communication
    • H04L65/1066Session management
    • H04L65/1069Session establishment or de-establishment

Definitions

  • This invention relates to a communication system and method.
  • VoIP Voice over internet protocol
  • a device such as a personal computer
  • VoIP Voice over internet protocol
  • IM video calling and instant messaging
  • VoIP communication system uses a peer-to-peer (“P2P”) topology built on proprietary protocols.
  • P2P client software provided by a P2P software provider on their PC, and register with the P2P system.
  • P2P client software provided by a P2P software provider on their PC
  • the client software is provided with a digital certificate from a server.
  • communication can subsequently be set up and routed between users of the P2P system without the further use of a server.
  • the users can establish their own communication routes through the P2P system based on the exchange of one or more digital certificates (or user identity certificates, "UIC”), which enable access to the P2P system.
  • UICC user identity certificates
  • Presence information is an indication of the current status of a user of the system. More specifically, presence information is displayed in the user interface of the client for each of the contacts that the user has stored, and allows the user to view a best guess regarding the current status of the contacts in the system.
  • Example presence states that may be displayed include "online”, “offline”, “away”, “not available” and “do not disturb”. The use of presence states provides a user with a best guess regarding the current status of a contact before attempting to communicate with the contact. For example, if the user is not online, and therefore unable to be contacted, then this is indicated to the user before attempting to make a call. Similarly, if a contact is busy and unlikely to answer, then this may also be communicated in advance via the presence state. This is a considerable advantage over PSTN systems, which do not provide any prior information on the probable state of a user. The only option in PSTN systems is to dial a number and wait and see if it is answered.
  • Mood messages are short text strings that are composed by the users to distribute information about themselves to their contacts and supplement their presence status.
  • the mood message of a contact is generally displayed next to the contact's name and presence status in the client.
  • Mood messages are useful for a number of reasons. For example, a mood message can be used to give more information or a reason for a particular presence status, e.g. if a user is offline, the mood message may say "On holiday", thereby explaining why the user is offline. Similarly, if a user's presence state is set to "do not disturb", the mood message may say "Busy working. Oniy contact me if urgent".
  • Mood messages are also useful for users that travel frequently, as a VoIP system can be accessed from anywhere in the world, but this is not reflected in the presence states. Therefore, it is useful for a user to show a mood message such as "In London” next to their presence state.
  • mood messages are popular use for mood messages as a means to easily share links to multimedia content on the internet.
  • users can include a hyperlink to a particular webpage that has some interesting multimedia content (such as a video, picture or audio recording) in the mood message, so that all the user's contacts can see the hyperlink and click on the hyperlink to visit the webpage.
  • some interesting multimedia content such as a video, picture or audio recording
  • the multimedia content at the website linked in the mood message can serve to stimulate conversation over the VoIP communication system.
  • this relies on the contacts of the user with the mood message actually clicking on the hyperiink and viewing the webpage.
  • This can be a problem, as a mere hyperlink does not generally encourage other users to view the media.
  • a method of accessing content at a user terminal connected to a communication network and executing a communication client comprising: said client displaying a list of contacts associated with a user of said client; said client retrieving a message from said communication network, wherein said message is related to a further user represented by one of said contacts displayed in said list of contacts, said message comprising a reference to content stored in a storage means accessible by said communication network; said client initiating a call to said further user over the communication network responsive to a user of said client selecting said one of said contacts in said list of contacts; and responsive to initiating said call, said client establishing communication with said storage means using said reference, accessing the content and displaying the content at said user terminal.
  • said content is a video.
  • said content is an audio recording.
  • the method further comprises the step of said client muting said content responsive to said further user answering said call.
  • the method further comprises the step of downloading the list of contacts from a network element.
  • said step of retrieving the message comprises communicating with a further client executed at a user terminal of the further user.
  • said message is a mood message created by said further user.
  • said step of displaying the content at said user terminal comprises said client executing media player means at said user terminal, wherein said content is displayed to said user using said media player means.
  • said media player means is embedded within said client.
  • said media player means comprises user-operable controls for controlling playback and setting audio properties of said content.
  • the method further comprises the step of ascertaining whether said reference to said content is listed in a database prior to displaying said content.
  • said reference is a network address of said content.
  • said network address is a uniform resource identifier.
  • said communication network is a packet based communication network.
  • said client is a voice over internet protocol communication client.
  • said voice over internet protocol communication client is a peer-to-peer communication client.
  • said client is an instant messaging communication client.
  • said user terminal is a personal computer. In another embodiment, said user terminal is a mobile device. According to another aspect of the present invention there is provided a computer program product comprising program code means which when executed by a computer implement the steps according to the above-defined method of accessing content at a user terminal.
  • a method of accessing content at a user termina! connected to a communication network and executing a communication client comprising: said client displaying a list of contacts associated with a user of said client; said client retrieving a message from said communication network, wherein said message is related to a further user represented by one of said contacts displayed in said list of contacts, said message comprising a reference to content stored in a storage means accessible by said communication network; said client receiving a communication request initiated by said further user over the communication network; and responsive to receiving said communication request, said client establishing communication with said storage means using said reference, accessing the content and displaying the content at said user terminal.
  • said communication request is an incoming call.
  • a computer program product comprising program code means which when executed by a computer implement the steps according to the above-defined method of accessing content at a user terminal.
  • a user terminal comprising: communication means for communicating with a communication network; display means; and processing means arranged to execute a communication client, wherein said client is configured to: display a list of contacts associated with a user of said client on said display means; retrieve a message from said communication network, wherein said message is related to a further user represented by one of said contacts displayed in said list of contacts, said message comprising a reference to content stored in a storage means accessible by said communication network; initiate a call to said further user over the communication network responsive to a user of said client selecting said one of said contacts in said list of contacts; and, responsive to initiating said call, establish communication with said storage means using said reference, access the content and display the content on said display means.
  • a user terminal comprising: communication means for communicating with a communication network; display means; and processing means arranged to execute a communication client, wherein said client is configured to: display a list of contacts associated with a user of said client on said display means; retrieve a message from said communication network, wherein said message is related to a further user represented by one of said contacts displayed in said list of contacts, said message comprising a reference to content stored in a storage means accessible by said communication network; receive a communication request initiated by said further user over the communication network; and, responsive to receiving said communication request, establish communication with said storage means using said reference, access the content and display the content at said user terminal.
  • Figure 1 shows a P2P communication system
  • Figure 2 shows a user interface of a client executed on a user terminai
  • Figure 3 shows a detailed view of a user terminal executing a client
  • Figures 4A to 4C show user interfaces for entering a mood message in a client
  • Figure 5A shows a user interface for adding a multimedia mood message
  • Figure 5B shows a user interface for selecting a video to add to a mood message
  • Figure 6A shows multimedia mood message data
  • Figure 6B and 6C shows a user interface containing a multimedia mood message
  • Figure 7 shows a client user interface comprising a contact with a multimedia mood message
  • Figures 8A to 8D show client user interfaces in which a video ringback is displayed
  • Figures 9A to 9B show client user interfaces in which a video ringback can be paused
  • Figures 1OA to 1 OC show client user interfaces with control options for a video ringback
  • Figure 11 shows a client user interface with options displayed following the end of a video
  • Figure 12 shows a client user interface with a message displayed if a video ringback affects call quality
  • Figure 13 shows a client user interface with a resized video
  • Figure 14 shows a client user interface for an unavailable video
  • Figure 15 shows a client user interface displaying a video during an incoming call.
  • FIG. 1 illustrates a P2P communication system 100.
  • a first user of the P2P communication system (denoted “User A” 102) operates a user terminal 104, which is shown connected to a network 106, such as the Internet.
  • the user terminal 104 may be, for example, a persona! computer (“PC"), personal digital assistant (“PDA”), a mobile phone, a gaming device or other embedded device able to connect to the network 106.
  • the user device is arranged to receive information from and output information to a user of the device.
  • the user device comprises a display such as a screen and a keyboard and mouse.
  • the user device 104 is connected to the network 106 via a network interface 108 such as a modem, and the connection between the user terminal 104 and the network interface 108 may be via a cable (wired) connection or a wireless connection.
  • the user terminal 104 is running a client 110, provided by the P2P software provider.
  • the client 110 is a software program executed on a local processor in the user terminal 104.
  • the user terminal 104 is also connected to a handset
  • the microphone and speaker does not necessarily have to be in the form of a handset, but can be in the form of a headphone or earphone with an integrated microphone, or as a separate loudspeaker and microphone independently connected to the user terminal 104.
  • FIG. 2 An exampie of a user interface 200 of the client 110 executed on the user terminal 104 of User A 102 is shown illustrated in Figure 2.
  • the ciient user interface 200 displays the username 202 of User A 102 in the P2P system, and User A can set his own presence state (that will be seen by other users) using a drop down list by selecting icon 204.
  • the client user interface 200 comprises a tab 206 labelled "contacts", and when this tab is selected the contacts stored by the user in a contact list are displayed.
  • this tab is selected the contacts stored by the user in a contact list are displayed.
  • five contacts of other users of the P2P system (User B to F) are shown listed in contact list 208.
  • Each of these contacts have authorised the user of the client 110 to view their contact details and online presence and mood message information.
  • Each contact in the contact list has a presence status icon associated with it. For example, the presence status icon for User B 210 indicates that User B is
  • the presence icon for User C 212 indicates that User C is "not available”
  • the presence icon for User D 214 indicates that User D's state is
  • the contact list for the users (e.g. the contact list 208 for User A) is stored in a contact server 113 shown in Figure 1.
  • the contact server 113 is contacted, and the contact list is downloaded to the user terminal 104. This allows the user to log into the P2P system from any terminal and still access the same contact list.
  • the contact server is also used to store the user's own mood message (e.g.
  • the mood message of User A 102) and a picture selected to represent the user can be downloaded to the client 110, and allows this information to consistent for the user when logging on from different terminals.
  • the client 110 also periodically communicates with the contact server 113 in order to obtain any changes to the information on the contacts in the contact list, or to update the stored contact list with any new contacts that have been added. Presence information is not stored centrally in the contact server. Rather, the client 110 periodically requests the presence information for each of the contacts in the contact list 208 directly over the P2P system.
  • the current mood message for each of the contacts, as well as a picture (avatar) that has been chosen to represent the contact are also retrieved by the client 110 directly from the clients of each of the contacts over the P2P system.
  • Calls to the P2P users in the contact list may be initiated over the P2P system by selecting the contact and clicking on a "call" button 222 using a pointing device such as a mouse.
  • the call may be initiated by typing in the P2P identity of a contact in the field 224.
  • the call set-up is performed using proprietary protocols, and the route over the Internet 106 between the calling user and called user is determined by the peer-to-peer system without the use of servers.
  • an illustrative route is shown between the caller User A (102) and the called party, User B (114), via other peers (116, 118, 120) of the P2P system. It will be understood that this route is merely an example, and that the call may be routed via fewer or more peers.
  • the client 110 performs the encoding and decoding of VoIP packets.
  • VoIP packets from the user terminal 104 are transmitted into the Internet 106 via the network interface 108, and routed to the computer terminal 122 of User B 114, via a network interface 123.
  • a client 124 (similar to the client 110) running on the user terminal 122 of User B 114 decodes the VoIP packets to produce an audio signal that can be heard by User B using the handset 126.
  • the client 124 executed on user terminal 122 encodes the audio signals into VoIP packets and transmits them across the Internet 106 to the user terminal 104.
  • the client 110 executed on user terminal 104 decodes the VoIP packets from User B 114, and produces an audio signal that can be heard by the user of the handset 112.
  • the VoIP packets for the P2P call described above are passed across the Internet 106 only, and the PSTN network is not involved. Furthermore, due to the P2P nature of the system, the actual voice calls between users of the P2P system can be made with no servers being used. This has the advantages that the system scales easily and maintains a high voice quality, and the call can be made free to the users.
  • FIG 3 illustrates a detailed view of the user terminal (104) on which is executed client 110.
  • the user terminal 10 comprises a central processing unit (“CPU") 302, to which is connected a display 304 such as a screen, an input device such as a keyboard 306, a pointing device such as a mouse 308, a speaker 310 and a microphone 312.
  • the speaker 310 and microphone 312 may be integrated into a handset 112 or headset, or may be separate.
  • the CPU 302 is connected to a network interface 108 as shown in Figure 1.
  • Figure 3 also illustrates an operating system ("OS") 314 executed on the CPU 302.
  • OS operating system
  • the software stack shows a protocol layer 322, a client engine layer 320 and a client user interface layer (“Ul”) 318.
  • Each layer is responsible for specific functions. Because each layer usually communicates with two other layers, they are regarded as being arranged in a stack as shown in Figure 3.
  • the operating system 314 manages the hardware resources of the computer and handles data being transmitted to and from the network via the network interface 108.
  • the client protocol layer 322 of the client software communicates with the operating system 314 and manages the connections over the P2P system.
  • Processes requiring higher level processing are passed to the client engine layer 320, which handles the processing required for the user to make and receive calls over the P2P system.
  • the client engine 320 also communicates with the client user interface layer 318.
  • the client engine 320 may be arranged to control the client user interface layer 318 to present information to the user via the user interface of the client (as shown in Figure 2) and to receive information from the user via the user interface. The control of the client user interface 318 will be explained in more detail hereinafter.
  • Figure 4A shows a portion of the client user interface for User A 102 (as shown in Figure 2) that is used by User A to set his mood message.
  • the panel 402 is shown when User A uses the pointing device 308 of the user terminal 104 to select the field displaying his username (labelled 202 in Figure 2) in the client user interface (200 in Figure 2).
  • Panel 402 displays the username 202 and presence icon 204 for User A (as in Figure 2).
  • panel 402 also displays an avatar 404 that User A has selected to represent himself, and a text-entry box 406 in which User A can enter a mood message.
  • the panel 402 is of the form shown in Figure 4C.
  • User A has typed the text message "This is a sample text message” 410 and has also typed a webpage address "http://www.skype.com” 412, which is automatically hyperlinked by the client 112 such that the address 412 becomes a clickable link in the Ul, whereby if the user clicks the link using the pointing device 308, the user terminal 104 executes a web-browser program that navigates to the webpage address and displays the webpage to the user on the display device 304.
  • the mood message that was typed by User A is transmitted to the contact server 113 and stored in User A's data record.
  • the mood message data is not transmitted immediately to the contact server 113, but is sent with the next periodic update message from the client 110 to the contact server 113.
  • the mood message of User A 102 is also periodically communicated to each of User As contacts over the P2P system when this is retrieved by the contacts.
  • FIG. 5A 1 illustrates the Ul for adding a multimedia mood message.
  • a video is incorporated into a user's mood message.
  • the same technique may be used for incorporating other types of media into mood messages, such as images and sound recordings.
  • Figure 5A shows a portion of the client Ul for User A 102 similar to that shown in Figure 4A.
  • Panel 502 comprises a mood message entry area 504 which displays a different message to that in Figure 4A.
  • the message invites the user to enter a text message or to add a video into their mood message, and has a hyperlink embedded for the word "video" 506.
  • the user therefore not only has the option to write a text message, but also to add a video to their mood message.
  • other types of media such as still images or audio
  • the hyperlink in Figure 5A is changed accordingly, for example to read "video or photo”.
  • a cursor is shown as in Figure 4B, and the user can type a text message, as in Figure 4C, which is then distributed to the contacts of User A as described previously. If the user uses the pointing device to click on the word "video”, then the user begins the process for seiecting a video for insertion into the mood message.
  • the selection of a video is made by the user through the presentation of a Ul showing a selection of available videos, which is displayed to the user responsive to the user clicking on the word "video".
  • a video selection Ul 508 is illustrated in Figure 5B.
  • the Ul 508 can be considered to comprise two separate elements.
  • the first element is the frame of the U! 508, comprising tabs (510, 512, 514), navigation buttons 516, search field 518, mood message preview area 520 and control buttons (522, 524).
  • the second element is a pane 526 which shows different information for each of the tabs (510, 512, 514). Specifically, pane 526 displays representations of the available videos,
  • the information for the above-two elements in Ul 508 is fetched from different sources. Firstly, the information relating to the frame of the Ul is fetched from a content directory 127.
  • the content directory 127 is shown in Figure 1 connected to the network 106.
  • the client 110 is arranged to communicate with the content directory 127 and retrieve the information to allow the client 110 to present the Ul 508 to the user. In particular, the client 110 needs to fetch the information regarding the tabs (510, 512, 514).
  • this address is in the form of a webpage uniform resource locator ("URL").
  • URL uniform resource locator
  • FIG. 1 illustrates three separate content providers (128, 130, 132) connected to network 106.
  • these three content providers (128, 130, 132) respectively provide the information displayed for the three tabs (510, 512, 514) of Ul 508.
  • the content providers (128, 130, 132) provide the information that populates pane 526 and also provide the actual video data.
  • the content providers are third parties, and separate from the P2P software provider. Note that the media content that is made available by the content provider can be provided by individual users uploading the media to the content provider.
  • the Ul 508 shows the content for tab 510, labelled "Partner 1".
  • the client uses the address obtained from the content directory 127 for tab 510 to access content provider 128 in order to display the information for this tab. More specifically, the address from the content directory 127 is a URL of a webpage provided by the content provider 128, and the client downloads this webpage (in the form of hypertext mark-up language (“HTML”)) over the network 106 and displays the webpage in pane 526. If the user uses the pointing device to select a different tab, such as 512 or 514, then the address provided by the content directory 127 for the selected tab is used to fetch the information to be displayed for this tab.
  • HTML hypertext mark-up language
  • pane 526 displays a plurality of thumbnail images (e.g. thumbnail 528) representing each video and a title (e.g. title 530) for each video.
  • the thumbnail image for a video is a small image of a single frame from the video.
  • the thumbnail for each video is generated from the video file itself (e.g. it can be the first frame of the video, or a frame a predetermined time or percentage into the video).
  • a list of categories 532 is displayed to allow the user to view a sub-set of the available videos.
  • the user can search the videos by keywords using the search field 518.
  • the content directory 127 is provided by the P2P software provider. This allows the P2P software provider to present a portal to several content providers (128, 130, 132) that are able to provide videos to the users, thereby avoiding the requirement for the P2P software provider to host and/or stream its own videos.
  • the content directory 127 can be updated to add or remove content providers as required.
  • L)I 508 When L)I 508 is displayed on the users terminal 104, the user can use the pointing device to browse the available videos from the different content providers (by using the different tabs 510-514, the categories 532 and the search field 518). When a user sees a video that he is interested in, he clicks the thumbnail to view a preview of the video. When the user has chosen a video, he can select the "save mood message" button 522 to save the video to his mood message.
  • the user can also add a video to his mood message without using the video selection Ul.
  • the user can copy and paste a URL of a video on a webpage directly into his mood message (e.g. into the field illustrated in Figure 4B).
  • the client detects that this URL is linking to a video (or other type of media), and automatically generates the required multimedia mood message data (described with reference to Figure 6A below) in order for the video mood message to be created for this video.
  • the client can also check that the video URL is from an approved partner (e.g. one of the content providers 128, 130, 132).
  • the user can additionally add a video to his mood message by selecting a control displayed on the webpage of a content provider in association with a particular video.
  • a content provider (128, 130, 132) can display videos that can be viewed by a user using a web-browser.
  • the webpage displays an "add to mood" button in proximity with the videos, such that if the button is activated by the user, then data regarding the video is passed to the client to allow the client to generate the required mood message data. Therefore, the client can create a multimedia mood message in the same way as described previously, but without using the video selection Ul.
  • the user can also add videos from a webpage, even if the webpage is not from a content provider that displays a specific "add to mood" button as part of the webpage itself.
  • the P2P software provider can provide plug-in software for the web-browser program that detects that the webpage being viewed contains an embedded video, and overlays an "add to mood" button in proximity to the video in the webpage. If the user selects the button, then data is passed to the client that allows the client to generate the multimedia mood message data in the same way as described above.
  • the client can check that the webpage is from an approved partner (e.g. one of the content providers 128, 130, 132) before permitting the video to be included in a mood message.
  • the client generates mood message data to enable the mood message to be displayed in the client and passed to the contacts of the user.
  • the mood message data 602 that is generated by the client is illustrated in Figure 6A.
  • the mood message data 602 that is generated and stored by the client comprises two main parts.
  • the first part is a media object 604, which contains the data related to the media (e.g. the video).
  • the second part of the mood message data 602 is a text comment 606 that the user has typed to accompany the video (if present).
  • the media object 604 comprises several data items.
  • a media type field 608 defines the type of media (e.g. video, photo, audio etc.)
  • a media title field 610 includes the title of the media.
  • a content provider identity 612 is used to identify which content provider the media originated from (e.g. 128, 130 or 132 in Figure 1).
  • a content provider uniform resource identifier (“URI") 614 provides an address of where further information may be found regarding the content provider (e.g. the address of the content provider webpage).
  • a media URI 616 is the address of where the media is stored, and this is the address from where the media is streamed when it is played.
  • the thumbnail image for the media is stored in the media object at 618. Preferably, the thumbnail image is provided by the content provider.
  • FIG. 6B shows the same panel 502 of the client Ul as illustrated in Figure 5A.
  • the panel 502 now includes the thumbnail image 620 for the video selected by the user (this is read from the thumbnail image 618 in the media object 604).
  • the user also has the option to add a text comment at this stage to accompany the video (if this was not done previously, or if the user wishes to amend a previously entered comment). This is achieved by the user clicking in the text-entry region 622 and typing.
  • Figure 6C illustrates the case where the user has typed a message 624 in region 622.
  • the user in this case User A
  • the next stage is for the multimedia mood message to be passed to User A's contacts.
  • the video mood message data 602 as shown in Figure 6A is transmitted from the client 110 to the clients of the contacts of User A over the P2P system.
  • the multimedia mood message data 602 for User A is retrieved periodically by the contacts of User A 102.
  • the client 124 of User B 114 (who is a contact of User A) may periodically request mood message information from each of its contacts, thereby requesting the mood message information from User A.
  • the multimedia mood message information is transmitted from User A to User B responsive to such a request.
  • the multimedia mood message data 602 is transmitted to each of the contacts of User A as soon as the mood message is created or changed (i.e.
  • the multimedia mood message data 602 is pushed to the contacts at periodic intervals.
  • User B 114 is a contact of User A 1
  • the multimedia mood message data 602 is provided to the client 124 of User B over the P2P system.
  • the multimedia mood message data must be processed by the client 124 before the mood message is displayed to User B.
  • the client 124 reads the mood message data 602 and analyses the media object 604. For example, the client 124 will read the media type field 608 to determine the type of media referred to in the media object 604 (e.g. a video in this example).
  • the client will also extract the thumbnail 618 so that it may be displayed in the Ul of the client 124 next to the contact entry for User A.
  • the media URI 616 is extracted and hyperlinked to the thumbnail image in the client Ul.
  • the media title 610 is also read for display in the client Ul.
  • Figure 7 shows the client Ul 700 for User B with the contact for User A 702 expanded.
  • the contact for User A 702 shows User A's name 704, presence state 706, avatar 708, and buttons for initiating an IM chat 710 or voice call 712 with User A.
  • Next to the contact 702 for User A is displayed a thumbnail image 714 of the video chosen by User A for his mood message.
  • the thumbnail image 714 is extracted from the thumbnail field 618 of the multimedia mood message data 602.
  • the thumbnail image 714 also has a play button 716 overlaid on it, to indicate to the user that the video can be played by clicking on the thumbnail image 714. If User B uses his pointing device to place a pointer over the thumbnail image 714, then the text comment provided with the video (if any) is displayed. Alternatively, if no text comment has been provided, the title of the video is displayed.
  • a problem with User A 102 sharing multimedia content in a mood message is that it relies on User B 114 actively selecting to view the video in the mood message (by activating the play button 716).
  • a system and method is utilised whereby the video from User A's mood message is automatically displayed to a contact of User A (e.g. User B) whenever that contact attempts to establish a call with User A. More specifically, during the time between the call being initiated by the contact and the time at which User A answers the call, the video is played to the contact.
  • This technique can be referred to as a "video ringback".
  • FIG. 8A to 8D illustrate the process by which a video ringback is displayed to User B 114 when he is calling User A 102.
  • User A 102 has created a video mood message, and the mood message data has been passed to User B 114, such that the video mood message is displayed in User B's client Ul 700, next to User A's name (as illustrated in Figure 7).
  • User B 114 initiates a call with User A 102, by selecting the call button 712 in the client Ul 700 (or alternatively using call button 718).
  • the client 124 of User B 114 attempts to establish a connection with the client 110 of User A 102 over the network 106 in order to set up a voice call.
  • the client 124 of User B 114 also detects that the contact being called (User A in this example) has a video mood message. In response to detecting this, the client attempts to start a video ringback. The client 124 reads the mood message data 602, and specifically the media URI 616. The client uses the media URI 616 to contact the location at the content provider (128, 130, 132) where the video is stored, and initiate streaming of the video to the client 124 of User B 114.
  • the Ul 700 of the client 124 of User B at the beginning of this process is shown illustrated in Figure 8A.
  • the Ul shown in Figure 8A now displays a call tab 802 (indicating the name of the person being called) instead of the contact list tab.
  • Shown in the call tab 802 is a contact card 804 for the user being called (User A 102).
  • the contact card 804 comprises the name of the callee 806, their presence state 808 and avatar 810, as well as further control buttons 812 that are out of the scope of this description.
  • an embedded video player 814 Below the contact card is an embedded video player 814.
  • the client 124 is loading the video from User A's mood message (using the media URI 616), and this is reflected in the image displayed in the embedded video player 814.
  • the status of the call to User A 102 is shown at 816.
  • User A has not yet answered the call, and the status is "connecting".
  • FIG. 8B shows the client Ul 700 as in Figure 8A, except that the video from User A's mood message is now being played to User B in the embedded video player 814.
  • the c ⁇ ent 124 of User B 114 initially plays an audible ringing tone to User B 114 whilst the video begins to be played back, in order to indicate to the user that the call is waiting to be answered.
  • the ringing tone has been played for a predetermined period of time (e.g. 3 seconds)
  • the ringing tone is faded out, and the sound that accompanies the video is faded in.
  • control bar 820 comprises a volume control button 822, and activation of this button by User B 114 allows the volume to be restored, so that the sound of the video can be heard.
  • the client pauses the content responsive to User A answering the call, rather than the content being muted.
  • the message 818 displayed over the video in Figure 8C indicates that the video is paused.
  • the playback of the video can be resumed by the user by selecting a play button on the control bar 820, as described in more detail with reference to Figure 1OA to 1OC, below.
  • the above-described process ensures that a video shared by User A 102 in his mood message is seen by his contacts, due to it being played automatically to them during a time when otherwise only a normal ringing tone would be heard.
  • FIG. 9B The result of the user clicking on the video with the pointing device is shown in Figure 9B.
  • the playback of the video has been paused, which is indicated to the user with message 826.
  • the control bar 820 is displayed to the user to allow him to resume playback if desired.
  • FIG. 10A to 1OC illustrate in more detail the functions of the control bar 820.
  • the control bar 820 comprises a play/pause button 828, a progress bar 830, a volume button 822 and a menu button 832.
  • the play/pause button toggles between playing and pausing the video, and the symbol shown on the button changes accordingly, depending on the current state of the video (i.e. whether it is currently playing or paused).
  • volume button 822 When the volume button 822 is activated, a volume slider is shown to allow the user to change the volume, or the sound can be muted.
  • a message is shown overlaid on the video.
  • Figure 1OB shows an example of this for the case where the video is both paused and muted, as reflected in message 834.
  • Figure 10C shows the options that are presented to User B 114 when the menu button 832 is selected.
  • the first option is a "chat about this" 836 option. If the "chat about this" option 836 is activated, a dialogue box is opened that displays to User B a list of all his contacts. User B can select from this list the contacts with which he wishes to initiate an IM chat conversation. When he has selected the contacts, an IM chat session is established with these contacts, and the video from the embedded video player 814 is automatically inserted into the beginning of the IM chat conversation.
  • the second option is a "add to my mood” option 838. If this option is selected, then the video that is shown in the embedded video player 814 (i.e. video from User A's mood message) is added to User B's mood message.
  • the third option is a "more info” option 840.
  • the "more info” option provides the user with more information regarding the video. Preferably, this is provided by the client 124 executing a web browser and navigating to a webpage of the content provider, the address of which was provided in the content provider URI 614 field of the mood message data 602.
  • the fourth option is an "auto-play video” option 842.
  • this option is enabled by default. When enabled, this option allows videos that are in a callee's mood message to be played automatically when a call is made to the cailee (as described with reference to Figures 8A to 8D). If this option is disabled, then the video is not played automatically on making the call, but instead a static frame of the video is shown with the control bar 820 below it, allowing the video to be played if desired.
  • the client 124 preferably monitors the user behaviour, so that if the user pauses the ringback video a predetermined number of times consecutively, then the client will pop- up a message to the user, suggesting that the "auto-play video” option be disabled.
  • the menu button 832 can also present a fifth option (not shown in Figure 10C).
  • the fifth option allows the user to report problematic content to the content provider, if this option is selected, then a web browser program is executed on the user's terminal, which navigates to a webpage comprising a form which the user can fill out to report a problem with the content (e.g. copyrighted material, unsuitable subject-matter, etc).
  • the information entered in this form is provided to the content provider, who can choose to remove the video in response thereto.
  • FIG 11 shows the client Ul 700 when the video has completed playing.
  • Two buttons are displayed to the user, overlaid over the video.
  • the first is a "play again” button 844 that allows the user to playback the video from the start. Activation of the play/pause button 828 also has the same effect.
  • the second button is a "chat about this” button 846, which has the same function as selecting the "chat about this" option 836 from the menu button 832 as described above.
  • Playback of the video to the user has requirements both in terms of processing power (i.e. CPU resources) and on the network connection of the user's terminal (i.e. bandwidth resources). As a result of this, it can sometimes be found that playback of the video after the called user has answered can affect the quality of the call.
  • the client monitors the CPU usage, bandwidth, and/or call quality, and if it is determined that the video is detrimentally affecting these aspects, then the video playback is stopped. This is reported to the user as shown in Figure 12, using message 848.
  • User B 114 may need to use controls related to the voice call, such as adjusting sound settings, placing the call on hold or muting his microphone. The controls require space on the call tab 802.
  • the video is paused and reduced in size as shown in Figure 13.
  • the user has opened an audio settings panel 850, and this has resulted in the size of the video being reduced accordingly.
  • the client 124 will perform a check to ensure that the media URI 616 included in the mood message does not relate to a video that has been "blacklisted".
  • a video may be blacklisted if, for example, it is found to infringe copyright or contains offensive material.
  • a blacklist advertiser 134 is shown connected to network 106, and connected to the blacklist advertiser is a blacklist database 136. Note that the blacklist database can be a centralised database or a distributed P2P database. If any videos are "blacklisted", then the address (i.e. media UR! of the video is stored in the blacklist DB 136.
  • the client When a user attempts to access a given media URI, the client first sends a message to the blacklist advertiser 134 containing the media URI.
  • the blacklist advertiser 134 compares the media URl sent by the client with those listed in the blacklist DB 136.
  • blacklist advertiser 134 sends a message to the client indicating that the video has been blacklisted, in response to this the client does not display the media, but instead displays a notification of the problem. Conversely, if the media URI 616 is not listed in the blacklist DB 136, then the blacklist advertiser 134 sends a message to indicate to the client that the media can be displayed to the user.
  • Figure 15 shows the client Ul 200 for User A 102 (as shown in Figure 2). However, in contrast to Figure 2, a call is being made from User B 114 to User A 102. This causes a call tab 1502 to be shown (rather than the contacts tab 206 of Figure 2).
  • the call tab comprises the contact card 1504 of the user calling User A, which displays the callers name 1506, presence state 1510 and avatar 1510.
  • a status field 1512 indicates that this is an incoming call from User B.
  • An embedded video player 1514 displays the video from User B's mood message.
  • the video in the embedded video player 1514 is displayed by the client 110 using the media URI from User B's mood message to stream the video from the content provider (128, 130, 132), in the same manner as described above with reference to Figures 8A to 8D.
  • the video displayed to User A in the embedded video player 1514 is muted (in a similar manner to Figure 8C) but the video continues to play.
  • User A has the same controls over the video shown when there is an incoming call as described above for User B when placing a call.
  • the same control bar and menu options are available as described above with reference to Figure 10A to 10C.
  • the above described system and method ensures that multimedia content shared by users is shown to their contacts in two ways. Firstly, a caller is shown the multimedia content of the called user when he is waiting for the called user to answer the cali. Secondly, the called user is shown the multimedia content of the caller when the incoming call is displayed in the client.

Abstract

A method of accessing content at a user terminal connected to a communication network and executing a communication client. The method comprises: the client displaying a fist of contacts associated with a user of the client; the client retrieving a message from the communication network, wherein the message is related to a further user represented by one of the contacts displayed in the list of contacts, the message comprising a reference to content stored in a storage means accessible by the communication network; the client initiating a call to the further user over the communication network responsive to a user of the client selecting the one of the contacts in the list of contacts; and responsive to initiating the call, the client establishing communication with the storage means using the reference, accessing the content and displaying the content at the user terminal.

Description

COMMUNICATION SYSTEM AND METHOD
This invention relates to a communication system and method.
Voice over internet protocol ("VoIP") communication systems allow the user of a device, such as a personal computer, to make calls across a computer network such as the Internet. These systems are beneficial to the user as they are often of significantly lower cost than fixed line or mobile networks. This may particularly be the case for long distance calls. To use VoIP, the user must install and execute client software on their device. The client software provides the VoIP connections as well as other functions such as registration and authentication. In addition to voice communication, the client may also provide video calling and instant messaging ("IM").
One type of VoIP communication system uses a peer-to-peer ("P2P") topology built on proprietary protocols. To access the peer-to-peer system, the user must execute P2P client software provided by a P2P software provider on their PC, and register with the P2P system. When the user registers with the P2P system the client software is provided with a digital certificate from a server. Once the client software has been provided with the certificate, communication can subsequently be set up and routed between users of the P2P system without the further use of a server. In particular, the users can establish their own communication routes through the P2P system based on the exchange of one or more digital certificates (or user identity certificates, "UIC"), which enable access to the P2P system. The exchange of the digital certificates between users provides proof of the user's identities and that they are suitably authorised and authenticated in the P2P system. Therefore, the presentation of digital certificates provides trust in the identity of the user. It is therefore a characteristic of peer-to-peer communication that the communication is not routed using a server but directly from end-user to end- user. Further details on such a P2P system are disclosed in WO 2005/009019. One of the advantages of VoIP communication systems, compared to the public switched telephone network ("PSTN"), is that supplementary information can be provided about the users of the VoIP communication system. This supplementary information can take the form of presence information and a "mood message".
Presence information is an indication of the current status of a user of the system. More specifically, presence information is displayed in the user interface of the client for each of the contacts that the user has stored, and allows the user to view a best guess regarding the current status of the contacts in the system. Example presence states that may be displayed include "online", "offline", "away", "not available" and "do not disturb". The use of presence states provides a user with a best guess regarding the current status of a contact before attempting to communicate with the contact. For example, if the user is not online, and therefore unable to be contacted, then this is indicated to the user before attempting to make a call. Similarly, if a contact is busy and unlikely to answer, then this may also be communicated in advance via the presence state. This is a considerable advantage over PSTN systems, which do not provide any prior information on the probable state of a user. The only option in PSTN systems is to dial a number and wait and see if it is answered.
Mood messages are short text strings that are composed by the users to distribute information about themselves to their contacts and supplement their presence status. The mood message of a contact is generally displayed next to the contact's name and presence status in the client. Mood messages are useful for a number of reasons. For example, a mood message can be used to give more information or a reason for a particular presence status, e.g. if a user is offline, the mood message may say "On holiday", thereby explaining why the user is offline. Similarly, if a user's presence state is set to "do not disturb", the mood message may say "Busy working. Oniy contact me if urgent". Mood messages are also useful for users that travel frequently, as a VoIP system can be accessed from anywhere in the world, but this is not reflected in the presence states. Therefore, it is useful for a user to show a mood message such as "In London" next to their presence state.
One popular use for mood messages is to utilise the mood message as a means to easily share links to multimedia content on the internet. For example, users can include a hyperlink to a particular webpage that has some interesting multimedia content (such as a video, picture or audio recording) in the mood message, so that all the user's contacts can see the hyperlink and click on the hyperlink to visit the webpage.
The multimedia content at the website linked in the mood message can serve to stimulate conversation over the VoIP communication system. However, this relies on the contacts of the user with the mood message actually clicking on the hyperiink and viewing the webpage. This can be a problem, as a mere hyperlink does not generally encourage other users to view the media.
There is therefore a need for a technique to address the aforementioned problems and provide for the efficient sharing and display of multimedia content by the users.
According to one aspect of the present invention there is provided a method of accessing content at a user terminal connected to a communication network and executing a communication client, comprising: said client displaying a list of contacts associated with a user of said client; said client retrieving a message from said communication network, wherein said message is related to a further user represented by one of said contacts displayed in said list of contacts, said message comprising a reference to content stored in a storage means accessible by said communication network; said client initiating a call to said further user over the communication network responsive to a user of said client selecting said one of said contacts in said list of contacts; and responsive to initiating said call, said client establishing communication with said storage means using said reference, accessing the content and displaying the content at said user terminal. In one embodiment, said content is a video. In another embodiment, said content is an audio recording.
Preferably, the method further comprises the step of said client muting said content responsive to said further user answering said call. Preferably, the method further comprises the step of downloading the list of contacts from a network element. Preferably, said step of retrieving the message comprises communicating with a further client executed at a user terminal of the further user. Preferably, said message is a mood message created by said further user.
Preferably, said step of displaying the content at said user terminal comprises said client executing media player means at said user terminal, wherein said content is displayed to said user using said media player means. Preferably, said media player means is embedded within said client. Preferably, said media player means comprises user-operable controls for controlling playback and setting audio properties of said content.
Preferably, the method further comprises the step of ascertaining whether said reference to said content is listed in a database prior to displaying said content. Preferably, if said reference to said content is listed in said database, said user terminal does not display said content. Preferably, said reference is a network address of said content. Preferably, said network address is a uniform resource identifier.
Preferably, said communication network is a packet based communication network. In one embodiment, said client is a voice over internet protocol communication client. Preferably, said voice over internet protocol communication client is a peer-to-peer communication client. In another embodiment, said client is an instant messaging communication client.
In one embodiment, said user terminal is a personal computer. In another embodiment, said user terminal is a mobile device. According to another aspect of the present invention there is provided a computer program product comprising program code means which when executed by a computer implement the steps according to the above-defined method of accessing content at a user terminal.
According to another aspect of the present invention there is provided a method of accessing content at a user termina! connected to a communication network and executing a communication client, comprising: said client displaying a list of contacts associated with a user of said client; said client retrieving a message from said communication network, wherein said message is related to a further user represented by one of said contacts displayed in said list of contacts, said message comprising a reference to content stored in a storage means accessible by said communication network; said client receiving a communication request initiated by said further user over the communication network; and responsive to receiving said communication request, said client establishing communication with said storage means using said reference, accessing the content and displaying the content at said user terminal.
Preferably, said communication request is an incoming call.
According to another aspect of the present invention there is provided a computer program product comprising program code means which when executed by a computer implement the steps according to the above-defined method of accessing content at a user terminal.
According to another aspect of the present invention there is provided a user terminal comprising: communication means for communicating with a communication network; display means; and processing means arranged to execute a communication client, wherein said client is configured to: display a list of contacts associated with a user of said client on said display means; retrieve a message from said communication network, wherein said message is related to a further user represented by one of said contacts displayed in said list of contacts, said message comprising a reference to content stored in a storage means accessible by said communication network; initiate a call to said further user over the communication network responsive to a user of said client selecting said one of said contacts in said list of contacts; and, responsive to initiating said call, establish communication with said storage means using said reference, access the content and display the content on said display means.
According to another aspect of the present invention there is provided a user terminal comprising: communication means for communicating with a communication network; display means; and processing means arranged to execute a communication client, wherein said client is configured to: display a list of contacts associated with a user of said client on said display means; retrieve a message from said communication network, wherein said message is related to a further user represented by one of said contacts displayed in said list of contacts, said message comprising a reference to content stored in a storage means accessible by said communication network; receive a communication request initiated by said further user over the communication network; and, responsive to receiving said communication request, establish communication with said storage means using said reference, access the content and display the content at said user terminal.
For a better understanding of the present invention and to show how the same may be put into effect, reference will now be made, by way of example, to the following drawings in which:
Figure 1 shows a P2P communication system;
Figure 2 shows a user interface of a client executed on a user terminai;
Figure 3 shows a detailed view of a user terminal executing a client;
Figures 4A to 4C show user interfaces for entering a mood message in a client;
Figure 5A shows a user interface for adding a multimedia mood message;
Figure 5B shows a user interface for selecting a video to add to a mood message;
Figure 6A shows multimedia mood message data; Figure 6B and 6C shows a user interface containing a multimedia mood message;
Figure 7 shows a client user interface comprising a contact with a multimedia mood message; Figures 8A to 8D show client user interfaces in which a video ringback is displayed;
Figures 9A to 9B show client user interfaces in which a video ringback can be paused;
Figures 1OA to 1 OC show client user interfaces with control options for a video ringback;
Figure 11 shows a client user interface with options displayed following the end of a video;
Figure 12 shows a client user interface with a message displayed if a video ringback affects call quality; Figure 13 shows a client user interface with a resized video;
Figure 14 shows a client user interface for an unavailable video; and
Figure 15 shows a client user interface displaying a video during an incoming call.
Reference is first made to Figure 1 , which illustrates a P2P communication system 100. Note that whilst this illustrative embodiment is described with reference to a P2P communication system, other types of communication system could also be used, such as instant messaging systems and other, non-P2P, VoIP systems. A first user of the P2P communication system (denoted "User A" 102) operates a user terminal 104, which is shown connected to a network 106, such as the Internet. The user terminal 104 may be, for example, a persona! computer ("PC"), personal digital assistant ("PDA"), a mobile phone, a gaming device or other embedded device able to connect to the network 106. The user device is arranged to receive information from and output information to a user of the device. In a preferred embodiment of the invention the user device comprises a display such as a screen and a keyboard and mouse. The user device 104 is connected to the network 106 via a network interface 108 such as a modem, and the connection between the user terminal 104 and the network interface 108 may be via a cable (wired) connection or a wireless connection.
The user terminal 104 is running a client 110, provided by the P2P software provider. The client 110 is a software program executed on a local processor in the user terminal 104. The user terminal 104 is also connected to a handset
112, which comprises a speaker and microphone to enable the user to listen and speak in a voice call. The microphone and speaker does not necessarily have to be in the form of a handset, but can be in the form of a headphone or earphone with an integrated microphone, or as a separate loudspeaker and microphone independently connected to the user terminal 104.
An exampie of a user interface 200 of the client 110 executed on the user terminal 104 of User A 102 is shown illustrated in Figure 2. The ciient user interface 200 displays the username 202 of User A 102 in the P2P system, and User A can set his own presence state (that will be seen by other users) using a drop down list by selecting icon 204.
The client user interface 200 comprises a tab 206 labelled "contacts", and when this tab is selected the contacts stored by the user in a contact list are displayed. In the example user interface in Figure 2, five contacts of other users of the P2P system (User B to F) are shown listed in contact list 208.
Each of these contacts have authorised the user of the client 110 to view their contact details and online presence and mood message information. Each contact in the contact list has a presence status icon associated with it. For example, the presence status icon for User B 210 indicates that User B is
"online", the presence icon for User C 212 indicates that User C is "not available", the presence icon for User D 214 indicates that User D's state is
"do not disturb", the presence icon for User E 216 indicates User E is "away", and the presence icon for User F 218 indicates that User F is "offline". Further presence indications can also be included. Next to the names of the contacts in pane 208 are the mood messages 220 of the contacts. The contact list for the users (e.g. the contact list 208 for User A) is stored in a contact server 113 shown in Figure 1. When the client 110 first logs into the P2P system the contact server 113 is contacted, and the contact list is downloaded to the user terminal 104. This allows the user to log into the P2P system from any terminal and still access the same contact list. The contact server is also used to store the user's own mood message (e.g. the mood message of User A 102) and a picture selected to represent the user (known as an avatar). This information can be downloaded to the client 110, and allows this information to consistent for the user when logging on from different terminals. The client 110 also periodically communicates with the contact server 113 in order to obtain any changes to the information on the contacts in the contact list, or to update the stored contact list with any new contacts that have been added. Presence information is not stored centrally in the contact server. Rather, the client 110 periodically requests the presence information for each of the contacts in the contact list 208 directly over the P2P system. Similarly, the current mood message for each of the contacts, as well as a picture (avatar) that has been chosen to represent the contact, are also retrieved by the client 110 directly from the clients of each of the contacts over the P2P system.
Calls to the P2P users in the contact list may be initiated over the P2P system by selecting the contact and clicking on a "call" button 222 using a pointing device such as a mouse. Alternatively, the call may be initiated by typing in the P2P identity of a contact in the field 224. Referring again to Figure 1, the call set-up is performed using proprietary protocols, and the route over the Internet 106 between the calling user and called user is determined by the peer-to-peer system without the use of servers. In Figure 1 , an illustrative route is shown between the caller User A (102) and the called party, User B (114), via other peers (116, 118, 120) of the P2P system. It will be understood that this route is merely an example, and that the call may be routed via fewer or more peers.
Following authentication through the presentation of digital certificates (to prove that the users are genuine subscribers of the P2P system - described in more detail in WO 2005/009019), the call can be made using VoIP. The client 110 performs the encoding and decoding of VoIP packets. VoIP packets from the user terminal 104 are transmitted into the Internet 106 via the network interface 108, and routed to the computer terminal 122 of User B 114, via a network interface 123. A client 124 (similar to the client 110) running on the user terminal 122 of User B 114 decodes the VoIP packets to produce an audio signal that can be heard by User B using the handset 126. Conversely, when User B 114 talks into handset 126, the client 124 executed on user terminal 122 encodes the audio signals into VoIP packets and transmits them across the Internet 106 to the user terminal 104. The client 110 executed on user terminal 104 decodes the VoIP packets from User B 114, and produces an audio signal that can be heard by the user of the handset 112.
The VoIP packets for the P2P call described above are passed across the Internet 106 only, and the PSTN network is not involved. Furthermore, due to the P2P nature of the system, the actual voice calls between users of the P2P system can be made with no servers being used. This has the advantages that the system scales easily and maintains a high voice quality, and the call can be made free to the users.
Figure 3 illustrates a detailed view of the user terminal (104) on which is executed client 110. The user terminal 10 comprises a central processing unit ("CPU") 302, to which is connected a display 304 such as a screen, an input device such as a keyboard 306, a pointing device such as a mouse 308, a speaker 310 and a microphone 312. The speaker 310 and microphone 312 may be integrated into a handset 112 or headset, or may be separate. The CPU 302 is connected to a network interface 108 as shown in Figure 1.
Figure 3 also illustrates an operating system ("OS") 314 executed on the CPU 302. Running on top of the OS 314 is a software stack 316 for the client 112. The software stack shows a protocol layer 322, a client engine layer 320 and a client user interface layer ("Ul") 318. Each layer is responsible for specific functions. Because each layer usually communicates with two other layers, they are regarded as being arranged in a stack as shown in Figure 3. The operating system 314 manages the hardware resources of the computer and handles data being transmitted to and from the network via the network interface 108. The client protocol layer 322 of the client software communicates with the operating system 314 and manages the connections over the P2P system. Processes requiring higher level processing are passed to the client engine layer 320, which handles the processing required for the user to make and receive calls over the P2P system. The client engine 320 also communicates with the client user interface layer 318. The client engine 320 may be arranged to control the client user interface layer 318 to present information to the user via the user interface of the client (as shown in Figure 2) and to receive information from the user via the user interface. The control of the client user interface 318 will be explained in more detail hereinafter.
Reference is now made to Figures 4A to 4C, which illustrate the known user interface for entering a mood message into a client. Figure 4A shows a portion of the client user interface for User A 102 (as shown in Figure 2) that is used by User A to set his mood message. The panel 402 is shown when User A uses the pointing device 308 of the user terminal 104 to select the field displaying his username (labelled 202 in Figure 2) in the client user interface (200 in Figure 2). Panel 402 displays the username 202 and presence icon 204 for User A (as in Figure 2). In addition, panel 402 also displays an avatar 404 that User A has selected to represent himself, and a text-entry box 406 in which User A can enter a mood message.
When User A uses the pointing device 308 of the user terminal 104 to select the text-entry box 406, the panel shown in Figure 4B is displayed. The text- entry box 406 is displayed in a different colour, and a cursor 408 is present in the text-entry box 406 to indicate to the user that text may be typed.
After User A has typed a mood message into text-entry box 406, then the panel 402 is of the form shown in Figure 4C. In this example, User A has typed the text message "This is a sample text message" 410 and has also typed a webpage address "http://www.skype.com" 412, which is automatically hyperlinked by the client 112 such that the address 412 becomes a clickable link in the Ul, whereby if the user clicks the link using the pointing device 308, the user terminal 104 executes a web-browser program that navigates to the webpage address and displays the webpage to the user on the display device 304.
The mood message that was typed by User A is transmitted to the contact server 113 and stored in User A's data record. In alternative embodiments, the mood message data is not transmitted immediately to the contact server 113, but is sent with the next periodic update message from the client 110 to the contact server 113. In addition, the mood message of User A 102 is also periodically communicated to each of User As contacts over the P2P system when this is retrieved by the contacts.
Reference is now made to Figure 5A1 which illustrates the Ul for adding a multimedia mood message. In the foliowing embodiment, a video is incorporated into a user's mood message. However, it will be appreciated that the same technique may be used for incorporating other types of media into mood messages, such as images and sound recordings.
Figure 5A shows a portion of the client Ul for User A 102 similar to that shown in Figure 4A. Panel 502 comprises a mood message entry area 504 which displays a different message to that in Figure 4A. In particular, the message invites the user to enter a text message or to add a video into their mood message, and has a hyperlink embedded for the word "video" 506. The user therefore not only has the option to write a text message, but also to add a video to their mood message. Note that in alternative embodiments, other types of media (such as still images or audio) may also be added to mood messages. In this instance, the hyperlink in Figure 5A is changed accordingly, for example to read "video or photo".
If the user uses the pointing device to click in the mood message entry area 504 (but not on the word "video") then a cursor is shown as in Figure 4B, and the user can type a text message, as in Figure 4C, which is then distributed to the contacts of User A as described previously. If the user uses the pointing device to click on the word "video", then the user begins the process for seiecting a video for insertion into the mood message.
In preferred embodiments, the selection of a video is made by the user through the presentation of a Ul showing a selection of available videos, which is displayed to the user responsive to the user clicking on the word "video". An example of such a video selection Ul 508 is illustrated in Figure 5B. The Ul 508 can be considered to comprise two separate elements. The first element is the frame of the U! 508, comprising tabs (510, 512, 514), navigation buttons 516, search field 518, mood message preview area 520 and control buttons (522, 524). The second element is a pane 526 which shows different information for each of the tabs (510, 512, 514). Specifically, pane 526 displays representations of the available videos,
The information for the above-two elements in Ul 508 is fetched from different sources. Firstly, the information relating to the frame of the Ul is fetched from a content directory 127. The content directory 127 is shown in Figure 1 connected to the network 106. The client 110 is arranged to communicate with the content directory 127 and retrieve the information to allow the client 110 to present the Ul 508 to the user. In particular, the client 110 needs to fetch the information regarding the tabs (510, 512, 514). Specifically, information is needed on the number of tabs (three are shown in Figure 5B, but more or less tabs may be shown - for example if there is only a single content provider then no distinct tabs are required), the titles of the tabs, and the address of where the information to populate pane 526 for each tab is located. Preferably, this address is in the form of a webpage uniform resource locator ("URL").
Once the information regarding the frame of Ul 508 has been retrieved from the content directory 127, the information for pane 526 is retrieved from a content provider. Figure 1 illustrates three separate content providers (128, 130, 132) connected to network 106. In this exemplary embodiment, these three content providers (128, 130, 132) respectively provide the information displayed for the three tabs (510, 512, 514) of Ul 508. The content providers (128, 130, 132) provide the information that populates pane 526 and also provide the actual video data. Preferably, the content providers are third parties, and separate from the P2P software provider. Note that the media content that is made available by the content provider can be provided by individual users uploading the media to the content provider.
In the example shown in Figure 5B, the Ul 508 shows the content for tab 510, labelled "Partner 1". The client uses the address obtained from the content directory 127 for tab 510 to access content provider 128 in order to display the information for this tab. More specifically, the address from the content directory 127 is a URL of a webpage provided by the content provider 128, and the client downloads this webpage (in the form of hypertext mark-up language ("HTML")) over the network 106 and displays the webpage in pane 526. If the user uses the pointing device to select a different tab, such as 512 or 514, then the address provided by the content directory 127 for the selected tab is used to fetch the information to be displayed for this tab.
The purpose of the information shown in pane 526 is to provide the user with a selection of videos that are available to be included in the user's mood message. Pane 526 displays a plurality of thumbnail images (e.g. thumbnail 528) representing each video and a title (e.g. title 530) for each video. The thumbnail image for a video is a small image of a single frame from the video. The thumbnail for each video is generated from the video file itself (e.g. it can be the first frame of the video, or a frame a predetermined time or percentage into the video). Preferably, a list of categories 532 is displayed to allow the user to view a sub-set of the available videos. Furthermore, the user can search the videos by keywords using the search field 518.
Preferably, the content directory 127 is provided by the P2P software provider. This allows the P2P software provider to present a portal to several content providers (128, 130, 132) that are able to provide videos to the users, thereby avoiding the requirement for the P2P software provider to host and/or stream its own videos. The content directory 127 can be updated to add or remove content providers as required.
When L)I 508 is displayed on the users terminal 104, the user can use the pointing device to browse the available videos from the different content providers (by using the different tabs 510-514, the categories 532 and the search field 518). When a user sees a video that he is interested in, he clicks the thumbnail to view a preview of the video. When the user has chosen a video, he can select the "save mood message" button 522 to save the video to his mood message.
In an alternative embodiment, the user can also add a video to his mood message without using the video selection Ul. For example, the user can copy and paste a URL of a video on a webpage directly into his mood message (e.g. into the field illustrated in Figure 4B). The client detects that this URL is linking to a video (or other type of media), and automatically generates the required multimedia mood message data (described with reference to Figure 6A below) in order for the video mood message to be created for this video. In some embodiments, the client can also check that the video URL is from an approved partner (e.g. one of the content providers 128, 130, 132).
in a further alternative embodiment, the user can additionally add a video to his mood message by selecting a control displayed on the webpage of a content provider in association with a particular video. For example, the webpage of a content provider (128, 130, 132) can display videos that can be viewed by a user using a web-browser. The webpage displays an "add to mood" button in proximity with the videos, such that if the button is activated by the user, then data regarding the video is passed to the client to allow the client to generate the required mood message data. Therefore, the client can create a multimedia mood message in the same way as described previously, but without using the video selection Ul.
In yet further embodiments, the user can also add videos from a webpage, even if the webpage is not from a content provider that displays a specific "add to mood" button as part of the webpage itself. The P2P software provider can provide plug-in software for the web-browser program that detects that the webpage being viewed contains an embedded video, and overlays an "add to mood" button in proximity to the video in the webpage. If the user selects the button, then data is passed to the client that allows the client to generate the multimedia mood message data in the same way as described above. In preferred embodiments, the client can check that the webpage is from an approved partner (e.g. one of the content providers 128, 130, 132) before permitting the video to be included in a mood message.
Further details regarding the process by which the user creates a multimedia mood message is the subject of a co-pending patent application entitled "Multimedia Mood Messages" by the same applicant (identified by attorney reference number 313765. GB), and is not discussed further here.
Once the multimedia mood message has been chosen by the user, the client generates mood message data to enable the mood message to be displayed in the client and passed to the contacts of the user. The mood message data 602 that is generated by the client is illustrated in Figure 6A. The mood message data 602 that is generated and stored by the client comprises two main parts. The first part is a media object 604, which contains the data related to the media (e.g. the video). The second part of the mood message data 602 is a text comment 606 that the user has typed to accompany the video (if present).
The media object 604 comprises several data items. A media type field 608 defines the type of media (e.g. video, photo, audio etc.) A media title field 610 includes the title of the media. A content provider identity 612 is used to identify which content provider the media originated from (e.g. 128, 130 or 132 in Figure 1). A content provider uniform resource identifier ("URI") 614 provides an address of where further information may be found regarding the content provider (e.g. the address of the content provider webpage). A media URI 616 is the address of where the media is stored, and this is the address from where the media is streamed when it is played. The thumbnail image for the media is stored in the media object at 618. Preferably, the thumbnail image is provided by the content provider.
The multimedia mood message then appears to the user in the client Ul as shown in Figure 6B and 6C. Figure 6B shows the same panel 502 of the client Ul as illustrated in Figure 5A. However, the panel 502 now includes the thumbnail image 620 for the video selected by the user (this is read from the thumbnail image 618 in the media object 604). The user also has the option to add a text comment at this stage to accompany the video (if this was not done previously, or if the user wishes to amend a previously entered comment). This is achieved by the user clicking in the text-entry region 622 and typing. Figure 6C illustrates the case where the user has typed a message 624 in region 622.
Therefore, at this stage, the user (in this case User A) has selected a video to be included in his mood message, and generated corresponding multimedia mood message data 602. The next stage is for the multimedia mood message to be passed to User A's contacts.
Referring again to Figure 1 , the video mood message data 602 as shown in Figure 6A is transmitted from the client 110 to the clients of the contacts of User A over the P2P system. According to one embodiment, the multimedia mood message data 602 for User A is retrieved periodically by the contacts of User A 102. For example, the client 124 of User B 114 (who is a contact of User A) may periodically request mood message information from each of its contacts, thereby requesting the mood message information from User A. The multimedia mood message information is transmitted from User A to User B responsive to such a request. In an alternative embodiment, the multimedia mood message data 602 is transmitted to each of the contacts of User A as soon as the mood message is created or changed (i.e. it is pushed to the contacts immediately), in a further alternative embodiment, the multimedia mood message data 602 is pushed to the contacts at periodic intervals. As User B 114 is a contact of User A1 the multimedia mood message data 602 is provided to the client 124 of User B over the P2P system. The multimedia mood message data must be processed by the client 124 before the mood message is displayed to User B. The client 124 reads the mood message data 602 and analyses the media object 604. For example, the client 124 will read the media type field 608 to determine the type of media referred to in the media object 604 (e.g. a video in this example). The client will also extract the thumbnail 618 so that it may be displayed in the Ul of the client 124 next to the contact entry for User A. The media URI 616 is extracted and hyperlinked to the thumbnail image in the client Ul. The media title 610 is also read for display in the client Ul.
The result of the multimedia mood message data 602 being displayed in User B's client is illustrated in Figure 7. Figure 7 shows the client Ul 700 for User B with the contact for User A 702 expanded. The contact for User A 702 shows User A's name 704, presence state 706, avatar 708, and buttons for initiating an IM chat 710 or voice call 712 with User A. Next to the contact 702 for User A is displayed a thumbnail image 714 of the video chosen by User A for his mood message. The thumbnail image 714 is extracted from the thumbnail field 618 of the multimedia mood message data 602. The thumbnail image 714 also has a play button 716 overlaid on it, to indicate to the user that the video can be played by clicking on the thumbnail image 714. If User B uses his pointing device to place a pointer over the thumbnail image 714, then the text comment provided with the video (if any) is displayed. Alternatively, if no text comment has been provided, the title of the video is displayed.
As mentioned previously, a problem with User A 102 sharing multimedia content in a mood message is that it relies on User B 114 actively selecting to view the video in the mood message (by activating the play button 716). To counter this, a system and method is utilised whereby the video from User A's mood message is automatically displayed to a contact of User A (e.g. User B) whenever that contact attempts to establish a call with User A. More specifically, during the time between the call being initiated by the contact and the time at which User A answers the call, the video is played to the contact. This technique can be referred to as a "video ringback".
Reference is now made to Figures 8A to 8D, which illustrate the process by which a video ringback is displayed to User B 114 when he is calling User A 102. In this example, User A 102 has created a video mood message, and the mood message data has been passed to User B 114, such that the video mood message is displayed in User B's client Ul 700, next to User A's name (as illustrated in Figure 7). User B 114 initiates a call with User A 102, by selecting the call button 712 in the client Ul 700 (or alternatively using call button 718). The client 124 of User B 114 then attempts to establish a connection with the client 110 of User A 102 over the network 106 in order to set up a voice call.
In addition, the client 124 of User B 114 also detects that the contact being called (User A in this example) has a video mood message. In response to detecting this, the client attempts to start a video ringback. The client 124 reads the mood message data 602, and specifically the media URI 616. The client uses the media URI 616 to contact the location at the content provider (128, 130, 132) where the video is stored, and initiate streaming of the video to the client 124 of User B 114.
The Ul 700 of the client 124 of User B at the beginning of this process is shown illustrated in Figure 8A. Compared to Figure 7, the Ul shown in Figure 8A now displays a call tab 802 (indicating the name of the person being called) instead of the contact list tab. Shown in the call tab 802 is a contact card 804 for the user being called (User A 102). The contact card 804 comprises the name of the callee 806, their presence state 808 and avatar 810, as well as further control buttons 812 that are out of the scope of this description.
Below the contact card is an embedded video player 814. In Figure 8A, the client 124 is loading the video from User A's mood message (using the media URI 616), and this is reflected in the image displayed in the embedded video player 814. The status of the call to User A 102 is shown at 816. In Figure 8A, User A has not yet answered the call, and the status is "connecting".
When the video located at the media URI 616 has been loaded, the video is played back to User B 114, whilst he is waiting for User A 102 to answer the call. This is illustrated in Figure 8B, which shows the client Ul 700 as in Figure 8A, except that the video from User A's mood message is now being played to User B in the embedded video player 814.
Preferably, the cϋent 124 of User B 114 initially plays an audible ringing tone to User B 114 whilst the video begins to be played back, in order to indicate to the user that the call is waiting to be answered. After the ringing tone has been played for a predetermined period of time (e.g. 3 seconds), the ringing tone is faded out, and the sound that accompanies the video is faded in.
When User A 102 answers the call, the video playback is muted, so that the sound of the video is not playing over the top of User A's voice. This is shown in Figure 8C. The embedded video player 814 is showing a message 818 over the top of the video to indicate to User B 114 that the video has been muted. The playback of the video (minus the sound) continues behind the message 818. The status 816 now shows that the cail is connected, and displays the duration of the call.
If User B 114 wishes to unmute the video, so that he can continue to hear the sound of the video even after User A has answered the call, then this can be achieved by User B moving the cursor with the pointing device over the embedded video player in order to display a control bar 820, as shown in
Figure 8D. More detail on the operation of the control bar 820 is described below with reference to Figure 10A to 10C. The control bar 820 comprises a volume control button 822, and activation of this button by User B 114 allows the volume to be restored, so that the sound of the video can be heard.
In alternative embodiments, the client pauses the content responsive to User A answering the call, rather than the content being muted. In this case, the message 818 displayed over the video in Figure 8C indicates that the video is paused. The playback of the video can be resumed by the user by selecting a play button on the control bar 820, as described in more detail with reference to Figure 1OA to 1OC, below.
Therefore, the above-described process ensures that a video shared by User A 102 in his mood message is seen by his contacts, due to it being played automatically to them during a time when otherwise only a normal ringing tone would be heard.
When User B 114 first makes a call to User A 102, he may be surprised by the video that begins playing automatically. !t is therefore preferable to provide a way for the video to be rapidly paused by the user, if it is inappropriate for it to be played. This is illustrated in Figure 9A and 9B. When the video is being played to User B 114 whilst waiting for User A 102 to answer the call, if the cursor is brought anywhere over the embedded video player 814 then a message 824 is displayed to User B 114 indicating that the video can be paused by clicking on the playing video. This therefore gives the user a large region to click on with the pointing device, allowing the user to rapidly pause the video if required.
The result of the user clicking on the video with the pointing device is shown in Figure 9B. The playback of the video has been paused, which is indicated to the user with message 826. The control bar 820 is displayed to the user to allow him to resume playback if desired.
Reference is now made to Figure 10A to 1OC, which illustrate in more detail the functions of the control bar 820. At any point after the video has been initially paused by clicking on the video (as described above with reference to Figure 9A and 9B) or during a call, if the cursor is placed over the embedded video player 814, then the control bar 820 is displayed at the bottom of the video. The control bar 820 comprises a play/pause button 828, a progress bar 830, a volume button 822 and a menu button 832. The play/pause button toggles between playing and pausing the video, and the symbol shown on the button changes accordingly, depending on the current state of the video (i.e. whether it is currently playing or paused). When the volume button 822 is activated, a volume slider is shown to allow the user to change the volume, or the sound can be muted. When the video is either paused, muted, or both, a message is shown overlaid on the video. Figure 1OB shows an example of this for the case where the video is both paused and muted, as reflected in message 834.
Figure 10C shows the options that are presented to User B 114 when the menu button 832 is selected. The first option is a "chat about this" 836 option. If the "chat about this" option 836 is activated, a dialogue box is opened that displays to User B a list of all his contacts. User B can select from this list the contacts with which he wishes to initiate an IM chat conversation. When he has selected the contacts, an IM chat session is established with these contacts, and the video from the embedded video player 814 is automatically inserted into the beginning of the IM chat conversation.
The second option is a "add to my mood" option 838. If this option is selected, then the video that is shown in the embedded video player 814 (i.e. video from User A's mood message) is added to User B's mood message.
The third option is a "more info" option 840. The "more info" option provides the user with more information regarding the video. Preferably, this is provided by the client 124 executing a web browser and navigating to a webpage of the content provider, the address of which was provided in the content provider URI 614 field of the mood message data 602.
The fourth option is an "auto-play video" option 842. Preferably, this option is enabled by default. When enabled, this option allows videos that are in a callee's mood message to be played automatically when a call is made to the cailee (as described with reference to Figures 8A to 8D). If this option is disabled, then the video is not played automatically on making the call, but instead a static frame of the video is shown with the control bar 820 below it, allowing the video to be played if desired. Furthermore, the client 124 preferably monitors the user behaviour, so that if the user pauses the ringback video a predetermined number of times consecutively, then the client will pop- up a message to the user, suggesting that the "auto-play video" option be disabled.
In further embodiments, the menu button 832 can also present a fifth option (not shown in Figure 10C). The fifth option allows the user to report problematic content to the content provider, if this option is selected, then a web browser program is executed on the user's terminal, which navigates to a webpage comprising a form which the user can fill out to report a problem with the content (e.g. copyrighted material, unsuitable subject-matter, etc). The information entered in this form is provided to the content provider, who can choose to remove the video in response thereto.
Reference is now made to Figure 11 , which shows the client Ul 700 when the video has completed playing. Two buttons are displayed to the user, overlaid over the video. The first is a "play again" button 844 that allows the user to playback the video from the start. Activation of the play/pause button 828 also has the same effect. The second button is a "chat about this" button 846, which has the same function as selecting the "chat about this" option 836 from the menu button 832 as described above.
Playback of the video to the user has requirements both in terms of processing power (i.e. CPU resources) and on the network connection of the user's terminal (i.e. bandwidth resources). As a result of this, it can sometimes be found that playback of the video after the called user has answered can affect the quality of the call. The client monitors the CPU usage, bandwidth, and/or call quality, and if it is determined that the video is detrimentally affecting these aspects, then the video playback is stopped. This is reported to the user as shown in Figure 12, using message 848. Furthermore, during a call, User B 114 may need to use controls related to the voice call, such as adjusting sound settings, placing the call on hold or muting his microphone. The controls require space on the call tab 802. If any of these controls are activated and there is insufficient room to accommodate the controls, then the video is paused and reduced in size as shown in Figure 13. In this example, the user has opened an audio settings panel 850, and this has resulted in the size of the video being reduced accordingly.
If the client 124 is unable to play the video from the media URl 616 provided by User A's mood message (e.g. due to the video being removed, or a network failure), then an error message is shown to the user. This is illustrated in Figure 14, where the embedded video player 814 shows a "video unavailable" message.
In preferred embodiments, before the video is displayed to User B, the client 124 will perform a check to ensure that the media URI 616 included in the mood message does not relate to a video that has been "blacklisted". A video may be blacklisted if, for example, it is found to infringe copyright or contains offensive material. Referring to Figure 1 , a blacklist advertiser 134 is shown connected to network 106, and connected to the blacklist advertiser is a blacklist database 136. Note that the blacklist database can be a centralised database or a distributed P2P database. If any videos are "blacklisted", then the address (i.e. media UR!) of the video is stored in the blacklist DB 136. When a user attempts to access a given media URI, the client first sends a message to the blacklist advertiser 134 containing the media URI. The blacklist advertiser 134 compares the media URl sent by the client with those listed in the blacklist DB 136.
If the media URI matches one listed in the blacklist DB 136, then blacklist advertiser 134 sends a message to the client indicating that the video has been blacklisted, in response to this the client does not display the media, but instead displays a notification of the problem. Conversely, if the media URI 616 is not listed in the blacklist DB 136, then the blacklist advertiser 134 sends a message to indicate to the client that the media can be displayed to the user.
In preferred embodiments, in addition to the video in User A's mood message being played to User B when User B is waiting for User A to answer a call, if
User B also has a video mood message, then this video can be displayed to
User A as part of the incoming call alert. In other words, if both the users have video mood messages, then the video of User A is displayed to User B, and the video of User B is displayed to User A. This is illustrated with reference to Figure 15.
Figure 15 shows the client Ul 200 for User A 102 (as shown in Figure 2). However, in contrast to Figure 2, a call is being made from User B 114 to User A 102. This causes a call tab 1502 to be shown (rather than the contacts tab 206 of Figure 2). The call tab comprises the contact card 1504 of the user calling User A, which displays the callers name 1506, presence state 1510 and avatar 1510. A status field 1512 indicates that this is an incoming call from User B. An embedded video player 1514 displays the video from User B's mood message.
The video in the embedded video player 1514 is displayed by the client 110 using the media URI from User B's mood message to stream the video from the content provider (128, 130, 132), in the same manner as described above with reference to Figures 8A to 8D. When User A 102 answers the call, the video displayed to User A in the embedded video player 1514 is muted (in a similar manner to Figure 8C) but the video continues to play.
User A has the same controls over the video shown when there is an incoming call as described above for User B when placing a call. In particular, the same control bar and menu options are available as described above with reference to Figure 10A to 10C.
Therefore, the above described system and method ensures that multimedia content shared by users is shown to their contacts in two ways. Firstly, a caller is shown the multimedia content of the called user when he is waiting for the called user to answer the cali. Secondly, the called user is shown the multimedia content of the caller when the incoming call is displayed in the client.
While this invention has been particularly shown and described with reference to preferred embodiments, it will be understood to those skiiled in the art that various changes in form and detail may be made without departing from the scope of the invention as defined by the appendant claims.

Claims

CLAIMS:
1. A method of accessing content at a user terminal connected to a communication network and executing a communication client, comprising: said client displaying a list of contacts associated with a user of said client; said client retrieving a message from said communication network, wherein said message is related to a further user represented by one of said contacts displayed in said list of contacts, said message comprising a reference to content stored in a storage means accessible by said communication network; said client initiating a call to said further user over the communication network responsive to a user of said client selecting said one of said contacts in said list of contacts; and responsive to initiating said call, said client establishing communication with said storage means using said reference, accessing the content and displaying the content at said user terminal.
2. A method according to claim 1 , wherein said content is a video.
3. A method according to claim 1 , wherein said content is an audio recording.
4. A method according to any preceding claim, further comprising the step of said ciient muting said content responsive to said further user answering said call.
5. A method according to any preceding claim, further comprising the step of downloading the list of contacts from a network element.
6. A method according to any preceding claim, wherein said step of retrieving the message comprises communicating with a further client executed at a user terminal of the further user.
7. A method according to any preceding claim, wherein said message is a mood message created by said further user.
8. A method according to any preceding claim, wherein said step of displaying the content at said user terminal comprises said client executing media player means at said user terminal, wherein said content is displayed to said user using said media player means.
9. A method according to claim 8, wherein said media player means is embedded within said client.
10. A method according to claim 8 or 9, wherein said media player means comprises user-operable controls for controlling playback and setting audio properties of said content.
11. A method according to any preceding claim, further comprising the step of ascertaining whether said reference to said content is listed in a database prior to displaying said content.
12. A method according to claim 11 , wherein if said reference to said content is listed in said database, said user terminal does not display said content.
13. A method according to any preceding claim, wherein said reference is a network address of said content.
14. A method according to claim 13, wherein said network address is a uniform resource identifier.
15. A method according to any preceding claim, wherein said communication network is a packet based communication network.
16. A method according to any preceding claim, wherein said client is a voice over internet protocol communication client.
17. A method according to claim 16, wherein said voice over internet protocol communication client is a peer-to-peer communication client.
18. A method according to any of claims 1 to 15, wherein said client is an instant messaging communication client.
19. A method according to any preceding claim, wherein said user terminal is a personal computer.
20. A method according to any of claims 1 to 18, wherein said user terminal is a mobile device.
21. A computer program product comprising program code means which when executed by a computer implement the steps according to the method of any of claims 1 to 20.
22. A method of accessing content at a user terminal connected to a communication network and executing a communication client, comprising: said client displaying a list of contacts associated with a user of said client; said client retrieving a message from said communication network, wherein said message is related to a further user represented by one of said contacts displayed in said iist of contacts, said message comprising a reference to content stored in a storage means accessible by said communication network; said client receiving a communication request initiated by said further user over the communication network; and responsive to receiving said communication request, said client establishing communication with said storage means using said reference, accessing the content and displaying the content at said user terminal.
23. A method according to claim 22, wherein said communication request is an incoming call.
24. A method according to claim 22 or 23, wherein said content is a video.
25. A method according to claim 22 or 23, wherein said content is an audio recording.
26. A method according to any of claims 22 to 25, further comprising the step of said client muting said content responsive to said user answering said communication request.
27. A method according to any of claims 22 to 26, further comprising the step of downloading the list of contacts from a network element.
28. A method according to any of claims 22 to 27, wherein said step of retrieving the message comprises communicating with a further client executed at a user terminal of the further user.
29. A method according to any of claims 22 to 28, wherein said message is a mood message created by said further user.
30. A method according to any of claims 22 to 29, wherein said step of displaying the content at said user terminal comprises said client executing media player means at said user terminal, wherein said content is displayed to said user using said media player means.
31. A method according to claim 30, wherein said media player means is embedded within said client.
32. A method according to claim 30 or 31 , wherein said media player means comprises user-operable controls for controlling playback and setting audio properties of said content.
33. A method according to any of claims 22 to 32, further comprising the step of ascertaining whether said reference to said content is listed in a database prior to displaying said content.
34. A method according to claim 33, wherein if said reference to said content is listed in said database, said user terminal does not display said content.
35. A method according to any of claims 22 to 34, wherein said reference is a network address of said content.
36. A method according to claim 35, wherein said network address is a uniform resource identifier.
37. A method according to any of claims 22 to 36, wherein said communication network is a packet based communication network.
38. A method according to any of claims 22 to 37, wherein said client is a voice over internet protocol communication client.
39. A method according to claim 38, wherein said voice over internet protocol communication client is a peer-to-peer communication client.
40. A method according to any of claims 22 to 37, wherein said client is an instant messaging communication client.
41. A method according to any of claims 22 to 40, wherein said user terminal is a personal computer.
42. A method according to any of claims 22 to 40, wherein said user terminal is a mobile device.
43. A computer program product comprising program code means which when executed by a computer implement the steps according to the method of any of claims 22 to 42.
44. A user terminal comprising: communication means for communicating with a communication network; display means; and processing means arranged to execute a communication client, wherein said client is configured to: display a list of contacts associated with a user of said client on said display means; retrieve a message from said communication network, wherein said message is related to a further user represented by one of said contacts displayed in said list of contacts, said message comprising a reference to content stored in a storage means accessible by said communication network; initiate a call to said further user over the communication network responsive to a user of said client selecting said one of said contacts in said list of contacts; and, responsive to initiating said call, establish communication with said storage means using said reference, access the content and display the content on said display means.
45. A user terminal according to claim 44, wherein said content is a video.
46. A user terminal according to claim 44, wherein said content is an audio recording.
47. A user terminal according to any of claims 44 to 46, wherein said client is further configured to mute said content responsive to said further user answering said call.
48. A user terminal according to any of claims 44 to 47, wherein said client is further configured to download the list of contacts from a network element.
49. A user terminal according to any of claims 44 to 48, wherein said client is configured to retrieve the message by communicating with a further client executed at a user terminal of the further user.
50. A user terminal according to any of claims 44 to 49, wherein said message is a mood message created by said further user.
51. A user terminal according to any of claims 44 to 50, wherein said client is further configured to execute media player means at said user terminal, wherein said content is displayed to said user using said media player means.
52. A user terminal according to claim 51 , wherein said media player means is embedded within said client.
53. A user terminal according to claim 51 or 52, wherein said media player means comprises user-operable controls for controlling playback and setting audio properties of said content.
54. A user terminal according to any of claims 44 to 53, said client is further configured to ascertain whether said reference to said content is listed in a database prior to displaying said content.
55. A user terminal according to claim 54, wherein if said reference to said content is listed in said database, said user terminal does not display said content.
56. A user terminal according to any of claims 44 to 55, wherein said reference is a network address of said content.
57. A user terminal according to claim 56, wherein said network address is a uniform resource identifier.
58. A user terminal according to any of claims 44 to 57, wherein said communication network is a packet based communication network.
59. A user terminal according to any of ciaims 44 to 58, wherein said client is a voice over internet protocol communication client.
60. A user terminal according to claim 59, wherein said voice over internet protocol communication client is a peer-to-peer communication client.
61. A user terminal according to any of claims 44 to 58, wherein said client is an instant messaging communication client.
62. A user terminal according to any of claims 44 to 61 , wherein said user terminal is a personal computer.
63. A user terminal according to any of claims 44 to 61 , wherein said user terminal is a mobile device.
64. A user terminal comprising: communication means for communicating with a communication network; display means; and processing means arranged to execute a communication client, wherein said client is configured to: display a list of contacts associated with a user of said client on said display means; retrieve a message from said communication network, wherein said message is related to a further user represented by one of said contacts displayed in said list of contacts, said message comprising a reference to content stored in a storage means accessible by said communication network; receive a communication request initiated by said further user over the communication network; and, responsive to receiving said communication request, establish communication with said storage means using said reference, access the content and display the content at said user terminal.
65. A user terminal according to claim 64, wherein said communication request is an incoming call.
66. A user terminal according to claim 64 or 65, wherein said content is a video.
67. A user terminal according to claim 64 or 65, wherein said content is an audio recording.
68. A user terminal according to any of claims 64 to 67, wherein said client is further configured to mute said content responsive to said user answering said communication request.
69. A user terminal according to any of claims 64 to 68, wherein said client is further configured to download the list of contacts from a network element.
70. A user terminal according to any of claims 64 to 69, wherein said client is configured to retrieve the message by communicating with a further client executed at a user terminal of the further user.
71. A user terminal according to any of claims 64 to 70, wherein said message is a mood message created by said further user.
72. A user terminal according to any of claims 64 to 71 , wherein said client is further configured to execute media player means at said user terminal, wherein said content is displayed to said user using said media player means.
73. A user terminal according to claim 72, wherein said media player means is embedded within said client.
74. A user terminal according to claim 72 or 73, wherein said media player means comprises user-operable controls for controlling playback and setting audio properties of said content.
75. A user terminal according to any of claims 64 to 74, said client is further configured to ascertain whether said reference to said content is listed in a database prior to displaying said content.
76. A user terminal according to claim 75, wherein if said reference to said content is listed in said database, said user terminal does not display said content.
77. A user terminal according to any of claims 64 to 76, wherein said reference is a network address of said content.
78. A user terminal according to claim 77, wherein said network address is a uniform resource identifier.
79. A user terminal according to any of claims 64 to 78, wherein said communication network is a packet based communication network.
80. A user terminal according to any of claims 64 to 79, wherein said client is a voice over internet protocol communication client.
81. A user terminal according to claim 80, wherein said voice over internet protocol communication client is a peer-to-peer communication client.
82. A user terminal according to any of claims 64 to 79, wherein said client is an instant messaging communication client.
83. A user terminal according to any of claims 64 to 82, wherein said user terminal is a personal computer.
84. A user terminal according to any of claims 64 to 82, wherein said user terminal is a mobile device.
EP08735558.2A 2007-07-03 2008-03-28 Communication system and method for displaying contact user content Active EP2174476B1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
GBGB0712878.8A GB0712878D0 (en) 2007-07-03 2007-07-03 Communication system and method
PCT/EP2008/053721 WO2009003734A2 (en) 2007-07-03 2008-03-28 Communication system and method for displaying contact user content

Publications (2)

Publication Number Publication Date
EP2174476A2 true EP2174476A2 (en) 2010-04-14
EP2174476B1 EP2174476B1 (en) 2018-08-15

Family

ID=38421127

Family Applications (1)

Application Number Title Priority Date Filing Date
EP08735558.2A Active EP2174476B1 (en) 2007-07-03 2008-03-28 Communication system and method for displaying contact user content

Country Status (4)

Country Link
US (1) US9584563B2 (en)
EP (1) EP2174476B1 (en)
GB (1) GB0712878D0 (en)
WO (1) WO2009003734A2 (en)

Families Citing this family (22)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8418067B2 (en) * 2004-01-15 2013-04-09 Microsoft Corporation Rich profile communication with notifications
GB0712878D0 (en) 2007-07-03 2007-08-08 Skype Ltd Communication system and method
GB0712877D0 (en) * 2007-07-03 2007-08-08 Skype Ltd Multimedia mood messages
GB2459332A (en) * 2008-04-24 2009-10-28 Skype Ltd Sorting a list of communication events received at a user terminal
GB0820447D0 (en) * 2008-11-07 2008-12-17 Skype Ltd Location information in a communications system
TWI429267B (en) * 2009-02-13 2014-03-01 Htc Corp Method, communication device and computer program product for displaying caller information
US8380872B2 (en) * 2009-12-31 2013-02-19 Nice Systems Ltd. Peer-to-peer telephony recording
GB201014610D0 (en) * 2010-09-02 2010-10-13 Skype Ltd Download logic for web content
GB201014611D0 (en) * 2010-09-02 2010-10-13 Skype Ltd Help channel
US9427661B1 (en) * 2012-03-05 2016-08-30 PlayStudios, Inc. Social networking game with integrated social graph
EP2736231A4 (en) * 2012-06-30 2015-04-15 Huawei Tech Co Ltd Method of dynamic expression display and mobile terminal
US10824297B2 (en) * 2012-11-26 2020-11-03 Google Llc System for and method of accessing and selecting emoticons, content, and mood messages during chat sessions
US9363472B2 (en) * 2013-03-14 2016-06-07 Samsung Electronics Co., Ltd. Video injection for video communication
US9648066B2 (en) * 2014-08-29 2017-05-09 The Boeing Company Peer to peer provisioning of data across networks
US10079795B2 (en) * 2014-12-14 2018-09-18 Sriram Chakravarthy Unified messaging leveraging contacts across existing identities without compromising privacy
US9448763B1 (en) * 2015-05-19 2016-09-20 Spotify Ab Accessibility management system for media content items
WO2016189350A1 (en) * 2015-05-23 2016-12-01 Yogesh Chunilal Rathod Calling to user(s) for real-time sharing, participation, e-commerce, workflow, communication & collaboration in the event of acceptance of call by caller user(s)
DK179034B1 (en) 2016-06-12 2017-09-04 Apple Inc Devices, methods, and graphical user interfaces for dynamically adjusting presentation of audio outputs
JP2019020530A (en) * 2017-07-13 2019-02-07 キヤノン株式会社 Signal processor, control method and program
WO2019054044A1 (en) * 2017-09-14 2019-03-21 ソニー株式会社 Information processing device, information processing method, and program
CN109981907B (en) * 2019-04-04 2022-06-28 腾讯科技(深圳)有限公司 Communication connection establishing method, device, terminal and storage medium
US11610434B2 (en) * 2021-02-02 2023-03-21 T-Mobile Usa, Inc. Emotional response capturing with automatic reply

Family Cites Families (37)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20050055627A1 (en) * 1997-03-07 2005-03-10 Signaturemail.Com, Llc System and method for personalizing electronic mail messages
US6941345B1 (en) * 1999-12-03 2005-09-06 Nortel Networks Limited Real-time, text-based messaging between devices in plural communities
US6603840B2 (en) * 2000-12-06 2003-08-05 At&T Corp. Technique for linking telephony and multimedia information
WO2002091692A1 (en) * 2001-04-13 2002-11-14 Girard Gregory D Ditributed edge switching system for voice-over-packet multiservice network
US6694145B2 (en) 2001-12-27 2004-02-17 Nokia Corporation Synchronization of signaling messages and multimedia content loading
US20030154446A1 (en) * 2002-01-28 2003-08-14 Constant Nicholas Robert Character-based, graphically expressive mobile messaging system
US7689649B2 (en) * 2002-05-31 2010-03-30 Aol Inc. Rendering destination instant messaging personalization items before communicating with destination
US20030225848A1 (en) * 2002-05-31 2003-12-04 Brian Heikes Remote instant messaging personalization items
JP4373324B2 (en) * 2002-05-06 2009-11-25 テレフオンアクチーボラゲット エル エム エリクソン(パブル) Multi-user multimedia message service
US7200636B2 (en) * 2002-11-01 2007-04-03 Sun Microsystems, Inc. Method and apparatus for applying personalized rules to e-mail messages at an e-mail server
US20040162878A1 (en) 2002-11-22 2004-08-19 Lewis Michael Pescatello System and method to facilitate real-time communications and content sharing among users over a network
US20040120316A1 (en) * 2002-12-18 2004-06-24 Mccormack Tony Routing of web-based contacts
WO2005009019A2 (en) 2003-07-16 2005-01-27 Skype Limited Peer-to-peer telephone system and method
CN100375424C (en) * 2004-03-22 2008-03-12 国际商业机器公司 Multimedia information receiving and transmitting method system, gateway and customer's equipment
US7496633B2 (en) * 2004-04-23 2009-02-24 Yahoo! Inc. System and method facilitating a shared content experience using enhanced messaging including a displayable status indicator
US7624149B2 (en) * 2004-09-15 2009-11-24 Microsoft Corporation Instant messaging with audio
US20060168015A1 (en) * 2004-11-30 2006-07-27 Brian Fowler Instant messenger as a web-based communicator
US7616750B2 (en) * 2005-02-25 2009-11-10 Microsoft Corporation Method and system for forwarding telephone calls based on presence information
US7475112B2 (en) * 2005-03-04 2009-01-06 Microsoft Corporation Method and system for presenting a video conference using a three-dimensional object
US20080313056A1 (en) * 2005-06-03 2008-12-18 Shadow Enterprises Inc. Ordering and Image Transmission System and Method Utilizing Instant Messaging
US20060294195A1 (en) * 2005-06-23 2006-12-28 Sony Ericsson Mobile Communcations Ab Multi-media message instant send
DE102005036424B3 (en) 2005-08-03 2006-11-30 Combots Product Gmbh & Co. Kg Instant messaging system, has communication partners, where data is stored for graphical representation of icons in peripheral device, so that communication partner connects with its terminal for producing connection
US20070130327A1 (en) * 2005-12-05 2007-06-07 Kuo Cynthia Y Browser system and method for warning users of potentially fraudulent websites
US20070143414A1 (en) * 2005-12-15 2007-06-21 Daigle Brian K Reference links for instant messaging
US8121626B1 (en) * 2006-06-05 2012-02-21 Callwave, Inc. Method and systems for short message forwarding services
US20070287477A1 (en) * 2006-06-12 2007-12-13 Available For Licensing Mobile device with shakeable snow rendering
EP2047372A4 (en) * 2006-07-10 2010-09-22 Vringo Inc Pushed media content delivery
US8250152B2 (en) * 2006-08-03 2012-08-21 International Business Machines Corporation E-mail delivery options usability tool
US7483969B2 (en) * 2006-10-09 2009-01-27 Microsoft Corporation Managing presence based on relationship
US7995728B1 (en) * 2006-10-31 2011-08-09 Sprint Spectrum L.P. Personal ringback tone content selection and activation
US8413059B2 (en) * 2007-01-03 2013-04-02 Social Concepts, Inc. Image based electronic mail system
US7769951B2 (en) * 2007-04-10 2010-08-03 Yahoo! Inc. Intelligent caching of user data for real time communications
WO2008151033A2 (en) * 2007-05-30 2008-12-11 Iskoot, Inc. Methods and systems for propagating information across a network
US9036621B2 (en) * 2007-06-19 2015-05-19 At&T Intellectual Property I, L.P. System for deploying voice over internet protocol services
GB0712878D0 (en) 2007-07-03 2007-08-08 Skype Ltd Communication system and method
GB0712877D0 (en) * 2007-07-03 2007-08-08 Skype Ltd Multimedia mood messages
US8401169B2 (en) * 2007-12-19 2013-03-19 At&T Intellectual Property I, L.P. System and method of delivering ringback audio content

Also Published As

Publication number Publication date
EP2174476B1 (en) 2018-08-15
US9584563B2 (en) 2017-02-28
US20090013059A1 (en) 2009-01-08
GB0712878D0 (en) 2007-08-08
WO2009003734A3 (en) 2009-04-16
WO2009003734A2 (en) 2009-01-08

Similar Documents

Publication Publication Date Title
EP2174476B1 (en) Communication system and method for displaying contact user content
US8209385B2 (en) Multimedia mood messages
US10938754B2 (en) Instant messaging communication system and method
US8325213B2 (en) Video communication system and method
US20090316688A1 (en) Method for controlling advanced multimedia features and supplemtary services in sip-based phones and a system employing thereof
KR101298956B1 (en) A method and apparatus for providing enhanced features to multicast content services and multiplayer gaming services
KR100964211B1 (en) Method and system for providing multimedia portal contents and addition service in a communication system
US8625751B2 (en) System and method of audible caller identification via a multimedia device
US20070223668A1 (en) Inserting content into a connection using an intermediary
KR100810253B1 (en) Method and system for providing service menu in a communication system
US20170366494A1 (en) Message Delivery System and Method
JP2008252865A (en) Technique for call integration with television set-top box (stb)
JP2007028410A (en) Program, apparatus and method for relaying registration or extraction of voice information to electronic bulletin board
US8862993B2 (en) Audio device control method and apparatus
US20070072648A1 (en) Method and apparatus for identifying a calling party
US20070165800A1 (en) Connection control apparatus, method, and program
US20080162650A1 (en) User-chosen media content
JP2006129379A (en) Image recorded message providing system for cellular phone terminal
JP2005020617A (en) Telephone terminal equipment and message management method for telephone terminal equipment
KR20040055015A (en) A method for providing an avatar along with audio source and a system enabling the method

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

17P Request for examination filed

Effective date: 20100202

AK Designated contracting states

Kind code of ref document: A2

Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MT NL NO PL PT RO SE SI SK TR

AX Request for extension of the european patent

Extension state: AL BA MK RS

111Z Information provided on other rights and legal means of execution

Free format text: AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LT LU LV MC MT NL NO PL PT RO SE SI SK TR

Effective date: 20100304

RAP1 Party data changed (applicant data changed or rights of an application transferred)

Owner name: SKYPE

D11X Information provided on other rights and legal means of execution (deleted)
DAX Request for extension of the european patent (deleted)
17Q First examination report despatched

Effective date: 20160225

GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: GRANT OF PATENT IS INTENDED

INTG Intention to grant announced

Effective date: 20180214

GRAS Grant fee paid

Free format text: ORIGINAL CODE: EPIDOSNIGR3

GRAJ Information related to disapproval of communication of intention to grant by the applicant or resumption of examination proceedings by the epo deleted

Free format text: ORIGINAL CODE: EPIDOSDIGR1

GRAL Information related to payment of fee for publishing/printing deleted

Free format text: ORIGINAL CODE: EPIDOSDIGR3

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: EXAMINATION IS IN PROGRESS

GRAR Information related to intention to grant a patent recorded

Free format text: ORIGINAL CODE: EPIDOSNIGR71

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: GRANT OF PATENT IS INTENDED

GRAA (expected) grant

Free format text: ORIGINAL CODE: 0009210

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE PATENT HAS BEEN GRANTED

INTC Intention to grant announced (deleted)
AK Designated contracting states

Kind code of ref document: B1

Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MT NL NO PL PT RO SE SI SK TR

INTG Intention to grant announced

Effective date: 20180706

REG Reference to a national code

Ref country code: CH

Ref legal event code: EP

Ref country code: GB

Ref legal event code: FG4D

Ref country code: AT

Ref legal event code: REF

Ref document number: 1031099

Country of ref document: AT

Kind code of ref document: T

Effective date: 20180815

REG Reference to a national code

Ref country code: IE

Ref legal event code: FG4D

REG Reference to a national code

Ref country code: DE

Ref legal event code: R096

Ref document number: 602008056489

Country of ref document: DE

REG Reference to a national code

Ref country code: NL

Ref legal event code: FP

REG Reference to a national code

Ref country code: LT

Ref legal event code: MG4D

REG Reference to a national code

Ref country code: AT

Ref legal event code: MK05

Ref document number: 1031099

Country of ref document: AT

Kind code of ref document: T

Effective date: 20180815

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: GR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20181116

Ref country code: NO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20181115

Ref country code: SE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20180815

Ref country code: AT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20180815

Ref country code: IS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20181215

Ref country code: FI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20180815

Ref country code: BG

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20181115

Ref country code: LT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20180815

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: ES

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20180815

Ref country code: HR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20180815

Ref country code: LV

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20180815

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: EE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20180815

Ref country code: PL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20180815

Ref country code: RO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20180815

Ref country code: CZ

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20180815

Ref country code: IT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20180815

REG Reference to a national code

Ref country code: DE

Ref legal event code: R097

Ref document number: 602008056489

Country of ref document: DE

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20180815

Ref country code: DK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20180815

PLBE No opposition filed within time limit

Free format text: ORIGINAL CODE: 0009261

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT

26N No opposition filed

Effective date: 20190516

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20180815

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MC

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20180815

REG Reference to a national code

Ref country code: CH

Ref legal event code: PL

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LU

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20190328

REG Reference to a national code

Ref country code: BE

Ref legal event code: MM

Effective date: 20190331

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: CH

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20190331

Ref country code: LI

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20190331

Ref country code: IE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20190328

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: BE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20190331

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: TR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20180815

REG Reference to a national code

Ref country code: NL

Ref legal event code: PD

Owner name: MICROSOFT TECHNOLOGY LICENSING, LLC; US

Free format text: DETAILS ASSIGNMENT: CHANGE OF OWNER(S), ASSIGNMENT; FORMER OWNER NAME: SKYPE

Effective date: 20200417

REG Reference to a national code

Ref country code: DE

Ref legal event code: R081

Ref document number: 602008056489

Country of ref document: DE

Owner name: MICROSOFT TECHNOLOGY LICENSING LLC, REDMOND, US

Free format text: FORMER OWNER: SKYPE, DUBLIN, IE

Ref country code: DE

Ref legal event code: R082

Ref document number: 602008056489

Country of ref document: DE

Representative=s name: PAGE, WHITE & FARRER GERMANY LLP, DE

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MT

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20190328

Ref country code: PT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20181215

REG Reference to a national code

Ref country code: GB

Ref legal event code: 732E

Free format text: REGISTERED BETWEEN 20200820 AND 20200826

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: CY

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20180815

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: HU

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT; INVALID AB INITIO

Effective date: 20080328

REG Reference to a national code

Ref country code: DE

Ref legal event code: R079

Ref document number: 602008056489

Country of ref document: DE

Free format text: PREVIOUS MAIN CLASS: H04L0029080000

Ipc: H04L0065000000

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: NL

Payment date: 20230215

Year of fee payment: 16

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: FR

Payment date: 20230208

Year of fee payment: 16

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: GB

Payment date: 20230202

Year of fee payment: 16

Ref country code: DE

Payment date: 20230131

Year of fee payment: 16

P01 Opt-out of the competence of the unified patent court (upc) registered

Effective date: 20230517