GB2491872A - Automatic setup of audio/computer teleconference - Google Patents

Abstract

A system for the automatic setup of an audio/computer teleconference comprising node units 105 each having a phone connection, a telephone network connection, at least one computer network connection and a system control server 104. The node unit comprises code to transmit an outgoing audio announce message into a potential conference call via said telephone network audio connection to identify the node unit. The node can further receive via the network audio connection an incoming audio announce message from a conference call to which the node unit is already connected and determine from the incoming message an identifier for a remote node unit connecting to the conference call. It can then transmit to the server, via the computer network, identifiers for the local and remote node units. The server comprises code to receive the node unit identifiers and provide computer equipment connection data to computer equipment at node. The system can be used to add video content to a pre-existing telephone conference call.

Description

Video Conferencing Systems

FIELD OF THE INVENTION

This invention relates to systems and methods and computer program code for automatic connection of computer equipment between participants in a hosted audio teleconference.

BACKGROUND TO THE INVENTION

Use of video conferencing equipment is common place, but improvements in the ease of use of such equipment are still desirable.

Background prior art can be found in US4,560,883, US7,432,951, and in IEEE Paper: "A media synchronization survey: reference model, specification, and case studies", Blakowski and Stienmetz, 1996; Report IS-191 of the Advanced Television Systems Committee; and in "Voice and Video Conferencing Fundamentals," from Cisco Press.

We will describe techniques which improve on the prior art and facilitate automatically establishing a video conference call or similar digital shared connection.

SUMMARY OF THE INVENTION

According to a first aspect of the invention there is therefore provided a system for automatic connection of computer equipment between participants in an audio teleconference, the system comprising: at least two node units, each said node unit having a phone audio connection to provide an audio interface with a phone, a telephone network audio connection to provide an audio interface with a phone network, and at least one computer interface for interfacing with computer equipment and a computer network, the at least one computer interface comprising at least one computer network connection; and a system control server having an interface to said computer network; wherein a said node unit comprises node processor control code to: transmit an outgoing audio announce message into a potential conference call via said telephone network audio connection, wherein said outgoing audio announce message identifies said node unit connecting to said potential conference call; receive via said telephone network audio connection an incoming said audio announce message from a conference call to which the node unit is already connected; determine, from said incoming audio announce message, an identifier identifying a remote node unit connecting to said conference call; and transmit, via said computer network to said system control server, responsive to said received message, both an identifier for said node unit and said identifier for said remote node unit; and wherein said system control server comprises server processor control code to: receive said identifiers for said node unit and for said remote node unit; and provide computer equipment connection data to respective computer equipment for each of said identified node unit and said identified remote node unit, to enable connection between said respective computer equipment of said node units.

Embodiments of the system enable automatic establishment of a video conference call over an IF (internet protocol) computer network, automatically detecting establishment of the audio conference call set up and then setting up the corresponding video call between the computer equipment connected to the respective nodes, where possible.

In this way the audio conference call happens over the (audio) phone network in the usual way and the video call is established over the (separate) IF network, using the audio telephone network to set up the video call'. The set up of the video call happens automatically and transparently to the user, the node units in combination with the system control server providing the audio signalling over the audio telephone network which enables the digital switching' to set up the video call. The skilled person will appreciate that embodiments of these techniques are not dependent on a particular type of phone/phone network and will, for example, work with both a POTS (Plain Old Telephone System) network and a voice over Internet Protocol (VOIP) phone/network.

The teleconference may have just two nodes, that is it may be a point-to-point call, or may have more than two connected nodes.

The skilled person will appreciate that although preferred embodiments of the system implement a video call, the techniques we describe may also be employed with other forms of streamed media shared over a digital network between computer equipment coupled to the node units and, more generally, other forms of digital data sharing or "digital call", for example desktop sharing and the like.

In embodiments a node unit includes code to detect connection of the unit to a potential conference call. The outgoing audio announce message is transmitted following such potential connection detection. More particularly in embodiments the node sends a connection detection message to the server identifying detection of the potential conference call and receives a response to this message from the server for use in transmission of the outgoing audio announce message. This message may comprise a simple confirmation that the call is a potential video call, and/or it may comprise data for the audio announce message.

Thus broadly speaking, in embodiments a node unit announces connection of the phone to which is connected into a potential conference call, by sending an audio message, for example a tone, tone combination or tone sequence. A node unit also detects such audio messages sent from one or more other node units connected to a conference call, the audio messages being sent over the telephone network. However in embodiments, whilst an incoming audio announce message does identify a remote node, it does not identify it to the local node -instead the local node passes it on and the system control server does the identification. In a similar way, in embodiments the transmitted identifier for a node unit identifies the node unit to the server. In embodiments a node receiving an incoming audio announce message decodes the node identification data and sends this on to the server via the computer network, but the local does not (need to) know which remote node the message came from: in embodiments the directory information is held in the server.

When a node unit already on a conference call detects another node unit (phone) joining, it then sends to the system control server information which tells the server that the two identified node units are on a shared audio call. The server then enables set-up of a digital connection between the computer equipment connected to the respective node units, for example by providing connection data to the connected computer equipment, in embodiments via the node units, so that they can set up the digital connection. In embodiments the connection data comprises sharing identifier data for connecting the computer equipment of each respective node unit to a digital media sharing service. For example the sharing identifier data may comprise a Webex TM internet address. The respective node units may receive the same or complimentary sharing identified data to enable set up of the digital connection. In embodiments the computer equipment is connected to the computer (IP) network via the node units and the node units manage the forwarding of the sharing identifier data to the respective computer equipment. However, potentially, the computer equipment associated with each node unit may have a separate connection to the computer network and set up of the digital call may nonetheless be managed by the system control server by passing connect messages over the computer network.

In embodiments, three or more node units may each be connected into a pair of audio and digital conference calls which are separate in the sense that they operate over separate audio (analogue or VOIP) and digital (computer) networks but which are synchronised so as to give the impression to a user of a single combined audio and digital conference call, giving the impression that the call is operating seamlessly over a single network.

In some preferred embodiments, for security, when a node detects connection to a potential conference call it connects to the server and receives a temporary identifier which is used in place of the identifier it uses initially to connect to the server (and, optionally, this initial connection may be secure). In this way each node unit may be allocated a temporary identifier, similar to a one time password, which may be employed to conceal the true identities of the node units connecting to the conference call, to restrict the possibility of an eavesdropping attack on the digital call. Scope for such attack is nonetheless limited because of the separate use of the telephone network for signalling to set up the digital call.

In embodiments, therefore, a node comprises code to request and receive from the server a temporary identifier, for example a random, unique-in-time value, and this in encoded into the outgoing audio announce message. When another node unit receives this message and decodes the code this temporary identifier may be linked with the receiving nodes identifier and the two together passed to the control server, which is then able to identify the true identities of the two nodes since the control server maintains a table or other relationship linking the temporary and permanent or true node identities. In embodiments the messages passed between a node and the server include an IP address of the node and an encrypted identifier of the node.

As an alternative to use of a temporary identifier, a transformation or encryption of the node's identifier may be employed and encoded/decoded into/from the audio messages. In genera) a node unit will store such a unique identifier internally in non-volatile memory. The transformation may be implemented, using, for example, a look up table or a hash function. Where transformation or encryption of a true node identifier is employed, this may be lossless to reduce or avoid address conflicts.

The code in a node may treat any diafled number as a potential conference call but in some preferred embodiments to reduce the server load a node is configured to apply a first pass filter to screen out calls which are determined not to be conference calls.

Such a filter may be based on one or more of whether or not the dialled number is an internal extension. The country code of the dialled number, the area code and so forth.

For example, it is common to dial 9' for an outside line and a node may be programmed to filter out calls not beginning with a specified digit or digits. Similarly in the UK an audio conference bridge number typically begins 0845' and thus a node may be programmed to select as potential conference calls only those having a specified set of digits or one of a plurality of specified sets of digits. Additionally or alternatively the server may perform similar screening. Thus in embodiments the system includes code to monitor the telephone network audio connection to identify a phone number (which may include a PIN) dialled by a phone connected to the node unit. However it will be appreciated that it is not essential to be able to detect whether or not a number is a conference all as (preferably after filtering) all calls may be treated as potential conference calls so that later joining nodes may be connected.

It will appreciated that, in embodiments, an existing caller on an audio conference call will hear the audio announce message of a new caller joining the call, the node of the existing caller also hearing' this to determine that a new caller has joined. In embodiments a node includes code to detect and resolve a conflict between audio announce messages, for example by randomly backing off transmission of an outgoing audio announce message. s cc ( . * .).

30 in embodiments where the digital connection comprises a streamed video connection, preferably the system, more particularly a node, includes code to synchronise the audio and video carried over the two separate networks, for example by controlling delay of the audio data between the phone network and phone.

Related aspects of the invention provide, independently, a node unit and a system control server as described above, and corresponding processor control code.

In preferred embodiments the server is configured to store data linking or mapping an outgoing audio announce message as described above to the node unit transmitting the audio announce message. This may be achieved, for example, by the server sending data for inclusion the audio announce message to the node in the first place, or the node may employ an announce message comprising data that is permanently unique to the node, thus allowing the node transmitting the message to be identified.

In a related aspect the invention provides a method of automatic connection of computer equipment between participants in an audio teleconference in a system comprising: at least two node units, each said node unit having a phone audio connection to provide an audio interface with a phone, a telephone network audio connection to provide an audio interface with a phone network, and at least one computer interface for interfacing with computer equipment and a computer network, the at least one computer interface comprising at least one computer network connection; and a system control server having an interface to said computer network; the method comprising: setting up an audio conference call over said phone network via said node units; intercepting said audio conference call setup in said node units; and setting up a digital streamed media conference call between computer equipment at said respective node units, responsive to said intercepting of said audio conference call setup, wherein said streamed media conference call operates over said computer network; the method further comprising: using audio signalling over said phone network for said setting up of said digital streamed media conference call.

In some preferred embodiments the method further comprises using the audio signalling to announce that a second or subsequent phone has joined the audio conference call, detecting this, via the audio telephone network, at a first phone and then reporting, from a load associated with the first phone, that the first and second phones are on a shared audio conference call so that the system control server can set up a corresponding digital link between the computer equipment associated with the respective nodes.

Thus in embodiments audio signalling over the phone network comprises sending an audio signal for setting up the digital streamed media conference call from a node unit over the phone network, and receiving a response to the audio signal at the node via the computer network. Although the node unit may hear an audio signal from another node unit it does not listen to or act on this signalling, but instead uses the computer network as a return signal path to receive a call setup signal, bearing call setup data, from the server: Whilst a node can receive the incoming announce message, it does not have a concept of who it came from; in preferred embodiments the directory information is held in the server.

The invention further provides processor control code to implement the above-described systems, devices and methods, for example on a general purpose computer system or on a digital signal processor (DSP). The code is provided on a physical carrier such as a disc, CD-or DVD-ROM, programmed memory such as non-volatile memory (e.g. Flash) or read-only memory (Firmware). Code (and/or data) to implement embodiments of the invention may comprise source, object or executable code in a conventional programming language (interpreted or compiled) such as C, or assembly code. As the skilled person will appreciate such code and/or data may be distributed between a plurality of coupled components in communication with one another.

BRIEF DESCRIPTION OF THE DRAWINGS

These and other aspects of the intention will now be further described, by way of example only, with reference to the accompanying figures in which: Figure 1 a and 1 b show respectively, an overview and a detailed view of a video teleconferencing system according to an embodiment of the invention; Figures 2a and 2b show, respectively, first and second examples of message passing during automatic connection of computer equipment between participants in an audio teleconference, according to embodiments of the invention; Figure 3a and 3b show, respectively, a telephone interface and a node unit or endpoint' according to an embodiment of the invention; and Figures 4a and 4b illustrate a procedure which may be employed for a synchronisation process for a node unit or endpoint', according to an embodiment of the invention.

DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS

Broadly speaking we will describe a video conferencing system in which video is added to a pre-existing ordinary telephone call. Thus embodiments of the video conferencing system add video transmitted over an IP network to a telephone call placed using pre-existing telephone equipment and infrastructure. We will describe procedures for automatically establishing such a video conference, and also techniques for use in such a video conference for ensuring that the video and audio are matched in time, so that participant's lips move with the sound of their voices.

The approach conveys a number of benefits: The audio part of the call retains the simplicity, familiarity and reliability of an ordinary phone call. The video is an enhancement, but it does not get in the way of placing or conducting a phone call.

Users do not have to replace existing conference phones or use a different microphone for video conferences versus audio conferences. For conference calls, it becomes very easy to mix some participants with audio-only and others who have audio and video.

Moreover, if the IP network used by the video channel is heavily congested and video quality is affected, communication can continue uninterrupted as audio-only.

In a typical audio teleconference, the parties dial a number followed by a PIN (Personal Identification Number). Normally there is one PIN for the host and a separate PIN for the guests. Teleconference service providers have local phone numbers in many geographies to reduce the cost of participation. Having dialled a normally local number, plus a PIN, the participants are able to talk to each other in a multi-way audio conference.

Often, the audio participants will want to share other electronic information with each other, which could be presentations, white-board sketches or streaming video.

Currently this is either done by email (e.g. for PowerPoint), using a service provider that gives a URL for the participants to enter into their web browser (e.g WebEx), or using a unified communications provider where such services are integrated with the audio (e.g. Skype).

Whilst many people are familiar with audio conferences, accessing associated digital content to accompany the audio is often a time-consuming process which often delays the start of multi-party meetings. This is especially noticeable in video telephony.

Where AV/IT equipment is installed in meeting rooms to facilitate rich communication, associated remote controls, touch panels and the like often serve to be a barrier to effective use of this equipment.

At a high level, therefore, we will describe techniques which enable parties to access a hosted audio conference using their existing telephone equipment and conference provider, and having accessed said audio conference will automatically connect an associated piece of IT equipment with similar pieces of IT equipment in the rooms of the other audio conference participants to facilitate the fast and easy sharing of associated digital media. Broadly the system comprises 2 parts, a phone tap device (node unit or "endpoint"), and a system control server. The phone tap or node unit has two telephone interface; one to the local telephone and one to the exchange (PBX). It also has a network interface in order to communicate with the system control server, and an interface specific to the IT equipment being connected to the hosted audio conference.

When a user wishes to join an audio conference, they take the phone off-hook and dial as they would normally do. On dialling out, the Phone Tap recognizes DTMF tones and optionally sends the dialled number to the control server. The Phone Tap acts as a pass-through, enabling normal 2-way audio communication whilst being capable of recognizing tones played on the audio conference, received from the PBX.

The control sever optionally sends a unique-in-time code back to the Phone Tap, which on receipt the Phone Tap will play as tones out to the PBX. If the Phone Tap hears any tones being played while it is playing its tones, it will back-off and retry the code until it is certain that any other audio participants can have unambiguously received the code.

On detecting a series of tones coming from the PBX, the Phone Tap will encode these and report the code back to the control or rendezvous server. Using this information, the Rendezvous Server is able to determine which Phone Taps are connected to the same audio conference without Phone Taps having to reply to each others codes over the audio channel.

Having made this determination, the rendezvous server can send a unique-in-time identifier to each Phone Tap, which the Phone Tap passes back to the associated IT equipment. The IT equipment uses this unique identifier to automatically share digital media. Examples of this identifier are a URL for joining a shared-desktop, or an IP address for transmitting or receiving streaming data.

Referring now to Figure 1 a, this shows an overview of an embodiment of a video conferencing system 100 according to the invention. The system comprises IT (Information Technology) equipment which is automatically connected when connecting to an audio teleconference in a preferred embodiment comprising a pair of video capture/display units 107 each comprising a video camera 107a and a display 107b. The video units 107 are each connected to an lP (Internet Protocol) network 103, in particular a public IP network such as the Internet. The system 100 also includes a pair of standard audio telephones 106 each connected to a public POTS telephone network 110. The system also includes a pair of node units 105, also termed herein endpoints/phone taps. Each of units 105 is coupled between the phone 106 and the telephone network 110 and has a further network connection to a respective video unit 107. The skilled person will appreciate that in the arrangement of figure la a video unit 107 may have only a single network connection rather than the pair of network connections illustrated. Broadly speaking in operation the node units 105 are able to intercept and modify, in particular delay, sound on the telephone network, and displayed video.

Referring next to Figure lb this shows further details of the system of Figure la, illustrating a third party audio conference bridge 102 connected to the telephone network 110. Typically such an audio bridge enables multiple audio conferencing parties to dial the same number with a different pin to connect to a shared audio teleconference managed by audio bridge 102. The system 100 also comprises a system control or rendezvous' server 104, in embodiments accessible via the internet, and running processor control code (software) to implement a switching' procedure as described later. The server 104 operates in conjunction with the endpoint or node units 105, which also run software as described below, for automatically establishing IT connectivity between IT equipment units 107. Typically the phone 106 and IT equipment 107 of the nodes are located in different office locations. For simplicity Figure 1 shows just two nodes but the skilled person will appreciate that there may be many such nodes in a shared teleconference.

Figures 2a and 2b, these show messages passing between the various elements of system 100 according to a method for automatic connection of the IT equipment between participants in an audio teleconference according to an embodiment of the invention. In Figures 2a and 2b the two nodes are designated A' and B'. The message passing illustrated in Figures 2a and 2b is implemented by corresponding software running on the node units 105 and server 104. Figure 2b illustrates a variant of the protocol of Figure 2a in which some messages (messages 3 and 4 and 8 and 9) are omitted.

Thus, referring to the numbered messages in Figures 2a and 2b, the protocol of Figure 2a is as follows: 1) Telephone A dials into an audio bridge 2) Endpoint A detects the phone number that is dialled, but passes the audio through to the audio bridge 3) If Endpoint A decides that the phone number dialled may be able to share digital media, it sends a message to the Rendezvous (RV) server which includes the phone number and its own unique endpoint identifier 4) If the RV server further determines that it is likely that sharing of digital media may take place, the RV server sends a temporary unique code back to endpoint

A

5) Endpoint A announces itself in the audio bridge by playing a sequence of tones derived from the unique code received in (4). Endpoint A listens for tones from the audio bridge; if tones are heard whilst Endpoint A is announcing itself, it will back off and retry some random time later. Once tones have been successfully announced, Endpoint A enters listening' mode.

6) At some point later, Telephone B dials into the audio bridge 7) Endpoint B detects the phone number that is dialled, but passes the audio through to the audio bridge 8) If Endpoint B decides that the phone number dialled may be able to share digital media, it sends a message to the Rendezvous (RV) server which includes the phone number and its own unique endpoint identifier 9) If the RV server further determines that it is likely that sharing of digital media may take place, the RV server sends a temporary unique code back to endpoint

B

10) Endpoint B announces itself in the audio bridge by playing a sequence of tones derived from the unique code received in (9). Endpoint B listens for tones from the audio bridge; if tones are heard whilst Endpoint B is announcing itself, it will back off and retry some random time later. Once tones have been successfully announced, Endpoint B enters listening' mode.

11) The audio bridge transmits the tones played by Endpoint B back to Endpoint A over the POTS network 12) Endpoint A detects the tones, transforms them into a code and transmits that code to the RV server along with its own unique endpoint identifier 13) The RV server has been able to determine that Endpoints A and B are in the same audio conference by matching the code sent to Endpoint B with the code received from Endpoint A. The RV server generates a unique-in-time sharing-identifier which will allow IT equipment A and IT equipment B to share digital media through a 3ft1 party sharing service. The RV server sends this sharing-identifier to Endpoints A & B 14) Endpoints A and B each pass this sharing-identifier to their attached IT equipment 15) IT Equipment A and B use their sharing identifiers to connect using an IT sharing service In the variant of Figure 2b, messages 3, 4, 8 and 9 are omitted. Then the tones that the Endpoints use to announce themselves may be some transformation of the endpoint's own unique identifier. The transformation is chosen to allow Network Address Translation (NAT) at stage (13) and may, for example, be lossless.

Figure 3a shows a telephone interface portion of a node unit 105, illustrating a connection 300 to the telephone network 110, in this example the connection comprising a two wire interface. A connection 302 to analogue phone 106 likewise comprises a two wire interface. A subscriber line interface circuit 306 converts the differential signal into an audio transmit (from the phone) line 308 and an audio receive (to the phone) line 310,. a corresponding PSTN (Public Switched Telephone Network) interface 304 re-converts the signals back to the two-wire interface 300. A switch 314 selectively connects the audio receive line 310 to either the phone interface 304 or to a suitable termination to mute the phone 106. The audio receive signal 310b is also connected to a variable delay circuit 316 which in turn is connected to a speaker 107c.

With the switch in the position shown, sounds (audio) received from a remote telephone are not relayed to the local handset but instead pass through the variable delay circuit 316 before being played on a speaker 107c, which may be located next to the video display 1 07b so that the sound comes from the same direction as the image of the person speaking. (Alternatively the audio output of delay circuit 316 may be provided back to phone 106). With the switch in the alternative position a normal telephone call can take place where audio received from a remote telephone is relayed directly to the local handset. In this mode the output of delay circuit 316 is muted to prevent sound from being heard from the speaker 1 07c as well as the local telephone.

A second switch 319 selectively connects the audio transmit line 308b to either the phone interface 306 or to the output of a variable delay circuit 322. With this switch in the position shown audio transmitted by the local handset passes through the variable delay circuit 322 before being sent to the remote telephone. Via appropriate settings of the two switches, the interface 105 is able to selectively delay both sounds picked up by the local telephone's microphone and also sounds received from a remote telephone over the POTS network. In addition it is able to divert sounds received from the remote telephone for playback over a speaker instead of the telephone. Details of how the delays to be applied by circuits 316 and 322 are determined from the audio characteristics generated by circuits 334 and 324 are described in figure 3b.

Figure 3b shows details of an embodiment of a node unit or endpoint 105 in which like elements to those previously described are indicated by like reference numerals.

Thus referring to figure 3b an instruction to adjust the audio transmission delay is received via a logical connection 320 to lP network 103 and used to control a variable audio transmission delay module 322 coupled between the audio output of phone 106 and the POTS 110. In this way audio between 3 or more nodes in a conference call may be synchronised.

Audio received from POTS 110 and provided on line 310 is provided to an audio characterising module 324 which determines characteristic data for the audio signal from the telephone network and provides a characterising data output 326. An unload module 328 logically coupled to IF network 103, receives the video signal for the node from the network and extracts the audio characterising data from this video signal providing this data on line 330. Skew control circuit compares the data on lines 326 and 330, for example by performing a cross-correlation, to determine a time offset between the audio timing encoded on the video signal and the received audio timing, and this time offset data is provided on a set of outputs 332. One of outputs 332 controls a variable delay applied by module 316 to the received audio intended for output by phone 106. another of outputs 332 provides a delay control input to a received video delay module 334, which receives a video stream from video decoder 336 coupled to unload module 328, and provides a (delayed) video output to display 107b. A third output 332 is logically coupled to IF network 103 to provide data for adjusting an audio transmission delay to one or more other nodes or endpoints 105 in the video conference.

The local audio from phone 106 for transmission over the POTS 110 is similarly characterised by module 334, and this audio characterising data is buffered by module 336 and combined with video data in network payload data construction module 338.

This module also receives video from camera 107a, captured by module 340 and encoded by module 342, the digitally encoded video data and characteristic audio data being combined and provided to IF network 103.

The skilled person will appreciate that the above described modules may be implemented in hardware, software, or a combination of the two.

In operation the near end video conference unit samples sound recorded by the near end telephone and digitizes it. The digital samples are then processed to produce a simplified characteristic of the original audio signal. In its simplest form this characteristic divides the audio signal into equal length periods, and for each period record whether the average sound amplitude was greater than a threshold level of background noise. This characteristic gives a pattern of sound/no-sound values representing the transmitted audio signal. We will use this simple characteristic in the examples which follow because it makes the process of correlating the characteristics easy to visualise. However variations in gain across the telephone network, signal processing, distortion and noise contribute to reducing the reliability of using a threshold to characterise the audio signal. Preferably, rather than quantising the sound (for example, average sound amplitude) in a given period to a single bit using a threshold, it is quantised using multiple bits, for example 15 bits giving a range from 0- 32767. Greater reliability may be obtained by using more complex characteristics of the sound (for example taking a Fourier transform of the audio signal to determine the amount of energy in different frequency bands), at the cost of employing more compute power to calculate the characteristics, and at the cost of using more bandwidth in the computer network to transmit these. In theory, the entire digitised audio stream could be transmitted without modification or with a lossless compression technique.

Whichever characteristic is used, the near end video conference unit adds the characteristic to the video frame data it captured at the same time as the audio was recorded. It then transmits this characteristic with the video frame over the video transmission channel. This may be an lP network, but other transmission channels are possible. At the far end, the far end video conference unit receives the video frame with associated audio characteristic. It independently samples sound received by the far end telephone from the telephone network and then performs a substantially identical characterisation of the received sound, for comparison with the characteristic embedded in the video to determine a time offset between the two. The two characteristics may be correlated to determine the time (phase) offset.

Example 1:

Video frame rate is 30 frames per second, and each frame is embedded with an audio characteristic representing speech or no speech during the 33ms of time the video frame represents.

Over the first second of transmission, the near end produces the following characteristic from locally recorded audio: oo1o111o1001o11o111o1o10001o11 Over the same second, the far end identifies the following characteristic in the received audio signal: 1o11o10001o111o1001o11o111o1o1 On comparing the two characteristics, the receiver is able to determine a unique solution for the offset between them: oo1o111o1001o11o111o1o10001o11 1o11o10001o111o1001o11o111o1o1 In this example audio lags video by 7 frames which is 233ms. A 233ms delay can now be added to the video pipeline to bring the two into synchronisation.

There are two complications this approach has to handle which are introduced through the use of audio conferencing numbers.

Non-video participants If an audio conference between two video endpoints also includes a non-video participant then the characteristic derived from audio heard at the receiving endpoints will not match the characteristic transmitted with the video. The effect of audio mixing in the additional caller will be to average the audio signals for which we have characteristics with one which we do not have a characteristic for. For a simple case using a threshold characteristic, some periods of not-sound will be converted to periods of sound. The technique can still cope with this by looking for matching periods of sound, and ignoring periods of not-sound. The sound pattern encoded with the video becomes for example lxlllxlxlxlllxlxllx where x indicates "don't care". This increases the chances of a false match but this can be compensated for by using longer sequences of patterns for comparison.

Multi-way calling If there are more than two video endpoints in a conference then lip sync becomes more complex, because although each endpoint receives the video streams separately, it only gets a combined audio feed from the telephone network. Unless the audio delays from all the units to the conference bridge in the telephone network are equal it is difficult to for a receiving endpoint can correct the different audio delays mixed up in the combined audio feed. To address this endpoints apply a delay to the audio they transmit as well as the audio they receive. Each endpoint learns from the other endpoints how much delay it should apply to its outgoing audio. Using the sound characteristic technique an endpoint in a three way call will receive a sound characteristic from each of the other two endpoints in their video streams, as well as calculating a sound pattern from the audio it receives from the conference bridge.

Example 2:

Conference bridge sound characteristic lioioiililoiioloiloiol000loioo From A's video stream lxi lxxxi ixlxixx From B's video stream xlxi 1 lxlxixxlxi Here are the matches iloiolltiiolloiolioloi000ioloo lxi lxxxi lxixlxx xixiiixlxixxlxi In this example we have successfully identified a unique solution for both delays using just half a frame of sound characteristics.

Referring now to Figure 4a, this illustrates synchronization in a video conference with three endpoints A, B and C 105a,b,c sharing an audio conference bridge X 102, such as "MeetingZone". In Figure 4a: Vab is the end-to-end delay transmitting video from A to B. Aax is the end-to-end delay transmitting audio from A to the conference bridge X. Axa is the end-to-end delay transmitting audio from A to the conference bridge X. Each endpoint is capable of measuring the difference (skew) between an incoming audio stream and its corresponding video stream using the technique described above.

For endpoint A: Skewb = AbX+ Axa -Vab Skew0 = A0+ Axa -Vac The skew will be negative if the video arrives after the audio and positive if the video arrives first.

Each endpoint is capable of inserting additional delay into its incoming audio feed, outgoing audio feed and incoming video feed(s). So endpoint A is capable of increasing: Axa Aax Vba Vca In one approach each endpoint increases the incoming video delays until they are equally skewed relative to their respective audio streams ie. For a given endpoint, the algorithm is: increase the incoming V?? delays until all the Skew? values are equal.

Then if the Skew? values are greater than zero, increase all the V?? delays until Skew? is zero. If the Skew? values are less than zero, increase the incoming audio delay AX2 until they become zero. This approach does not require the endpoints to communicate with each other for synchronization as each endpoint can calculate and apply all necessary delays without referring to any other endpoint. However this simple method can increase the worst-case delay between callers. This is undesirable because as delays become longer it becomes harder for callers to maintain a natural interactive conversation. Furthermore this approach will not synchronise any video/audio pair to any other video/audio pair in the conference. It only ensures that considered individually, each stream has its audio in sync with its video.

A better solution uses the procedure of Figure 4b. In order to use this approach the endpoints need to determine the skew between the video streams. This can be achieved using a prior art technique described in Voice and Video conferencing Fundamentals'. In summary this is done by sharing a master reference clock across all the endpoints and encoding timestamps in the video streams which are relative to the master clock. With this additional information each endpoint can now calculate the skew between audio streams as follows: Skewb -Skew0= AbX+ Axa -Vab -(A+ Axa -Vac) =AbxAcx+VacVab Acx -AbX = Vac -Vab + Skewc -Skewb Now referring to the steps in Figure 4b: 1) Each endpoint measures the skew between the incoming audio streams contained within the mixed audio feed it receives from the conference bridge (Step 400). For example in the three way call shown in Figure 4a, endpoint A measures: SkewbC = AbX -Acx 2) Each endpoint then determines how much the other endpoints would have to increase their outgoing audio delays in order to make this skew value zero and transmits appropriate instructions to the other endpoints over the IF network (Step 402).

3) Each endpoint will receive instructions from every other endpoint (Step 404).

Continuing this three-way example, endpoint B will receive instructions to increase its outgoing audio delay AbX from endpoints A and C. Endpoint B applies the largest delay requested by any other endpoint.

4) Audio to audio skews measured in (1) will now be zero. Each endpoint now measures the skew between each audio stream and its corresponding video stream (Step 406).

5) If the audio is ahead of video, the endpoint increases its incoming audio delay to compensate; if video is ahead of audio, the endpoint increases its incoming video delay to compensate (Step 406).

Preferred embodiments of the above described techniques are used to synchronise audio with digital data streams carrying a corresponding video. However the skilled person will appreciate that the techniques may also be employed to synchronise digital data streams carrying other forms of digital data.

No doubt many other effective alternatives will occur to the skilled person. It will be understood that the invention is not limited to the described embodiments and encompasses modifications apparent to those skilled in the art lying within the spirit and scope of the claims appended hereto.

Claims (24)

1. CLAIMS: 1. A system for automatic connection of computer equipment between participants in an audio teleconference, the system comprising: at least two node units, each said node unit having a phone audio connection to provide an audio interface with a phone, a telephone network audio connection to provide an audio interface with a phone network, and at least one computer interface for interfacing with computer equipment and a computer network, the at least one computer interface comprising at least one computer network connection; and a system control server having an interface to said computer network; wherein a said node unit comprises node processor control code to: transmit an outgoing audio announce message into a potential conference call via said telephone network audio connection, wherein said outgoing audio announce message identifies said node unit connecting to said potential conference call; receive via said telephone network audio connection an incoming said audio announce message from a conference call to which the node unit is already connected; determine, from said incoming audio announce message, an identifier identifying a remote node unit connecting to said conference call; and transmit, via said computer network to said system control server, responsive to said received message, both an identifier for said node unit and said identifier for said remote node unit; and wherein said system control server comprises server processor control code to: receive said identifiers for said node unit and for said remote node unit; and provide computer equipment connection data to respective computer equipment for each of said identified node unit and said identified remote node unit, to enable connection between said respective computer equipment of said node units.
2. 2. A system as claimed in claim 1 wherein each said node unit comprises a pair of said network connections, one for interfacing with computer network and one for interfacing with said computer equipment for the node unit; and wherein said connection data comprises sharing identifier data for connecting the computer equipment of said respective said node unit to a digital media sharing service; and wherein said server processor control code is configured to send said sharing identifier data to each said node unit, for a said node unit to provide to said respective computer equipment for the node unit.
3. 3. A system as claimed in claim 2 comprising at least three said node units, wherein phones of said at least three node units connect with a shared audio teleconference bridge via respective telephone network audio connections, and wherein operation of said node and server processor control code connects said respective computer equipment of said at least three node units to said digital media sharing service such that said digital media sharing service operates over said computer network separately from audio of said node units shared by said audio teleconference bridge.
4. 4. A system as claimed in claim 1, 2 or 3 wherein said node processor control code further comprises code to request and receive a temporary identifier for the requesting node from said server, wherein said request includes one or both of an identifier of said requesting node and a computer network address of said requesting node; wherein said node processor control code further comprises code to encode into said outgoing audio announce message said temporary identifier, wherein said identifiers for said node unit and said remote node unit comprise, respectively, a said temporary identifier received from said server and a said temporary identifier from said remote node unit decoded from said incoming audio announce message; and wherein said server processor control code further comprises code to receive said request for a temporary identifier from said requesting node, generate a said temporary identifier, and return said generated temporary identifier to said requesting node using one or both of said identifier of said requesting node and said computer network address of said requesting node.
5. 5. A system as claimed in claim 1, 2 or 3 wherein said node processor control code further comprises code to generate an identifier for the node from an internal stored unique identifier fro the node, and to encode said generated identifier into said outgoing audio announce message.
6. 6. A system as claimed in any preceding claim wherein said node processor control code further comprises code to detect connection of a said node unit to a potential conference call; and to transmit said outgoing audio announce message following said potential connection detection.
7. 7. A system as claimed in claim 6 wherein said node processor control code further comprises code to send a connection detection message to said server identifying detection of said potential conference call; and to receive a response to said connection detection message from said server for use in said transmission of said outgoing audio announce message.
8. 8. A system as recited in any preceding claim wherein said node processor control code further comprises code to monitor said telephone network audio connection to identify a phone number dialled by a phone connected to the node unit, and wherein said server processor control code further comprises code to process said phone number to identify a said conference call and to transmit data to said node unit identifying said dialled number to confirm said potential conference call is a conference call.
9. 9. A system as recited in any preceding claim wherein said node processor control code further comprises code to monitor said telephone network audio connection to identify a phone number dialled by a phone connected to the node unit, and to process said phone number to detect said potential conference call.
10. 10. A system as claimed in any preceding claim wherein said node processor control code further comprises code to monitor said telephone network audio connection to detect and resolve a conflict between audio announce messages when transmitting said outgoing audio announce message.
11. 11. A system as claimed in any preceding claim wherein said computer equipment comprises video display computer equipment, wherein said connection between said respective computer equipment of said node units comprises a streamed video connection, and wherein a said node unit further comprises a system to sense and control delay of audio data between said phone audio connection and said audio interface with said phone network to synchronise said streamed video over said computer network with said audio over said telephone network audio connection.
12. 12. A system as recited in any preceding claim wherein said computer network is an IF network and said connection data comprises an IF address of a shared media server.
13. 13. A node unit as recited in any one of claims 1 to 12 comprising a phone audio connection to provide an audio interface with a phone, a telephone network audio connection to provide an audio interface with a phone network, and at least one computer interface for interfacing with computer equipment and a computer network, the at least one computer interface comprising at least one computer network connection; the node unit further comprising a processor, working memory and program memory storing node processor control code to: transmit an outgoing audio announce message into a potential conference call via said telephone network audio connection, wherein said outgoing audio announce message identifies said node unit connecting to said potential conference call; receive via said telephone network audio connection an incoming said audio announce message from a conference call to which the node unit is already connected; determine, from said incoming audio announce message, an identifier identifying a remote node unit connecting to said conference call; and transmit, responsive to said received message, an identifier for said node unit and said identifier for said remote node unit, via said computer network to said system control server.
14. 14. A node unit as claimed in claim 13 wherein said processor control code is further configured to detect connection of a said node unit to a potential conference call; and to transmit said outgoing audio announce message following said potential connection detection.
15. 15. A node unit as claimed in claim 14 wherein said processor control code is further configured to send a connection detection message to said server identifying detection of said potential conference call; and to receive a response to said connection detection message from said server for use in said transmission of said outgoing audio announce message.
16. 16. A physical data carrier carrying processor control code for a node unit as recited in any one of claims 1 to 12, said carrier carrying node processor control code to: transmit an outgoing audio announce message into a potential conference call via said telephone network audio connection, wherein said outgoing audio announce message identifies said node unit connecting to said potential conference call; receive via said telephone network audio connection, and decode, an incoming said audio announce message from a conference call to which the node unit is already connected; determine, from said incoming audio announce message, an identifier identifying a remote node unit connecting to said conference call; and transmit, responsive to said received message, an identifier for said node unit and said identifier for said remote node unit, via said computer network to said system control server.
17. 17. A physical data carrier as claimed in claim 16 wherein said node processor control code further comprises code to detect connection of a said node unit to a potential conference call; and to transmit said outgoing audio announce message following said potential connection detection.
18. 18. A physical data carrier as claimed in claim 17 wherein said node processor control code further comprises code to send a connection detection message to said server identifying detection of said potential conference call; and to receive a response to said connection detection message from said server for use in said transmission of said outgoing audio announce message.
19. 19. A system control server, in particular as recited in any one of claims 1 to 12, the server comprising an interlace to said computer network, a processor, working memory, and program memory storing server processor control code to receive said identifiers for said node unit and for said remote node unit, and to provide computer equipment connection data to respective computer equipment for each of said identified node unit and said identified remote node unit to enable connection between said respective computer equipment of said node units.
20. 20. A system control server as claimed in claim 19 wherein said server is configured to store data linking said outgoing audio announce message to the node unit transmitting said audio announce message.
21. 21. A physical data carrier carrying processor control code for a system control server as recited in any one of claims 1 to 12, said processor control code being configured to: receive said identifiers for said node unit and for said remote node unit, and to provide computer equipment connection data to respective computer equipment for each of said identified node unit and said identified remote node unit to enable connection between said respective computer equipment of said node units.
22. 22. A method of automatic connection of computer equipment between participants in an audio teleconference in a system comprising: at least two node units, each said node unit having a phone audio connection to provide an audio interface with a phone, a telephone network audio connection to provide an audio interface with a phone network, and at least one computer interface for interfacing with computer equipment and a computer network, the at least one computer interface comprising at least one computer network connection; and a system control server having an interface to said computer network; the method comprising: setting up an audio conference call over said phone network via said node units; intercepting said audio conference call setup in said node units; and setting up a digital streamed media conference call between computer equipment at said respective node units, responsive to said intercepting of said audio conference call setup, wherein said streamed media conference call operates over said computer network; the method further comprising: using audio signalling over said phone network for said setting up of said digital streamed media conference call.
23. 23. A method as claimed in claim 22 further comprising: announcing, using said audio signalling, that a subsequent phone has joined said conference call; detecting said announcing over said phone network at a first phone; reporting to said system control server, responsive to said detecting, that said first and subsequent phones share an audio conference call; and sending messages, from said system control server to said node units of said first and subsequent phones, to enable setting up, via said node units, of said digital streamed media conference call between computer equipment connected to said node units.
24. 24. A method as claimed in claim 22 or 23 wherein said using of said audio signalling over said phone network comprises sending an audio signal for setting up said digital streamed media conference call from said node unit over said phone network, and receiving a response to said audio signal at said node via said computer network.