GB2283152A - Audio transmission over a computer network - Google Patents

Abstract

A computer workstation includes an audio adapter card for generating a sequence of digital audio data samples, aid accumulating them into audio data blocks. These are then transferred one at a time into a queue in the main memory of the computer workstation. A first program loop on the workstation receives an interrupt from the audio adapter card indicating the transfer of another audio data block, and maintains a record of the head of the queue. Another program loop requests access to the computer network, transmits messages from the workstation, and maintains a record of the tail of the queue. Each audio packet transmitted from the workstation incorporates essentially all the audio data currently enqueued. <IMAGE>

Description

AUDIO TRANSMISSION OVER A COMPUTER NETWORK The present invention relates to the transmission of audio signals over a network between computer workstations, and in particular to systems in which the audio signals are encoded into digital packets for transmission.

Conventionally voice signals have been transmitted over standard telephone lines. However, with the increase in locations provided with local area networks (LANs) and the growing importance of multimedia communications, there has been considerable interest in the use of LANs to carry voice signals. This work is described for example in "Using Local Area Networks for Carrying Online Voice" by D Cohen, pages 13-21 and "Voice Transmission over an Ethernet Backbone" by P Ravasio, R Marcogliese, and R Novarese, pages 39-65, both in "Local Computer Networks" (edited by P Ravasio, G Hopkins, and N Naffah; North Holland, 1982). The basic principles of such a scheme are that a first terminal or workstation digitally samples a voice input signal at a regular rate (eg 8 kHz).A number of samples are then assembled into a data packet for transmission over the network to a second terminal, which then feeds the samples to a loudspeaker or equivalent device for playout, again at a constant 8 kHz rate.

As discussed in the above-mentioned articles, there are several problems with the use of a LAN for voice transmission. Firstly, there is a transmission delay over a LAN including an initial delay in gaining access to the LAN; eg if the LAN is based on a Token Ring structure, a node must wait to receive the token before it can transmit (see for example "Local Area Networks: Architectures and Implementations" by J Martin, Prentice Hall, 1989 for a general introduction to LANs). This transmission delay is variable, depending particularly on the utilisation of the LAN by other nodes at any given time. Thus the arrival of packets at a destination node is both delayed and irregular.

If the packets were played out in irregular fashion, this would have an extremely adverse effect on intelligibility of the voice signal.

Therefore, voice over LAN schemes utilise some degree of buffering at the reception end, to absorb such irregularities. Such buffering however introduces yet further delay between the original voice signal, and the audio output at the destination end. This delay may cause problems with echos, and more importantly, can render natural interactive two-way conversation difficult (in the same way that an excessive delay on a transatlantic conventional phone call can be highly intrusive).

The above-mentioned article by Ravasio et al describes the use of a buffer of up to three packets at both the transmitting and receiving ends. If the buffers are about to overflow, because there is a delay in sending or several packets arrive close together, older packets are discarded. This leads to some gaps in the playout signal, but these can be filled by interpolation or silence (the human ear is relatively tolerant of such interference). A slightly more sophisticated approach at the receiving station is described in "Adaptive Audio Playout Algorithm for Shared Packet Networks", by B Aldred, R Bowater, and S Woodman, IBM Technical Disclosure Bulletin, p 255-257, Vol 36 No 4, April 1993. Again, packets that arrive with more than a maximum allowed delay are discarded.The amount of buffering however is adaptively controlled depending on the number of discarded packets (any other appropriate measure of lateness could be used). If the number of discarded packets is high, the degree of buffering is increased, whilst if the number of disarded packets is low, the degree of buffering is decreased. The size of the buffer is altered by temporarily changing the play-out rate (this affects the pitch; a less noticeable technique would be to detect periods of silence and artificially increase or decrease them as appropriate).

An adaptive buffering system is also disclosed in US 5127001 for the reception of multiple channels. In this system the buffer queue length is monitored, and the playout frequency varied in accordance with buffer occupancy. It is suggested in US 5127001 that the transmission queue can be linked to the same frequency, which helps to provide synchronisation over the network. The use of elastic buffers at the transmitting and receiving stations is also described in US 4866704, in this case with particular reference to compensating for slight differences in the clock rates at different nodes.

An additional source of delay in voice communications over a LAN necessarily results from the accumulation of voice samples into data packets. Thus, if the data packet represents say 32 ms of voice samples, the earliest sample is nearly 32 ms old by the time that the packet is ready for transmission. Clearly this problem can be reduced by minimising the size of the data packets, but this runs into an alternative difficulty. Each data packet must include a fixed amount of header information, such as the address of the destination machine etc.

As the number of voice samples per packet is decreased, the ratio of useful data to fixed overheads in each packet becomes less and less.

Thus the overall efficiency of transmission decreases with smaller packet size, or put another way, the bandwidth required to transmit the same voice signal increases. Moreover, the strategies to avoid contention on the LAN may also prefer larger packet sizes. Therefore, the choice of packet size is necessarily a compromise between optimising LAN transmission efficiency, and minimising overall packet delay.

A slightly more complicated system is described in the abovementioned article by Ravasio et al, in which multiple telephones are connected to a Telephone Network Interface Module (T-NIM), which in turn is connected to an Ethernet. The T-NIM card effectively multiplexes the signals from each active telephone together, thereby increasing the packet size. This turns out to allow a more efficient utilisation of network resources, without increasing the delay experienced by any one telephone user.

In summary therefore, the prior art illustrates voice communications over a LAN utilising elastic buffering at both the receiving and transmitting ends, with late packets being discarded.

Packet size is an important parameter in view of the need to minimise overall delay, whilst avoiding inefficient use of bandwidth. The prior art systems often work well when the traffic on the LAN is low, but struggle to maintain acceptable performance at times of high utilisation of the LAN. This indicates a continuing need to improve systems for voice communications over a LAN, bearing in mind the considerations discussed above.

Accordingly, the invention provides a method of processing audio signals at a first workstation for transmission over a network to a second workstation, comprising the steps of: generating a sequence of digital audio samples at the first workstation; storing digital audio samples in a queue at the first workstation; obtaining access to the network at the first workstation; and transmitting a packet containing essentially all the digital audio samples that are currently in the queue each time access is obtained to the network.

Such an approach recognises that packet size, which has been been the focus of much attention in the prior art, is not necessarily the most important parameter in audio transmission over the network. In some circumstances, particularly at periods of high network utilisation, communications can be disrupted by long delays in obtaining access to the network. Such delays are extremely damaging to voice signals, which require low latency in order to maintain acceptable quality. Therefore, whenever network access is achieved, all the available data is transmitted, rather than just a single fixed packet, as in the prior art. Furthermore, the ability to transmit essentially all the enqueued data in a single packet obviates the need for deliberate buffering at the transmitting terminal, reducing the overall transmission delay.

Preferably the method further comprises the step of maintaining information indicating the number of digital audio samples in the queue, and updating said information each time a new digital audio sample is added to the queue or digital audio samples are transmitted from the queue. This allows the appropriate packet size for the next transmission to be readily calculated.

Typically the digital audio samples are generated at a constant frequency, and a predetermined number of digital audio samples are accumulated into blocks of audio data, and each transmitted packet comprises an integral number of blocks of audio data. The digital audio samples are stored in blocks of audio data, one block typically representing 4, 8 or 16 milliseconds of data. The aggregation into such blocks of data provides a larger and more efficient unit for processing in the workstation. The size of block to use can be selected in accordance with conventional considerations (ie a trade-off between granularity and efficiency). Unlike the prior art however, a transmitted message will not contain a constant number of blocks.Typically each packet transmitted will include all the available audio data blocks, although in some cases it may be desirable, for example for reasons of queue integrity, to always maintain one block of data in the queue.

Usually the audio communications will be two-way; in other words the second workstation will also be transmitting audio data over the network to the first workstation. The method therefore preferably further comprises the steps of storing a queue of digital audio samples to be played out; receiving a data packet from the network containing a variable number of digital audio samples; and adding the received digital audio samples to the queue of digital audio samples to be played out. The digital audio samples in the queue can be played out in the conventional manner (perhaps using some adaptive control of queue length). This sort of approach can also be readily extended to full multi-way audio conferencing.

The invention also provides a computer workstation comprising means for generating a sequence of digital audio samples for transmission over a computer network to a remote workstation; means for storing generated digital audio samples in a queue; means for obtaining access to the network; and means, responsive to access being obtained to the network, for transmitting a packet over the network containing essentially all the digital audio samples currently in the queue.

In a preferred embodiment, the means for generating a sequence of digital audio samples comprises an audio adapter card. It is further preferred that the audio adapter card accumulates the digital audio samples into audio data blocks, each containing a predetermined number of digital audio samples, and transfers one audio data block at a time into main memory of the workstation. Typically the audio adapter card raises an interrupt in the computer workstation each time an audio data block is transferred into main memory.

Preferably the computer workstation further comprises means for maintaining information indicating the number of audio data blocks in the queue, and means for updating said information each time a new audio data block is added to the queue or one or more audio data blocks are transmitted from the queue.

The computer network over which the audio data is transmitted will normally be a local area network, such as a Token Ring or an Ethernet, in which case the computer workstation will be further equipped with an appropriate adapter card and software to allow communication over the network.

An embodiment of the invention will now be described by way of example with reference to the following drawings: Figure 1 is a simplified schematic diagram of a computer system; Figure 2 is a simplified diagram showing the major components of the adapter card of Figure 1; Figures 3a and 3b schematically illustrate the queues of audio data at the transmitting and receiving stations respectively; Figure 4 is a simplified flow chart illustrating the processing performed by the digital signal processor at the transmitting station; Figure 5 is a simplified flow chart illustrating the processing performed by the system unit at the transmitting station; and Figure 6 is a simplified flow chart illustrating the processing performed at the receiving station.

Figure 1 is a simplified schematic diagram of a computer system which may be used for audio transmission. The computer has a system unit 10 including microprocessor 22, semi-conductor memory (ROM/RAM) 24, and a bus over which data is transferred 26. Other conventional components of the computer (eg display, keyboard, mouse, etc) will also normally be present, but since they are not relevant to an understanding of the present invention they have been omitted from the drawings. The computer of Figure 1 may be any conventional workstation, such as an IBM PS/2 computer.

The computer of Figure 1 is equipped with two adapter cards. The first of these is a Token Ring adapter card 30. This card, together with accompanying software, allows messages to be transmitted onto and received from a Token Ring. The operation of the token ring card is well-known, and so again will not be described in detail. The second card is an audio card 28 which is connected to a microphone and a loudspeaker (not shown) for audio input and output respectiveley. The system of Figure 1 is typically used for two-way voice communications over a LAN, but may also be used in other multimedia applications, where one node in the network generates a sound signal (eg from an optical disk), which is transmitted over the networ-k to be played out to a user at another node.

The audio card is shown in more detail in Figure 2. The card illustrated and used in this particular embodiment is an M-Wave card available from IBM, although other cards are available that perform an analogous function. The card contains an A/D converter 42 to digitise incoming audio signals from an attached microphone 40. The A/D converter is attached to a codec 44, which samples the incoming audio signal at a rate of 44.1 kHz into 16 bit samples (corresponding to the standard sampling rate/size for compact disks). Digitised samples are then passed to a digital signal processor (DSP) 46 on the card via a double buffer 48 (ie the codec loads a sample into one half of the double buffer while the codec reads the previous sample from the other half. The DSP is controlled by one or more programs stored in semiconductor memory 52 on the card.Data can be transferred by the DSP to and from the main PC bus.

Audio signals to be played out are received from by the DSP 46 from the PC bus 26, and processed in a converse fashion to incoming audio. That is, the output audio signals are passed through the DSP 46 and a double buffer 50 to the codec 44, from there to a D/A converter 54, and finally to a loudspeaker 56 or other appropriate output device.

In the particular embodiment shown, the DSP is programmed to transform samples from the codec from 16 bits at 44.1 kHz into a new digital signal having an 8 kflz sampling rate, with 8-bit samples on a p- law scale (essentially logarithmic), ie corresponding to CCITT standard G.711, using standard re-sampling techniques. The total bandwidth of the signal is therefore 64 kHz, The DSP also performs the opposite conversion on an incoming signal received from the PC, ie it converts the signal from 8-bit 8kHz to 16-bit, 44.1 kHz, again using known resampling techniques.

Note that this conversion between the two sampling formats is not an essential part of the invention. Some other audio cards may include native support for the 8 kHz format (ie the CODEC can operate according to G.711 format), or the 44.1 kHz samples could be used throughout. The latter option demands a much higher bandwidth and greatly increased processing speed: these might be acceptable if the audio signal being transmitted over the network needs to be of CD quality, but for normal voice communications the 64 kHz bandwidth signal of the G.711 format is perfectly adequate.

The procedure at a terminal which generates audio data is as follows (see also the flow charts of Figures 4 and 5). The DSP aggregates 64 samples in the G711 format together into blocks of 64 bytes, corresponding to 8 ms of data (step 130). This represents the basic unit of processing and transmission over the network. The DSP then transfers these blocks into a circular buffer in main memory 24 (see Figure 3a) on the computer system (step 132) using direct memory access (DMA), in accordance with known techniques. Thus every 8 ms, a new block of data is written into memory. The DSP effectively maintains a record of the location of the last block added to memory, and increments this by 64 bytes for each new transfer.Finally, the DSP raises an interrupt (step 134) in a thread running on the main processor 22, again in accordance with known interrupt processing techniques, informing it that another block of data has been added to memory. The DSP cycles through the loop shown in Figure 4 every 8 ms.

In the current implementation, the interrupt in fact is concurrent with the DMA transfer, and actually signals the presence of the block added to memory in the previous 8 ms cycle. There is no direct check that the DMA for this previous cycle, which is an asynchronous process, has completed, since in normal circumstances the DMA takes far less than 8 ms to finish. Such a check could be added if desired, although due to the real-time nature of audio signals, it is difficult to see what corrective action could be taken even if a problem were detected.

Figure 3a represents the series of memory 24 locations into which blocks of audio data from the DSP are received (normally this will be a linear set of addresses, but is configured in software as a circular buffer using known programming techniques). A thread 150 executing on the main processor 22 receives the interrupt from the DSP (step 152), and uses this to update a pointer P1 to the location of the most recently added block of data (step 154). Thread 150 simply cycles through this process, keeping track of the contents in the circular buffer.

Another thread 160 concurrently running on the main processor 24 also executes in a continuous loop, requesting access (step 162) to the network via the Token Ring card 30. This is performed using the NCB~SEND command in the NETBIOS interface (data transmission over a LAN is wellknown, and so will not be described in detail; see IBM Local Area Network Technical Reference, SC30-3383-03 for more information). This thread also maintains a pointer P2 representing the position in memory 24 of the last block of data sent (see Figure 3a). The difference between pointer P2, the position of the last block of data sent, and the pointer P1, the position of the block most recently added to the queue, determines how many blocks there are currently waiting to send (these are shown shaded in Figure 3a).Thus when thread 160 makes the call to NCB~SEND, it passes as parameters to the call the appropriate packet size and location in the circular buffer to allow the transmission of all the blocks currently waiting in the circular buffer. The thread 160 then waits for an acknowledgement that the packet has been received (step 164), a standard facility of NETBIOS, before updating pointer P2 to the new location of the last block of data sent.

At the receiving end a queue of audio blocks for output is maintained (see Figure 3b). The processing of this queue follows known prior art techniques. A pointer P3 is maintained indicating the current location in the queue to which incoming data blocks are added (readout occurs from the opposite end of the queue). The only difference from prior art processing is that the number of data blocks added to the queue at any one time is variable. The procedure to handle this is illustrated in Figure 6. An incoming data packet is received (step 170), and the number of audio blocks that it contains is determined (step 172). This can be achieved either from the size of the incoming message (NETBIOS provides this information), or alternatively, a separate field in the packet could be used to indicate the number of audio blocks contained in that particular packet. These audio blocks are then added to the queue (step 174) and pointer P3 updated accordingly. As stated above, processing from there on can be performed in conventional fashion (perhaps using an adaptive approach to control the length of the queue).

The system illustrated is particularly suitable for audio (voice) conferencing over a local area network, in which case each workstation will be typically acting as both a receiving and transmitting node, but could also be used in client server applications, where one node represents an audio source such as a multimedia server with compact disk facility, and play-out is to a user at another workstation, connected to the server via the network. Thus the sound can be transmitted from the server to the user's workstation using the approach described above.

Other modifications of the above system will also be readily apparent to the skilled person; for example, some form of data compression may be used to save bandwidth as the blocks are transmitted over the network, whether this be simple detection and excision of periods of silence, or the use of more sophisticated data compression techniques. Likewise, if the NETBIOS interface is not used (going either to a lower level interface, or perhaps a different type of network), there may be no need to specify the size of packet to be sent when network access is initially requested, but rather this can be determined at the time access to the network is actually obtained. In this case, a somewhat different processing strategy can be employed: each time access is granted, the difference between pointers P1 and P2 is calculated to determine the current number of audio blocks in the queue, and these are transmitted accordingly. P2 can then be updated to reflect the number of blocks sent without waiting for acknowledgement of their arrival.

Claims (15)

Claims

1. A method of processing audio signals at a first workstation for transmission over a network to a second workstation, comprising the steps of: generating a sequence of digital audio samples at the first workstation; storing digital audio samples in a queue at the first workstation; obtaining access to the network at the first workstation; and transmitting a packet containing essentially all the digital audio samples that are currently in the queue each time access is obtained to the network.

2. The method of claim 1, futher comprising the step of maintaining information indicating the number of digital audio samples in the queue, and updating said information each time a new digital audio sample is added to the queue or digital audio samples are transmitted from the queue.

3. The method of any preceding claim, wherein digital audio samples are generated at a constant frequency.

4. The method of claim 3, wherein a predetermined number of digital audio samples are accumulated into blocks of audio data, and each transmitted packet comprises an integral number of blocks of audio data.

5. The method of claim 4, wherein the digital audio samples are stored in blocks of audio data.

6. The method of claim 4 or 5, wherein each block of audio data represents 4 milliseconds of audio data.

7. The method of claim 4 or 5, wherein each block of audio data represents 8 milliseconds of audio data.

8. The method of claim 4 or 5, wherein each block of audio data represents 16 milliseconds of audio data.

9. The method of any preceding claim, further comprising the steps of: storing a queue of digital audio samples to be played out; receiving a data packet from the network containing a variable number of digital audio samples; and adding the received digital audio samples to the queue of digital audio samples to be played out.

10. A computer workstation comprising means for generating a sequence of digital audio samples for transmission over a computer network to a remote workstation; means for storing generated digital audio samples in a queue; means for obtaining access to the network; and means, responsive to access being obtained to the network, for transmitting a packet over the network containing essentially all the digital audio samples currently in the queue.

11. A computer workstation as claimed in claim 10, wherein the means for generating a sequence of digital audio samples comprises an audio adapter card.

12. A computer workstation as claimed in claim 1S, wherein the audio adapter card accumulates the digital audio samples into audio data blocks, each containing a predetermined number of digital audio samples, and transfers one audio data block at a time into main memory of the workstation.

13. A computer workstation as claimed in claim 12, further comprising means for maintaining information indicating the number of audio data blocks in the queue, and means for updating said information each time a new audio data block is added to the queue or one or more audio data blocks are transmitted from the queue.

14. A computer workstation as claimed in claim 12 or 13, wherein the audio adapter card raises an interrupt in the computer workstation each time an audio data block is transferred into main memory.

15. A computer workstation as claimed in any of claims 10 to 14, wherein the means for transmitting comprises a local area network adapter card.