CN101282386B - Method for forwarding synchronous mixed audio of VOIP server terminal - Google Patents

Abstract

The invention discloses a VOIP server terminal synchro audio-mix transmitting method, which belongs to sound signal processing technology field. Delayed estimation and compensation mechanism are employed aiming at network wobble and delay; estimation and compensation mechanism aiming at clock skew caused by clock deviation between the terminal and the server; furthermore, the audio-mix data is processed by designed cyclic buffers, synchronization between the cyclic buffers is kept by synchronous updating mechanism between a public cyclic buffer bit zone and the terminal cyclic buffers bit zones. Synchronous mechanism is employed in the invention, which enables synchro audio-mix of multi-path sound data by the server, and the data is synchronously transmitted to each terminals, which brings wider practical use to IP network sound conference.

Description

A kind of VOIP server end method for forwarding synchronous mixed audio

Technical field

The invention belongs to the voice process technology field.

Background technology

Along with development of internet technology, VOIP (Voice over Internet Protocol) is because its cheap cost of the phone call and good network amalgamation more and more have the trend that replaces the traditional PSTN phone.The TeleConference Bridge of IP based network utilizes original network line of enterprise and equipment to carry out the MPTY meeting, will use for enterprise saves huge telephone charges.As long as the caller passes through the PC terminal or the voip phone terminal just can be carried out meeting, and is very convenient.

The speed of conventional network equipment and bandwidth can be carried out the videoconference of IP based network fully.Along with the promotion of chip producer, will constantly descend based on the various mobile VOIP terminal of WI-FI and the price of other fixing VoIP terminal, this will make things convenient for enterprise to hold a meeting greatly, and the affairs of enhancing enterprises are passed on and are exchanged.

The key that realizes the TeleConference Bridge of IP network is the synchronous mixed audio algorithm of server, it need overcome shake (network jitter) and time-delay between the network, clock skewing between multi-path voice signal and the server (clock skew), the asynchronous problem that active situation caused of input voice.In addition, the high efficiency of audio mixing algorithm is also extremely important.

Summary of the invention

Be the shake (network jitter) between the network and delay time at three aspects noted earlier, back, the active situation of clock skewing that the clock jitter between terminal and the server causes (clock skew) and input voice, the present invention proposes a kind of server end of VOIP more efficiently method for forwarding synchronous mixed audio, its key step comprises:

1) decoded from the speech frame of RTP port taking-up by channel, carry out source coding again and obtain the raw tone waveform;

2) temporal information and the Playout Calc in conjunction with speech frame calculates Playouttime time of delay;

3), calculate the current state Skew status of clock skew by Clock Skew Calc in conjunction with the speech waveform and the Playout time of source coding;

4) the design cycle buffer is handled the audio mixing data;

5) last mixer Mixer utilizes skew status and playout time to control a plurality of flag bits of a plurality of cyclic buffers, the current speech waveform of wanting is added to the interval that flag bit limits, finish audio mixing, the multichannel audio mixing frame of output is forwarded to relevant terminal respectively.

Described step 2) Playout Calc module is at network jitter, adopt the estimation and the compensation mechanism of time-delay, by to estimating with the time-delay of last packet at first in the frame, receive time-delay estimation and the shake of network queue and the caused parlor variation of the change estimation of router before of any wrong error correction packets, adopt the classification compensation, calculate Playout time, eliminate the influence that above network factors is handled audio mixing.

Clock Skew Calc adopts the estimation and the compensation mechanism of clock skewing in the described step 3), estimates by the utilization index moving average, calculates the state of clock skewing, eliminates the influence that it is handled audio mixing.

Cyclic buffer in the described step 4) comprises public cyclic buffer and the cyclic buffer of setting up for every road terminal respectively.

Described step 5) set up synchronous update mechanism between public cyclic buffer flag bit and each terminal cyclic buffer flag bit keep between each cyclic buffer synchronously.

Beneficial effect of the present invention is: by the use of synchronization mechanism, make server to carry out synchronous mixed audio to the multi-path voice data, and be forwarded to each participant terminal synchronously, make the IP network voice conferencing more extensive in actual applications.

Description of drawings

Fig. 1 is a VOIP server end method for forwarding synchronous mixed audio schematic diagram;

Fig. 2 is the cyclic buffer structure in the Mixer module.

Embodiment

Below in conjunction with accompanying drawing the present invention is further set forth.

As shown in Figure 1, the speech frame that takes out from the RTP port is carried out channel-decoding on the one hand, and then carries out source coding and obtain the raw tone waveform, and the temporal information with speech frame is used for calculating playout time on the other hand.The current state that calculates clockskew with the speech waveform and the playout time of source coding, last mixer Mixer utilizes skew status and playout time to control the flag bit of a plurality of cyclic buffers, the current speech waveform of wanting is added to the interval that flag bit limits, finish audio mixing, the multichannel audio mixing frame of output is forwarded to terminal separately respectively.

Playout Calc module mainly calculates because the hysteresis reproduction time that factors such as network jitter and time-delay cause.

Clock Skew Calc module is then according in terminal and the server end, the timestamp difference of per two frame data judge the skew state (fast, slow, none).

Cyclic buffer in the Mixer module as shown in Figure 2.

The present invention mainly adopts following technology to realize the audio mixing and the transmission of multi-path voice:

1. the estimation of network jitter (network jitter) time-delay, monitoring and classification compensation mechanism;

2. adopt the monitoring and the compensation mechanism of clock skewing (clock skew);

3. the design cycle buffering area is handled and is deposited the audio mixing data;

4. keep between each cyclic buffer synchronously.

(1) shake of network, the estimation of time-delay and classification compensation mechanism;

Consider following three aspects:

In one frame at first with the time-delay of last packet;

The caused parlor of the shake of network queue and the change of router changes;

Receive the time-delay before of any wrong error correction packets;

(2) estimation and the compensation mechanism of employing clock skewing (clock skew);

d _n＝T _L(n)-T _R(n) (1)

T _{R (n)}And T _{L (n)}Be to be respectively the current time of advent and the timestamp of n frame,

$\mathrm{status}=\{\begin{array}{cc}\mathrm{slow}& E-D_n>\mathrm{skew}\_\mathrm{threshold}\\ \mathrm{fast}& E-D_n<\mathrm{skew}\_\mathrm{threshold}\\ \mathrm{none}& \mathrm{others}\end{array}---\left(2\right)$

D _nWith E is to be respectively average delay estimated value and movable time-delay.

Utilize above status to compensate operation again.

(3) the many cyclic buffers of design are deposited each road audio mixing data;

Except public cyclic buffer is arranged, set up cyclic buffer also for respectively every road terminal in the Mixer module.

(4) keep between each cyclic buffer synchronously;

Audio mixing is handled and is realized with subtraction.Public cyclic buffer is deposited the mixing voice data (mix of all active terminal _Total), these data send to inactive terminal after by subsequent processes.I active terminal cyclic buffer deposited the speech data mix of oneself _j, when sending, use mix again _TotalDeduct mix _jValue, send it back oneself after the subsequent processes.

${\mathrm{mix}}_{\mathrm{total}}=\underset{i}{\mathrm{\&Sigma;}}{\mathrm{mix}}_i---\left(3\right)$

mix _j＝mix _total-mix _j (4)

And the synchronous mark position " head " of each terminal cyclic buffer and " tail " follow public cyclic buffer to upgrade synchronously.

Tail increases a unit synchronously in every frame time, and each " head _i" in maximum compose to " head _Mix":

head _mix＝argmax(head _i) (5)

Based on network jitter is delayed time, the analysis-by-synthesis of the active situation of clock skewing and input speech signal reaches the requirement to the efficient utilization of resource, proposes a kind of server end of VOIP efficiently method for forwarding synchronous mixed audio.

Claims (1)

1. VOIP server end method for forwarding synchronous mixed audio, it comprises:

1) decoded from the speech frame of RTP port taking-up by channel, carry out source coding again and obtain the raw tone waveform;

2) in conjunction with the temporal information of speech frame, time of delay, computing module Playout Calc was at network jitter, adopt the estimation and the compensation mechanism of time-delay, by to estimating with the time-delay of last packet at first in the frame, receive time-delay estimation and the shake of network queue and the caused parlor variation of the change estimation of router before of any wrong error correction packets, adopt the classification compensation, calculate Playouttime time of delay, eliminate the influence that above network factors is handled audio mixing;

3) in conjunction with the speech waveform and the Playout time of source coding, clock skewing state computation module Clock Skew Calc adopts the estimation and the compensation mechanism of clock skewing, estimate by the utilization index moving average, calculate the current state skew status of clock skewing clock skew, eliminate the influence that it is handled audio mixing;

4) the design cycle buffer is handled the audio mixing data, and described cyclic buffer comprises public cyclic buffer and the cyclic buffer of setting up for every road terminal respectively;

5) last mixer Mixer utilizes skew status and playout time to control the flag bit of a plurality of cyclic buffers, the current speech waveform of wanting is added to the interval that flag bit limits, finish audio mixing, the multichannel audio mixing frame of output is forwarded to relevant terminal respectively; Set up synchronous update mechanism between public cyclic buffer flag bit and each terminal cyclic buffer flag bit keep between each cyclic buffer synchronously.

Images