CN101282386B - Method for forwarding synchronous mixed audio of VOIP server terminal - Google Patents
Method for forwarding synchronous mixed audio of VOIP server terminal Download PDFInfo
- Publication number
- CN101282386B CN101282386B CN2008100282214A CN200810028221A CN101282386B CN 101282386 B CN101282386 B CN 101282386B CN 2008100282214 A CN2008100282214 A CN 2008100282214A CN 200810028221 A CN200810028221 A CN 200810028221A CN 101282386 B CN101282386 B CN 101282386B
- Authority
- CN
- China
- Prior art keywords
- time
- delay
- estimation
- cyclic buffer
- terminal
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
- 230000001360 synchronised effect Effects 0.000 title claims abstract description 13
- 238000000034 method Methods 0.000 title claims abstract description 10
- 239000000872 buffer Substances 0.000 claims abstract description 32
- 125000004122 cyclic group Chemical group 0.000 claims abstract description 30
- 238000013461 design Methods 0.000 claims description 4
- 238000012937 correction Methods 0.000 claims description 3
- 230000002123 temporal effect Effects 0.000 claims description 3
- 230000003111 delayed effect Effects 0.000 abstract description 2
- 230000005236 sound signal Effects 0.000 abstract 1
- 238000012544 monitoring process Methods 0.000 description 2
- 238000005267 amalgamation Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 230000003139 buffering effect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 230000002708 enhancing effect Effects 0.000 description 1
- 238000003786 synthesis reaction Methods 0.000 description 1
Images
Landscapes
- Data Exchanges In Wide-Area Networks (AREA)
- Telephonic Communication Services (AREA)
Abstract
The invention discloses a VOIP server terminal synchro audio-mix transmitting method, which belongs to sound signal processing technology field. Delayed estimation and compensation mechanism are employed aiming at network wobble and delay; estimation and compensation mechanism aiming at clock skew caused by clock deviation between the terminal and the server; furthermore, the audio-mix data is processed by designed cyclic buffers, synchronization between the cyclic buffers is kept by synchronous updating mechanism between a public cyclic buffer bit zone and the terminal cyclic buffers bit zones. Synchronous mechanism is employed in the invention, which enables synchro audio-mix of multi-path sound data by the server, and the data is synchronously transmitted to each terminals, which brings wider practical use to IP network sound conference.
Description
Technical field
The invention belongs to the voice process technology field.
Background technology
Along with development of internet technology, VOIP (Voice over Internet Protocol) is because its cheap cost of the phone call and good network amalgamation more and more have the trend that replaces the traditional PSTN phone.The TeleConference Bridge of IP based network utilizes original network line of enterprise and equipment to carry out the MPTY meeting, will use for enterprise saves huge telephone charges.As long as the caller passes through the PC terminal or the voip phone terminal just can be carried out meeting, and is very convenient.
The speed of conventional network equipment and bandwidth can be carried out the videoconference of IP based network fully.Along with the promotion of chip producer, will constantly descend based on the various mobile VOIP terminal of WI-FI and the price of other fixing VoIP terminal, this will make things convenient for enterprise to hold a meeting greatly, and the affairs of enhancing enterprises are passed on and are exchanged.
The key that realizes the TeleConference Bridge of IP network is the synchronous mixed audio algorithm of server, it need overcome shake (network jitter) and time-delay between the network, clock skewing between multi-path voice signal and the server (clock skew), the asynchronous problem that active situation caused of input voice.In addition, the high efficiency of audio mixing algorithm is also extremely important.
Summary of the invention
Be the shake (network jitter) between the network and delay time at three aspects noted earlier, back, the active situation of clock skewing that the clock jitter between terminal and the server causes (clock skew) and input voice, the present invention proposes a kind of server end of VOIP more efficiently method for forwarding synchronous mixed audio, its key step comprises:
1) decoded from the speech frame of RTP port taking-up by channel, carry out source coding again and obtain the raw tone waveform;
2) temporal information and the Playout Calc in conjunction with speech frame calculates Playouttime time of delay;
3), calculate the current state Skew status of clock skew by Clock Skew Calc in conjunction with the speech waveform and the Playout time of source coding;
4) the design cycle buffer is handled the audio mixing data;
5) last mixer Mixer utilizes skew status and playout time to control a plurality of flag bits of a plurality of cyclic buffers, the current speech waveform of wanting is added to the interval that flag bit limits, finish audio mixing, the multichannel audio mixing frame of output is forwarded to relevant terminal respectively.
Described step 2) Playout Calc module is at network jitter, adopt the estimation and the compensation mechanism of time-delay, by to estimating with the time-delay of last packet at first in the frame, receive time-delay estimation and the shake of network queue and the caused parlor variation of the change estimation of router before of any wrong error correction packets, adopt the classification compensation, calculate Playout time, eliminate the influence that above network factors is handled audio mixing.
Clock Skew Calc adopts the estimation and the compensation mechanism of clock skewing in the described step 3), estimates by the utilization index moving average, calculates the state of clock skewing, eliminates the influence that it is handled audio mixing.
Cyclic buffer in the described step 4) comprises public cyclic buffer and the cyclic buffer of setting up for every road terminal respectively.
Described step 5) set up synchronous update mechanism between public cyclic buffer flag bit and each terminal cyclic buffer flag bit keep between each cyclic buffer synchronously.
Beneficial effect of the present invention is: by the use of synchronization mechanism, make server to carry out synchronous mixed audio to the multi-path voice data, and be forwarded to each participant terminal synchronously, make the IP network voice conferencing more extensive in actual applications.
Description of drawings
Fig. 1 is a VOIP server end method for forwarding synchronous mixed audio schematic diagram;
Fig. 2 is the cyclic buffer structure in the Mixer module.
Embodiment
Below in conjunction with accompanying drawing the present invention is further set forth.
As shown in Figure 1, the speech frame that takes out from the RTP port is carried out channel-decoding on the one hand, and then carries out source coding and obtain the raw tone waveform, and the temporal information with speech frame is used for calculating playout time on the other hand.The current state that calculates clockskew with the speech waveform and the playout time of source coding, last mixer Mixer utilizes skew status and playout time to control the flag bit of a plurality of cyclic buffers, the current speech waveform of wanting is added to the interval that flag bit limits, finish audio mixing, the multichannel audio mixing frame of output is forwarded to terminal separately respectively.
Playout Calc module mainly calculates because the hysteresis reproduction time that factors such as network jitter and time-delay cause.
Clock Skew Calc module is then according in terminal and the server end, the timestamp difference of per two frame data judge the skew state (fast, slow, none).
Cyclic buffer in the Mixer module as shown in Figure 2.
The present invention mainly adopts following technology to realize the audio mixing and the transmission of multi-path voice:
1. the estimation of network jitter (network jitter) time-delay, monitoring and classification compensation mechanism;
2. adopt the monitoring and the compensation mechanism of clock skewing (clock skew);
3. the design cycle buffering area is handled and is deposited the audio mixing data;
4. keep between each cyclic buffer synchronously.
(1) shake of network, the estimation of time-delay and classification compensation mechanism;
Consider following three aspects:
In one frame at first with the time-delay of last packet;
The caused parlor of the shake of network queue and the change of router changes;
Receive the time-delay before of any wrong error correction packets;
(2) estimation and the compensation mechanism of employing clock skewing (clock skew);
d
n=T
L(n)-T
R(n) (1)
T
R (n)And T
L (n)Be to be respectively the current time of advent and the timestamp of n frame,
D
nWith E is to be respectively average delay estimated value and movable time-delay.
Utilize above status to compensate operation again.
(3) the many cyclic buffers of design are deposited each road audio mixing data;
Except public cyclic buffer is arranged, set up cyclic buffer also for respectively every road terminal in the Mixer module.
(4) keep between each cyclic buffer synchronously;
Audio mixing is handled and is realized with subtraction.Public cyclic buffer is deposited the mixing voice data (mix of all active terminal
Total), these data send to inactive terminal after by subsequent processes.I active terminal cyclic buffer deposited the speech data mix of oneself
j, when sending, use mix again
TotalDeduct mix
jValue, send it back oneself after the subsequent processes.
mix
j=mix
total-mix
j (4)
And the synchronous mark position " head " of each terminal cyclic buffer and " tail " follow public cyclic buffer to upgrade synchronously.
Tail increases a unit synchronously in every frame time, and each " head
i" in maximum compose to " head
Mix":
head
mix=argmax(head
i) (5)
Based on network jitter is delayed time, the analysis-by-synthesis of the active situation of clock skewing and input speech signal reaches the requirement to the efficient utilization of resource, proposes a kind of server end of VOIP efficiently method for forwarding synchronous mixed audio.
Claims (1)
1. VOIP server end method for forwarding synchronous mixed audio, it comprises:
1) decoded from the speech frame of RTP port taking-up by channel, carry out source coding again and obtain the raw tone waveform;
2) in conjunction with the temporal information of speech frame, time of delay, computing module Playout Calc was at network jitter, adopt the estimation and the compensation mechanism of time-delay, by to estimating with the time-delay of last packet at first in the frame, receive time-delay estimation and the shake of network queue and the caused parlor variation of the change estimation of router before of any wrong error correction packets, adopt the classification compensation, calculate Playouttime time of delay, eliminate the influence that above network factors is handled audio mixing;
3) in conjunction with the speech waveform and the Playout time of source coding, clock skewing state computation module Clock Skew Calc adopts the estimation and the compensation mechanism of clock skewing, estimate by the utilization index moving average, calculate the current state skew status of clock skewing clock skew, eliminate the influence that it is handled audio mixing;
4) the design cycle buffer is handled the audio mixing data, and described cyclic buffer comprises public cyclic buffer and the cyclic buffer of setting up for every road terminal respectively;
5) last mixer Mixer utilizes skew status and playout time to control the flag bit of a plurality of cyclic buffers, the current speech waveform of wanting is added to the interval that flag bit limits, finish audio mixing, the multichannel audio mixing frame of output is forwarded to relevant terminal respectively; Set up synchronous update mechanism between public cyclic buffer flag bit and each terminal cyclic buffer flag bit keep between each cyclic buffer synchronously.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN2008100282214A CN101282386B (en) | 2008-05-22 | 2008-05-22 | Method for forwarding synchronous mixed audio of VOIP server terminal |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN2008100282214A CN101282386B (en) | 2008-05-22 | 2008-05-22 | Method for forwarding synchronous mixed audio of VOIP server terminal |
Publications (2)
Publication Number | Publication Date |
---|---|
CN101282386A CN101282386A (en) | 2008-10-08 |
CN101282386B true CN101282386B (en) | 2010-11-10 |
Family
ID=40014657
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN2008100282214A Expired - Fee Related CN101282386B (en) | 2008-05-22 | 2008-05-22 | Method for forwarding synchronous mixed audio of VOIP server terminal |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN101282386B (en) |
Families Citing this family (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN103248774B (en) * | 2012-02-13 | 2015-02-11 | 陈剑勇 | VoIP server synchronous sound mixing method and system |
CN105634757A (en) * | 2015-06-11 | 2016-06-01 | 宇龙计算机通信科技(深圳)有限公司 | Communication method, communication device, terminal and communication system |
CN107195308B (en) * | 2017-04-14 | 2021-03-16 | 苏州科达科技股份有限公司 | Audio mixing method, device and system of audio and video conference system |
CN109672946B (en) * | 2019-02-15 | 2023-12-15 | 深圳市昊一源科技有限公司 | Wireless communication system, forwarding equipment, terminal equipment and forwarding method |
CN114512139B (en) * | 2022-04-18 | 2022-09-20 | 杭州星犀科技有限公司 | Processing method and system for multi-channel audio mixing, mixing processor and storage medium |
Citations (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN101156444A (en) * | 2006-01-18 | 2008-04-02 | 华为技术有限公司 | Device, network appliance and method for video and audio signal transmission |
-
2008
- 2008-05-22 CN CN2008100282214A patent/CN101282386B/en not_active Expired - Fee Related
Patent Citations (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN101156444A (en) * | 2006-01-18 | 2008-04-02 | 华为技术有限公司 | Device, network appliance and method for video and audio signal transmission |
Also Published As
Publication number | Publication date |
---|---|
CN101282386A (en) | 2008-10-08 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
JP5632486B2 (en) | Method for scheduling transmissions in a communication network, corresponding communication node and computer program product | |
Singh et al. | VoIP: State of art for global connectivity—A critical review | |
US10454811B2 (en) | Apparatus and method for de-jitter buffer delay adjustment | |
KR100902456B1 (en) | Method and apparatus for managing end-to-end voice over internet protocol media latency | |
US7869420B2 (en) | Method and system for in-band signaling of multiple media streams | |
CN101282386B (en) | Method for forwarding synchronous mixed audio of VOIP server terminal | |
Sundstrom | Voice over IP, an engineering analysis | |
CN101288257A (en) | Method for signaling a device to perform no synchronization or include a synchronization delay on multimedia streams | |
US7769054B2 (en) | Method of conducting a communications session using incorrect timestamps | |
US8438016B2 (en) | Silence-based adaptive real-time voice and video transmission methods and system | |
WO2003013096A1 (en) | A speech transmitting method for saving the bandwidth | |
US20130036239A1 (en) | Synchronization of clocks between two communication terminals using tcp/ip | |
US7403497B2 (en) | Data transport between a media gateway and server | |
Labyad et al. | Performance analysis and comparative study of voice over IP using hybrid codec | |
US20130142192A1 (en) | Voice communication apparatus for intermittently discarding packets | |
EP3652738B1 (en) | Early transmission in packetized speech | |
JP5831095B2 (en) | Voice communication system, voice communication apparatus and program | |
JP2005151044A (en) | Voice mixing method, voice mixing system, and program for voice mixing | |
Li et al. | Implementation of a multimedia communication system over IP network | |
Narbutt et al. | Adaptive Anti-jitter Mechanism for Multi-Party Conferencing in a H. 323 Multi-Point Control Unit | |
RU41943U1 (en) | VIDEO CONFERENCE SYSTEM | |
Elliott | Stream synchronization for voice over IP conference bridges | |
Elliott et al. | Synchronization of Speaker Selection for Centralized Tandem Free VoIP Conferencing | |
Kneckt | point to point and point to multi point calls over IP | |
Ramos | Robust and reliable multimedia transmission over the Internet |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
C06 | Publication | ||
PB01 | Publication | ||
C10 | Entry into substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
C14 | Grant of patent or utility model | ||
GR01 | Patent grant | ||
C17 | Cessation of patent right | ||
CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20101110 Termination date: 20140522 |