CN116600162A - Progressive audio and video time stamp synchronization method and system - Google Patents
Progressive audio and video time stamp synchronization method and system Download PDFInfo
- Publication number
- CN116600162A CN116600162A CN202310736575.9A CN202310736575A CN116600162A CN 116600162 A CN116600162 A CN 116600162A CN 202310736575 A CN202310736575 A CN 202310736575A CN 116600162 A CN116600162 A CN 116600162A
- Authority
- CN
- China
- Prior art keywords
- synchronization
- audio
- node
- time stamp
- video data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 31
- 230000000750 progressive effect Effects 0.000 title claims abstract description 19
- 230000005540 biological transmission Effects 0.000 claims abstract description 16
- 238000012545 processing Methods 0.000 claims description 5
- 238000004891 communication Methods 0.000 claims description 4
- 238000001914 filtration Methods 0.000 claims description 3
- 238000013528 artificial neural network Methods 0.000 claims description 2
- 238000009499 grossing Methods 0.000 claims description 2
- 238000005516 engineering process Methods 0.000 description 4
- 238000013135 deep learning Methods 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 238000012544 monitoring process Methods 0.000 description 2
- 230000001360 synchronised effect Effects 0.000 description 2
- 230000007547 defect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 238000012423 maintenance Methods 0.000 description 1
- 230000000737 periodic effect Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/4302—Content synchronisation processes, e.g. decoder synchronisation
- H04N21/4305—Synchronising client clock from received content stream, e.g. locking decoder clock with encoder clock, extraction of the PCR packets
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/242—Synchronization processes, e.g. processing of PCR [Program Clock References]
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Synchronisation In Digital Transmission Systems (AREA)
Abstract
The invention provides a brand new progressive audio and video time stamp synchronization method and system. The method has the advantages of rapidness, accuracy, stability, reliability and the like, and is suitable for various network environments and scenes. By means of establishing a reference node and a reference clock, dynamically adjusting the clock synchronization state of the node and the like, the audio and video time stamp synchronization can be more accurately realized. The time stamp synchronization in audio and video data transmission can be realized efficiently, reliably and accurately, and the user's looking and feel experience is improved.
Description
Technical Field
The invention relates to the technical field of audio and video, in particular to an audio and video time stamp synchronization technology. In particular to a progressive audio and video time stamp synchronization method and system.
Background
With the wide application of multimedia data, audio and video technologies are rapidly developed. In the process of audio and video data transmission, processing and playing, the synchronization of audio and video time stamps is very critical. When the audio and video time stamps are not synchronous, the problems of image jitter, intermittent and the like can be caused, and the viewing experience of a user is affected. Therefore, achieving accurate audio and video timestamp synchronization is one of the hot spots of current audio and video technology research. With the increasing development of wireless communication technology, the real-time transmission requirement of multiple paths of audios and videos in the industries of video conferences, online education and the like is more and more urgent. However, since clocks of each transmission device are different, a time stamp deviation of an audio and video signal is easily caused, thereby affecting the effect of video playback and sound synchronization. The main methods currently solving this problem include synchronizing clocks using NTP (network time protocol) or adding time stamp information in the transmitted data packets. Some audio and video time stamp synchronization methods based on algorithms such as time domain filtering, statistics, deep learning and the like. However, these methods still have some problems, such as low synchronization accuracy for distributed video streams, very difficult signal discrimination, etc. Therefore, a more efficient and accurate audio-video timestamp synchronization method is needed.
Disclosure of Invention
The invention provides a progressive audio and video time stamp synchronization method and system. The method aims to realize accurate synchronization in audio and video data transmission, overcome the defects and the shortcomings existing in the prior art, and improve the efficiency and the accuracy of audio and video time stamp synchronization.
Specifically, the progressive audio and video time stamp synchronization method and system of the invention comprise the following steps.
S11: and determining an audio and video data transmission route, and selecting a specific reference node as a standard of the whole synchronous system. On a specific reference node, a time synchronization system is established and the offset between the clocks to this node and the other nodes is obtained. The offset may be obtained through a network or other communication means.
And S12, synchronizing the clocks of each node by using a set of accurate reference clocks. To optimize the synchronization effect, the clock rate may be adjusted in some nodes. This allows better maintenance of the synchronisation results.
And S13, processing each audio and video data stream according to the time stamp information received from the specific reference node so as to keep synchronization in the transmission process. The specific method comprises the following steps:
1. receiving an audio and video data stream and acquiring time stamp information of the data stream;
2. correcting the time stamp of each data packet;
3. the synchronization error is smoothed by interpolation or the like.
And S14, when a new node is added into the network, the clock synchronization state of the node can be dynamically adjusted through an incremental updating algorithm or other methods, so that progressive improvement is realized. The system clock bias is reduced while maintaining timestamp synchronization.
The invention provides a progressive audio and video time stamp synchronization method and a progressive audio and video time stamp synchronization system, which have the following advantages:
by determining a particular reference node and establishing a reference clock, it is possible to allow all nodes to perform more accurate time synchronization while reducing system clock skew.
By adopting the expansion type synchronization method, when a new node joins the network, the node synchronization state can be dynamically adjusted by incremental updating algorithm and the like, so that progressive improvement is realized.
Introducing machine learning techniques, such as deep learning, can further optimize the synchronization efficiency and accuracy of the algorithm.
The method is suitable for various network environments and can cope with different scenes and complex network topological structures.
Based on the method, high-efficiency, reliable and accurate audio and video data transmission can be realized, and the user's looking and feel experience is improved.
Drawings
Fig. 1 is a flowchart of a progressive audio and video time stamp synchronization method.
Fig. 2 is a schematic diagram of a progressive audio/video timestamp synchronization method and system implementation mode.
Fig. 3 is a block diagram of a PLL-based clock synchronization system.
Detailed Description
The following describes specific embodiments of the present invention in connection with specific examples.
The progressive audio and video time stamp synchronization method shown in fig. 1 specifically comprises the following steps:
s1: selecting a specific node in an audio and video data transmission route as a reference node, and establishing a time synchronization system;
s2: synchronizing and correcting the clocks of each node by using a set of accurate reference clocks, and optimizing clock synchronization results;
s3: processing each audio and video data stream according to the time stamp information received from the specific reference node so as to keep synchronization in the transmission process, including receiving the audio and video data stream, correcting the time stamp of each data packet, and smoothing the synchronization error by interpolation and other methods;
s4: when a new node is added to the network, the clock synchronization state of the node can be dynamically adjusted by an incremental update algorithm or other methods, thereby realizing progressive improvement and reducing the system clock deviation while maintaining the time stamp synchronization.
As shown in fig. 2, this embodiment mode includes one reference node (S) and three working nodes (A, B, C). The reference node sends a clock synchronization message to the working node via a network or other communication means, and the working node receives and uses the message to adjust its own internal clock. In order to make the clock synchronization more accurate, the reference node and the working node can perform periodic state monitoring, so that the clock synchronization is adjusted according to the monitoring data.
As shown in fig. 3, the time synchronization system in this embodiment mode includes a clock Phase Locked Loop (PLL). The PLL calculates the clock offset and rate adjustment value to be adjusted by comparing the time difference between the local clock and the reference clock and the rate of change of the reference clock. Thus, clock synchronization can be more accurate and stable.
In this embodiment, the node synchronization state may be dynamically adjusted by an incremental update algorithm as new nodes join the network, thereby achieving progressive improvement. In particular, when a new node is connected into the network, there may be some skew between its internal clock and the reference clock. At this time, the clock offset of the node may be calculated by an incremental update algorithm and transmitted to the reference node. After receiving the information, the reference node calculates the relative time difference between the node and other nodes and sends corresponding clock adjustment information. When the node receives the clock adjustment information, the node can synchronize according to the clock of the reference node, so that the time synchronism of the whole system is maintained.
The incremental update algorithm may be implemented in a variety of ways, such as Kalman filtering, linear prediction, neural networks, and so forth. In this embodiment, an incremental update algorithm based on linear prediction is used. The algorithm predicts the clock offset of the nodes by analyzing the track data among the nodes and adjusts the clock offset by taking the clock offset as a reference value. The algorithm can reduce the system clock deviation and improve the synchronization efficiency and precision while keeping time synchronization.
Claims (7)
1. A progressive audio and video time stamp synchronization method, the method comprising the steps of:
s1: selecting a specific node in an audio and video data transmission route as a reference node, and establishing a time synchronization system;
s2: synchronizing and correcting the clocks of each node by using a set of accurate reference clocks, and optimizing clock synchronization results;
s3: processing each audio and video data stream according to the time stamp information received from the specific reference node so as to keep synchronization in the transmission process, including receiving the audio and video data stream, correcting the time stamp of each data packet, and smoothing the synchronization error by interpolation and other methods;
s4: when a new node is added to the network, the clock synchronization state of the node can be dynamically adjusted by an incremental update algorithm or other methods, thereby realizing progressive improvement and reducing the system clock deviation while maintaining the time stamp synchronization.
2. The method according to claim 1, wherein a specific node is selected as a reference node in the audio/video data transmission route, and a time synchronization system is established, specifically, the audio/video data transmission route is determined, and a specific reference node is selected as a reference of the whole synchronization system. On a specific reference node, a time synchronization system is established and the offset between the clocks to this node and the other nodes is obtained. The offset may be obtained through a network or other communication means.
3. The method of claim 1, wherein the incremental update algorithm is implemented by any one of Kalman filtering, linear prediction, neural network, etc.
4. A progressive audio video timestamp synchronization system comprising:
a specific reference node for determining the transmission route of the audio and video data and establishing a time synchronization system;
a set of accurate reference clocks for synchronizing and correcting the clocks of each node;
an audio/video data processor for processing each audio/video data stream according to the time stamp information received from the specific reference node so as to keep synchronization during transmission;
and the incremental updating module is used for dynamically adjusting the clock synchronization state of the node so as to realize progressive improvement and reduce the system clock deviation while maintaining the time stamp synchronization.
5. A system according to claim 3, wherein said audio video data processor comprises a receiver, a time stamp corrector and a synchronization error smoother.
6. The system of claim 3, wherein the incremental update module is implemented using a linear prediction based incremental update algorithm.
7. The system of claim 3, wherein the incremental update module comprises a trajectory analyzer, a predictor, and a clock adjuster.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310736575.9A CN116600162A (en) | 2023-06-21 | 2023-06-21 | Progressive audio and video time stamp synchronization method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310736575.9A CN116600162A (en) | 2023-06-21 | 2023-06-21 | Progressive audio and video time stamp synchronization method and system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116600162A true CN116600162A (en) | 2023-08-15 |
Family
ID=87593835
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310736575.9A Pending CN116600162A (en) | 2023-06-21 | 2023-06-21 | Progressive audio and video time stamp synchronization method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116600162A (en) |
-
2023
- 2023-06-21 CN CN202310736575.9A patent/CN116600162A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| JP5497051B2 (en) | Improved method, system and apparatus for signal synchronization | |
| CN101160900B (en) | Stream synchronization method of multimedia live transmission in packet network and device thereof | |
| KR101496108B1 (en) | Method and apparatus for time sychronisation in wireless networks | |
| US7606928B2 (en) | Method and device for controlling receiver buffer fullness level in multimedia streaming | |
| EP2150062B1 (en) | Improved method, system and apparatus for synchronizing signals | |
| US6493832B1 (en) | Communication apparatus which handles a time stamp | |
| EP1256197B1 (en) | Reference time distribution over a network | |
| US7035246B2 (en) | Maintaining a global time reference among a group of networked devices | |
| AU2002258481A1 (en) | Maintaining a global time reference among a group of networked devices | |
| EP3053286A1 (en) | Method and devices for synchronization | |
| CN106712885A (en) | Clock synchronization method, network device and network system | |
| CN102065060B (en) | Media stream switching synchronization method and streaming media server | |
| WO2005002100A1 (en) | High accuracy network clock synchronization | |
| JP2007306497A (en) | Clock synchronization method and circuit in packet transmission | |
| US7424080B1 (en) | Method and system for providing jitter-free transmissions for demodulated data stream | |
| CN116600162A (en) | Progressive audio and video time stamp synchronization method and system | |
| CN117957798A (en) | Dynamic packet rate for improved timing and performance | |
| CN118044135A (en) | Dynamic traffic load compensation |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |