CN109194985B - Audio and video Ethernet transmission method and system - Google Patents
Audio and video Ethernet transmission method and system Download PDFInfo
- Publication number
- CN109194985B CN109194985B CN201811377474.2A CN201811377474A CN109194985B CN 109194985 B CN109194985 B CN 109194985B CN 201811377474 A CN201811377474 A CN 201811377474A CN 109194985 B CN109194985 B CN 109194985B
- Authority
- CN
- China
- Prior art keywords
- code stream
- audio
- video
- clock
- decoding
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/234—Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs
- H04N21/2343—Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/236—Assembling of a multiplex stream, e.g. transport stream, by combining a video stream with other content or additional data, e.g. inserting a URL [Uniform Resource Locator] into a video stream, multiplexing software data into a video stream; Remultiplexing of multiplex streams; Insertion of stuffing bits into the multiplex stream, e.g. to obtain a constant bit-rate; Assembling of a packetised elementary stream
- H04N21/23608—Remultiplexing multiplex streams, e.g. involving modifying time stamps or remapping the packet identifiers
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/242—Synchronization processes, e.g. processing of PCR [Program Clock References]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/434—Disassembling of a multiplex stream, e.g. demultiplexing audio and video streams, extraction of additional data from a video stream; Remultiplexing of multiplex streams; Extraction or processing of SI; Disassembling of packetised elementary stream
- H04N21/4344—Remultiplexing of multiplex streams, e.g. by modifying time stamps or remapping the packet identifiers
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs
- H04N21/4402—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Compression Or Coding Systems Of Tv Signals (AREA)
- Synchronisation In Digital Transmission Systems (AREA)
Abstract
The invention provides an audio and video Ethernet transmission method and a system thereof, wherein the transmission method is based on at least an encoding system, a decoding system and an Ethernet transmission system; the transmission method at least comprises the following steps: coding the audio and video to generate audio and video code stream and time information; multiplexing the audio and video code stream and the time information to form a code stream; the code stream is transmitted to a decoding system through the Ethernet transmission system; the decoding system receives the code stream through the Ethernet port and buffers the code stream; carrying out data smooth statistics on the code stream; synchronously generating a decoding clock; recovering the original rate of the code stream according to the decoding clock; and decoding the code stream and recovering audio and video information. The method solves the defects that the buffer area of the decoder is easy to overflow and the clock of the decoding end and the clock of the encoding end can not keep synchronous during low-delay audio and video transmission, and provides a high-efficiency audio and video transmission mode.
Description
Technical Field
The present invention relates to audio and video transmission technologies, and in particular, to a low-latency audio and video transmission method and system.
Background
With the continuous development of digital media, audio and video transmission is more frequent, and the requirements on transmission quality and time are more strict. The delay of the current low-delay coding and decoding system is generally lower than 40MS, and compared with the common coding and decoding system with the previous delay of 800-1000 MS, the low-delay coding and decoding system has low resistance to code rate jitter. Fig. 1 is a schematic diagram of a proprietary medium transmission system, in the system using the proprietary medium transmission, after a code stream is transmitted through the proprietary medium, an original code rate is normally reserved, a code stream speed is stable, and an arrival time of clock information is determined. Therefore, the received code stream can be directly transmitted to a decoder for decoding after being demultiplexed. The code stream contains the clock information of the encoding end and is used for clock recovery of decoding of the far-end decoder. Because the arrival time of the system clock information is determined, the decoding end can recover the decoding clock by using the time information, and the synchronization performance of system encoding and decoding is ensured.
Compared with the transmission of a special medium, the audio and video system which uses the Ethernet for transmission can have longer transmission distance, lower wiring cost and more flexible transmission scheme, but because the whole coding and decoding delay of the low-delay coding and decoding system does not exceed 40 milliseconds, the space of a decoder buffer is very small, and the system is extremely sensitive to the rate change of the code stream. Meanwhile, after the code stream is transmitted by the Ethernet, the original code rate is lost, the speed of the code stream shows a severe fluctuation state, and the arrival time of clock information is uncertain. When there is clock skew, decoding delay cannot be controlled, and decoding errors and delay performance deterioration may occur after long-time operation. In order to solve the problems, the invention provides a low-delay audio and video transmission method and a low-delay audio and video transmission system.
Disclosure of Invention
The invention aims to provide a low-delay audio and video transmission method and a low-delay audio and video transmission system, which are used for finishing low-delay transmission of audio and video under the conditions that code stream data is not lost and a decoding end clock and a coding end clock are kept synchronous.
In order to achieve the above purpose, the invention provides a technical scheme that: an audio and video Ethernet transmission method is based on at least an encoding system, a decoding system and an Ethernet transmission system; wherein the encoding system and the decoding system respectively comprise at least one Ethernet port;
the transmission method at least comprises the following steps:
(1) carrying out progressive encoding on the audio and video to generate an audio and video code stream and time information;
(2) multiplexing the audio and video code stream and the time information to form a code stream;
(3) the code stream is transmitted to a decoding system through the Ethernet transmission system;
(4) the decoding system receives the code stream through the Ethernet port and buffers the code stream;
(5) carrying out data smooth statistics on the code stream;
(6) synchronously generating a decoding clock;
(7) recovering the original rate of the code stream according to the decoding clock;
(8) and decoding the code stream and recovering audio and video information.
Wherein the step (4) comprises detecting the buffer state of the code stream.
Wherein, the step (6) comprises calculating a preset water level and generating a decoding clock by using a clock control algorithm;
and the step (7) comprises demultiplexing the code stream, decoding time information in the code stream and counting the data volume of the code stream.
In order to realize the method, the invention also provides an audio and video Ethernet transmission system, which comprises an Ethernet system and at least two terminal systems, wherein the terminal systems at least comprise an encoding system and a decoding system;
wherein the encoding system comprises at least:
the encoder is used for encoding the audio and video to generate an audio and video code stream and encoding an audio and video clock to generate time information;
the multiplexer is used for multiplexing the audio and video code stream and the time information to generate a code stream; and at least one Ethernet port for transmitting the code stream;
the decoding system comprises at least:
at least one ethernet port for receiving the transcoded data;
the buffer is used for buffering the code stream;
the smoother is used for carrying out data smooth statistics on the code stream;
the clock module is used for synchronously generating a decoding clock;
the clock restorer is used for restoring the original rate of the code stream according to the decoding clock; and
and the decoder is used for decoding the code stream and recovering the audio and video information.
The buffer comprises a state detection module for detecting the buffer state of the code stream.
The clock module is also used for calculating a preset water level and generating a decoding clock by using a clock control algorithm.
The clock restorer comprises a clock restoring module, and the clock restoring module is used for demultiplexing the code stream, decoding time information in the code stream and counting data volume of the code stream.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art that other alternative embodiments can be obtained based on these drawings without inventive effort.
Fig. 1 is a schematic diagram of a system for audio/video transmission using a dedicated medium.
Fig. 2 is a schematic diagram of an encoding system for audio/video transmission using ethernet according to the present invention.
Fig. 3 is a schematic diagram of a decoding system for audio/video transmission using ethernet according to the present invention.
Effects of the invention
Compared with the prior art, the invention has the advantages that:
(1) the audio and video Ethernet transmission method and the system provided by the invention can have longer transmission distance, lower wiring cost and more flexible transmission scheme;
(2) the method and the system provided by the invention can solve the problem that the space of a decoder buffer is very small under low delay, and the system is extremely sensitive to the rate change of the code stream. Meanwhile, after the code stream is transmitted by the Ethernet, the audio and video output is still carried out according to the original code rate.
(3) The method and the system provided by the invention have stability, and the phenomena of decoding delay uncontrollable, decoding error caused by long-time operation, delay performance degradation and the like can not occur under the condition of clock deviation.
Detailed Description
The present invention is further illustrated by the following examples, which should not be construed as limiting the scope of the invention.
The example provides an audio and video Ethernet transmission system, which comprises an Ethernet system, an encoding system and a decoding system; wherein the encoding system comprises at least an encoder, a multiplexer, an Ethernet port and other necessary components; the decoding system includes at least an ethernet port, a buffer, a smoother, a clock module, a clock recoverer, a clock recovery module, a decoder, and other necessary components. The step flow of audio and video Ethernet transmission according to the system is as follows:
as shown in fig. 2, the encoder encodes audio and video to be transmitted, including audio and video encoding and clock encoding, wherein the clock encoding generates an encoding clock and further generates clock information (PCR _ Enc), the audio and video encoding generates an audio and video code stream according to the encoding clock, performs encoding rate control according to the encoding clock, and generates a code stream output clock; and finally, the multiplexer multiplexes the audio and video code stream according to the clock information and the code stream output clock to generate a code stream, and the code stream is sent to a decoding system through the Ethernet port.
As shown in fig. 3, after the decoding system buffers the received code stream, the state of the buffer is sampled according to the arrival time of the time information (PCR), and the state detection module obtains the buffer Level (Buff _ Level) according to the sampling of the buffer Level.
The smoother processes the buffer water Level (Buff _ Level) by using an infinite impulse response low-pass filter (IIR LPF), eliminates high-frequency interference, and obtains an average buffer water Level (Buff _ Avg).
The method comprises the steps of using a PID control algorithm commonly used in industrial control as a Clock control algorithm, using an average buffer water level (Bufflevel _ Avg) obtained by a data smoothing module as an input of a Clock module, comparing the average buffer water level (Bufflevel _ Avg) with a preset buffer water level (Bufflevel _ Ref) to obtain a Clock error, obtaining a Clock control quantity (Clock _ Ctr) after calculation of the PID algorithm as an output to generate a decoding Clock, and further generating Clock information (PRC _ DEC) by the decoding Clock.
The clock restorer receives clock information, wherein the clock restorer demultiplexes the code stream to obtain a time information Data packet of the code stream, decodes the Data packet to extract time information (PCR _ TS) intervals, and calculates Data quantity (Data _ Bytes) according to the statistical intervals;
the clock recoverer estimates a Rate (Rate), wherein:
Rate＝DataBytes/(PCR_n–PCR_n-1)；
and smoothing the code Rate (Rate) by a finite impulse response low pass filter (FIR LPF) to obtain an average code Rate (Rate _ Avg) to cope with the abnormal state.
And the clock restorer outputs stable code streams according to the rate control buffer.
And finally, the decoder decodes the stable code stream according to the clock information and outputs the audio and video.
According to the method steps and the corresponding device, the example performs audio and video transmission tests under two network environments, and the test results are shown in the following tables 1 and 2:
TABLE 1
TABLE 2
The method and the system for transmitting the audio and video ethernet provided by the invention are described in detail, a specific example is applied in the text to explain the principle and the implementation mode of the invention, and the description of the implementation is only used for helping to understand the method and the core idea of the invention; while the invention has been described in detail and with reference to specific embodiments thereof, it will be apparent to one skilled in the art that various changes in form and detail may be made therein without departing from the spirit and scope of the invention as defined by the appended claims.
Claims (4)
1. An audio and video Ethernet transmission method is characterized in that the transmission method is based on at least an encoding system, a decoding system and an Ethernet transmission system; wherein the encoding system and the decoding system respectively comprise at least one Ethernet port;
the transmission method at least comprises the following steps:
(1) coding the audio and video to generate audio and video code stream and time information;
(2) multiplexing the audio and video code stream and the time information to form a code stream;
(3) the code stream is transmitted to a decoding system through the Ethernet transmission system;
(4) the decoding system receives the code stream through the Ethernet port and buffers the code stream;
(5) carrying out data smoothing treatment on the code stream to obtain an average buffer water level;
(6) comparing the average buffer water level with a preset buffer water level to obtain a clock error, and generating a decoding clock by using a clock control algorithm;
(7) receiving clock information, demultiplexing the code stream, decoding time information in the code stream, counting the data volume of the code stream, and estimating the rate according to the data volume of the code stream;
(8) the clock restorer outputs a stable code stream according to the rate control buffer;
(9) and decoding the code stream and recovering audio and video information.
2. The av ethernet transmission method according to claim 1, wherein said step (4) comprises detecting a buffer status of said codestream.
3. An audio and video Ethernet transmission system is characterized in that the system comprises an Ethernet system and at least two terminal systems, wherein each terminal system at least comprises a coding system and a decoding system;
wherein the encoding system comprises at least:
the encoder is used for encoding the audio and video to generate an audio and video code stream and encoding an audio and video clock to generate time information;
the multiplexer is used for multiplexing the audio and video code stream and the time information to generate a code stream; and
at least one Ethernet port for transmitting the code stream;
the decoding system comprises at least:
at least one ethernet port for receiving the transcoded data;
the buffer is used for buffering the code stream;
the smoother is used for smoothing data of the code stream to obtain an average buffer water level;
the clock module is used for comparing the average buffer water level with a preset buffer water level to obtain a clock error and generating a decoding clock by using a clock control algorithm;
the clock restorer is used for receiving clock information, demultiplexing the code stream, decoding time information in the code stream, counting the data volume of the code stream, estimating the rate according to the data volume of the code stream, and outputting a stable code stream according to the rate control buffer; and the decoder is used for decoding the code stream and recovering the audio and video information.
4. The audiovisual ethernet transmission system of claim 3, wherein said buffer comprises a status detection module for detecting a buffer status of said codestream.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811377474.2A CN109194985B (en) | 2018-11-19 | 2018-11-19 | Audio and video Ethernet transmission method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811377474.2A CN109194985B (en) | 2018-11-19 | 2018-11-19 | Audio and video Ethernet transmission method and system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109194985A CN109194985A (en) | 2019-01-11 |
| CN109194985B true CN109194985B (en) | 2021-03-26 |
Family
ID=64939977
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811377474.2A Active CN109194985B (en) | 2018-11-19 | 2018-11-19 | Audio and video Ethernet transmission method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109194985B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112616077A (en) * | 2020-12-14 | 2021-04-06 | 杭州当虹科技股份有限公司 | CBR code rate sending control method based on IP |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2003284002A (en) * | 2002-03-27 | 2003-10-03 | Hitachi Ltd | Digital data recording device and reproducing device |
| CN101466045A (en) * | 2007-12-20 | 2009-06-24 | 上海奇码数字信息有限公司 | Adaptive decoding synchronous device, synchronous method as well as video decoding and displaying system |
| CN101484939A (en) * | 2006-07-06 | 2009-07-15 | 高通股份有限公司 | Clock drift compensation techniques for audio decoding |
| CN103079081A (en) * | 2009-04-08 | 2013-05-01 | 索尼公司 | Playback device, playback method, and recording medium |
| CN105306987A (en) * | 2015-10-23 | 2016-02-03 | 深圳国微技术有限公司 | Device for controlling output code rate of TS stream interface |
| CN108449567A (en) * | 2018-03-23 | 2018-08-24 | 广州市奥威亚电子科技有限公司 | A kind of method and device being used for transmission digital video |
-
2018
- 2018-11-19 CN CN201811377474.2A patent/CN109194985B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2003284002A (en) * | 2002-03-27 | 2003-10-03 | Hitachi Ltd | Digital data recording device and reproducing device |
| CN101484939A (en) * | 2006-07-06 | 2009-07-15 | 高通股份有限公司 | Clock drift compensation techniques for audio decoding |
| CN101466045A (en) * | 2007-12-20 | 2009-06-24 | 上海奇码数字信息有限公司 | Adaptive decoding synchronous device, synchronous method as well as video decoding and displaying system |
| CN103079081A (en) * | 2009-04-08 | 2013-05-01 | 索尼公司 | Playback device, playback method, and recording medium |
| CN105306987A (en) * | 2015-10-23 | 2016-02-03 | 深圳国微技术有限公司 | Device for controlling output code rate of TS stream interface |
| CN108449567A (en) * | 2018-03-23 | 2018-08-24 | 广州市奥威亚电子科技有限公司 | A kind of method and device being used for transmission digital video |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109194985A (en) | 2019-01-11 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN1981492B (en) | Buffer level signaling for rate adaptation in multimedia streaming | |
| KR101849749B1 (en) | Method for synchronizing an audio and a video in the transcoding system | |
| KR101253185B1 (en) | Digital broadcasting system and data processing method | |
| US11115453B2 (en) | Methods and apparatus for communicating delay information and minimizing delays | |
| EP1497937A2 (en) | Apparatus and method for synchronization of audio and video streams | |
| JP5972616B2 (en) | Reception device, clock restoration method, and program | |
| EP1501228A1 (en) | Encoded packet transmission/reception method, device thereof, and program | |
| CN101379556B (en) | Controlling a time-scaling of an audio signal | |
| US20150030088A1 (en) | Clock recovery for media stream in bursty network channel | |
| CN109194985B (en) | Audio and video Ethernet transmission method and system | |
| KR101787424B1 (en) | Mechanism for clock recovery for streaming content being communicated over a packetized communication network | |
| KR20120040519A (en) | Adaptive multimedia decoding device and method for scalable satellite broadcasting | |
| CN101076120B (en) | Strong system for maintaining audio frequency/video frequency synchronization | |
| US8854964B2 (en) | Method and apparatus for determining a transport bit rate for a Multiprogram transport stream | |
| CN1303771C (en) | A communication system having bad frame indicator means for resynchronization purposes | |
| KR100864009B1 (en) | Lip-synchronize method | |
| CN106603216B (en) | e1 timing recovery device of wireless microwave communication system and application thereof | |
| CN101378512B (en) | Method and device for synchronizing audio and video | |
| US6829304B1 (en) | Method for clock recovery in MPEG systems | |
| CN100539688C (en) | First kind of data flow is converted to the method and apparatus of second kind of data flow | |
| CN111988641B (en) | Transport stream multiplexing audio and video time synchronization method | |
| Nakamura et al. | Super high resolution video codec system with multiple MPEG-s HDTV codec LSI's | |
| JPH0289472A (en) | Picture coding transmitter | |
| JP6229597B2 (en) | Data decoding apparatus and data decoding method | |
| JP3705231B2 (en) | Transport stream splicer |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |