CN105142002B - Audio/video live broadcasting method, device and control method, device - Google Patents
Audio/video live broadcasting method, device and control method, device Download PDFInfo
- Publication number
- CN105142002B CN105142002B CN201510483445.4A CN201510483445A CN105142002B CN 105142002 B CN105142002 B CN 105142002B CN 201510483445 A CN201510483445 A CN 201510483445A CN 105142002 B CN105142002 B CN 105142002B
- Authority
- CN
- China
- Prior art keywords
- duration
- buffering area
- buffer
- data
- value
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 52
- 239000000872 buffer Substances 0.000 claims abstract description 184
- 230000003139 buffering effect Effects 0.000 claims abstract description 151
- 238000011084 recovery Methods 0.000 claims abstract description 11
- 238000004904 shortening Methods 0.000 claims 1
- 230000005540 biological transmission Effects 0.000 description 20
- 230000006870 function Effects 0.000 description 14
- 230000006872 improvement Effects 0.000 description 14
- 238000010586 diagram Methods 0.000 description 9
- 238000005516 engineering process Methods 0.000 description 4
- 239000012556 adjustment buffer Substances 0.000 description 3
- 230000008859 change Effects 0.000 description 3
- 230000003068 static effect Effects 0.000 description 3
- 238000004891 communication Methods 0.000 description 2
- 230000007423 decrease Effects 0.000 description 2
- 230000001934 delay Effects 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000009467 reduction Effects 0.000 description 2
- 230000003044 adaptive effect Effects 0.000 description 1
- 230000006837 decompression Effects 0.000 description 1
- 230000003247 decreasing effect Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 230000002452 interceptive effect Effects 0.000 description 1
- 238000012886 linear function Methods 0.000 description 1
- 238000002360 preparation method Methods 0.000 description 1
- 238000004321 preservation Methods 0.000 description 1
- 230000008569 process Effects 0.000 description 1
- 230000001360 synchronised effect Effects 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/433—Content storage operation, e.g. storage operation in response to a pause request, caching operations
- H04N21/4331—Caching operations, e.g. of an advertisement for later insertion during playback
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/433—Content storage operation, e.g. storage operation in response to a pause request, caching operations
- H04N21/4335—Housekeeping operations, e.g. prioritizing content for deletion because of storage space restrictions
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/434—Disassembling of a multiplex stream, e.g. demultiplexing audio and video streams, extraction of additional data from a video stream; Remultiplexing of multiplex streams; Extraction or processing of SI; Disassembling of packetised elementary stream
- H04N21/4341—Demultiplexing of audio and video streams
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/439—Processing of audio elementary streams
- H04N21/4392—Processing of audio elementary streams involving audio buffer management
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
- H04N21/44004—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving video buffer management, e.g. video decoder buffer or video display buffer
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
The invention discloses it is a kind of based on TCP live broadcast systems eliminate delay variation control method, including：Judge whether current buffer is empty；When not for sky, when currently the adjustment period reaches, if the maximum buffer data duration and the difference of minimal buffering area data duration in the currently adjustment period are less than preset buffering area duration dithering threshold, and minimal buffering area data duration is more than preset minimal buffering area duration threshold value, and current buffer data duration is shortened；When for sky, area to be buffered is arrived to be empty when receiving first data packet, and initial buffer area data duration is increased, and the initial buffer area data duration after increase is the increasing function of second largest delay variation value；Initial buffer area data duration is that buffering area plays required buffered data duration to be empty to recovery；Second largest delay variation value is buffering area be it is empty before preset time period to the time for receiving first data packet in received continuous two data packets time difference in Second Largest Value.
Description
Technical field
The present invention relates to network transmission technology field more particularly to a kind of delay variation is eliminated based on TCP live broadcast systems
Audio/video live broadcasting method, device and corresponding control method, device.
Background technology
Video living transmission system be scene vision signal (such as camera, video camera) and voice signal (such as microphone) into
Acquisition encodes row in real time, is sent to server up by network, user can in real time be watched with used network.One simplest
Video living transmission system model is made of five links：Acquisition, transmission, decoding, plays coding.Since the factor of reality to regard
The realization of frequency live broadcast system is not so simple, is related to many professional techniques.
Delay variation (jitter) is as an important qos parameter in video living transmission system, to voice communication quality
There is direct influence.Therefore it needs to prevent delay variation using certain measure, so as to ensure certain live streaming quality.
Delay variation (jitter) refers to that the variation due to various delays leads to the packet arrival rate in network
Variation.Network delay refers to an IP packet in the time needed for transmission over networks, it includes algorithm time delay, coding delay, network
Propagation delay time and jitter buffer time delay.Network delay shake is exactly that network delay is for a moment big for a moment small, in this case, even if
Sender is timing transmission data packet, and the reception of recipient also can not equally periodically, and a packet also connects in the sometimes one period
It can not receive, several packets are received in the sometimes one period.If network jitter is more serious, data packet is it is also possible to abandon.
In this way, the sound that recipient hears is caused to be exactly a card.
In order to eliminate delay variation, generally use jitter buffer technology (JitterBuffer) sets one in recipient
A buffering area, datagram initially enter buffer pool and keep in when reaching, subsequent system again with smooth rate by data packet from buffering
It is taken out in pond, user is played to after decompression.This buffer technology can effectively eliminate voice jitter within certain limits, and carry
High tone quality.
The buffer depth of JitterBuffer depends on the degree of network jitter, and network jitter is bigger, and buffer depth is bigger,
The delay of broadcasting is bigger.So JitterBuffer is that the higher broadcasting postponed to exchange smoothness for is utilized, compared to sound
For one card, slightly bigger delay but more smooth effect, subjective experience will be more preferably.Certain JitterBuffer
Buffer depth be not necessarily it is constant always, can be according to the variation of network jitter degree and dynamic adjust.Work as network
When being restored to very steady unobstructed, buffer depth can be very small, and delay in this way can be reduced.
At present there are mainly two types of buffer control methods：Static jitter buffer control method and self-adapted jitter buffer controlling party
Method.
In static jitter buffer control method, the size of buffering area is always fixed value after transmission channel foundation, for
Shake will be dropped beyond the data of buffer size.The advantages of this method is：Model is simple, is easily achieved, reliability height；
Shortcoming is：When network jitter is larger, packet loss is high, and when network delay and shake very little, delay is larger, and applicability is not
It is very strong.
In self-adapted jitter buffer control method, buffer size changes with the jitter conditions of real network.With static state
The advantages of jitter buffer control method is compared, self-adapted jitter buffer control method be：Packet loss is opposite when network jitter is larger
It is relatively low, and network delay and shake hour, delay are relatively small.But the self-adapted jitter buffer controlling party that the prior art uses
Method is to increase according to the jitter conditions of real network or reduce buffer size, and the value increased or decreased is often one
Definite value.That is, the jitter conditions of the network for needing increase/reduction buffering area, are that current buffer is increased plus/minus
A few definite value, and ignore existing difference between the jitter conditions for the network for needing increase/reduction buffering area, so that dynamic
State adjustment is not smooth enough.Therefore, it is not applied to for the higher video living transmission system of smoothness requirements yet.
Invention content
The purpose of the embodiment of the present invention is to provide a kind of audio/video that delay variation is eliminated based on TCP live broadcast systems and played
Method, apparatus and control method, device, can according to network state dynamic effectively adjustment buffer size, and ensure it is smooth,
Ensure to make time delay small as possible under the premise of playing smoothness.
An embodiment of the present invention provides it is a kind of based on TCP live broadcast systems eliminate delay variation control method, including：
Judge whether current buffer is empty；
When buffering area for sky, when currently adjustment the period reach when, if currently adjustment the period in maximum buffer data
The difference of duration and minimal buffering area data duration is less than preset buffering area duration dithering threshold, and minimal buffering area number
It is more than preset minimal buffering area duration threshold value according to duration, then shortens current buffer data duration, and the value shortened is less than
Or equal to the difference between the minimal buffering area data duration and the preset minimal buffering area duration threshold value；
When buffering area is sky, area to be buffered to be empty to when receiving first data packet, then by initial buffer area data when
Long increase, and the initial buffer area data duration after increase is the increasing function of second largest delay variation value；Wherein, it is described initial slow
It is that buffering area plays required buffered data duration to be empty to recovery to rush area's data duration；The second largest delay variation value is
Buffering area is continuous two received by the preset time period to the time for receiving first data packet before sky
Second Largest Value in the time difference of a data packet.
As the improvement of said program, the initial buffer area data duration after increase is more than preset maximum cushioning
During area's duration threshold value, using the preset maximum buffer duration threshold value as during the initial buffer area data after the increase
It is long；It, will be described default when the initial buffer area data duration after the increase is less than preset minimal buffering area duration threshold value
Minimal buffering area duration threshold value as the initial buffer area data duration after the increase.
As the improvement of said program, when the second largest delay variation value is less than preset minimal time delay jitter value,
Using the preset minimal time delay jitter value as the second largest delay variation value；When the second largest delay variation value is more than
During preset maximum delay jitter value, using the preset maximum delay jitter value as the second largest delay variation value.
As the improvement of said program, further include：The time difference stacking of continuously receive two data packets is preserved.
As the improvement of said program, the initial buffer area data duration after the increase is second largest delay variation value
Linear increasing function.
As the improvement of said program, the initial buffer area data duration after the increase is second largest delay variation value
Piecewise linearity increasing function.
As the improvement of said program, 1/3 ￣ 1/2 of the preset time period for a complete adjustment period.
The embodiment of the present invention is also corresponding to disclose a kind of control device that delay variation is eliminated based on TCP live broadcast systems, packet
It includes：
Judgment module, for judging whether current buffer is empty；
The first adjustment module is not sky for working as buffering area, when currently the adjustment period reaches, if in the currently adjustment period
The difference of maximum buffer data duration and minimal buffering area data duration be less than preset buffering area duration dithering threshold, and
The minimal buffering area data duration is more than preset minimal buffering area duration threshold value, then current buffer data duration contracts
It is short, and the value shortened be less than or equal to the minimal buffering area data duration and the preset minimal buffering area duration threshold value it
Between difference；
Second adjustment module for working as buffering area for sky, then increases initial buffer area data duration, and first after increase
Beginning buffer data duration is the increasing function of second largest delay variation value；Wherein, the initial buffer area data duration is buffering
Area plays required buffered data duration to be empty to recovery；The second largest delay variation value is that buffering area is pre- before sky
If the Second Largest Value in the period in the time difference of received continuous two data packets.
As the improvement of said program, the second adjustment module is additionally operable to：Initial buffer area number after the increase
When being more than preset maximum buffer duration threshold value according to duration, using the preset maximum buffer duration threshold value as the increasing
Initial buffer area data duration after big；Initial buffer area data duration after the increase is less than preset minimal buffering area
During duration threshold value, using the preset minimal buffering area duration threshold value as the initial buffer area data duration after the increase.
As the improvement of said program, the second adjustment module is additionally operable to：When the second largest delay variation value is less than
During preset minimal time delay jitter value, using the preset minimal time delay jitter value as the second largest delay variation value；When
The second largest delay variation value be more than preset maximum delay jitter value when, using the preset maximum delay jitter value as
The second largest delay variation value.
As the improvement of said program, further include：
Stacking preserving module, the time difference stacking of two data packets for that will continuously receive preserve.
As the improvement of said program, the initial buffer area data duration after the increase is second largest delay variation value
Linear increasing function.
As the improvement of said program, the initial buffer area data duration after the increase is second largest delay variation value
Piecewise linearity increasing function.
As the improvement of said program, 1/3 ￣ 1/2 of the preset time period for a complete adjustment period.
The embodiment of the invention also discloses a kind of audio/video live broadcasting method based on TCP, including：
Received data packet simultaneously will be in the data pack buffer to buffering area；
The buffer data duration of the buffering area is adjusted according to control method as described above；
Data packet is obtained in buffering area after being adjusted based on buffer data duration and decodes broadcasting.
The embodiment of the invention also discloses a kind of audio/video live broadcast device based on TCP, including：
Data reception module, and will be in the data pack buffer to buffering area for received data packet；
Control device as described above, for the buffer data duration to adjusting the buffering area；And
Playing module is decoded, data packet is obtained in the buffering area after being adjusted based on buffer data duration and decodes broadcasting.
Compared with prior art, the audio/video live streaming side disclosed by the invention that delay variation is eliminated based on TCP live broadcast systems
Method, device and control method, device can effectively solve the problems, such as time delay and shake in video living transmission system, in adaptive jitter
On buffer control method, by judging whether current buffer is empty, and in the case where buffering area be sky/be sky, according to
Current network state to buffer data duration shorten/increase adjustment respectively, and the value adjusted be according to network state and
The value of variation.Therefore, can according to network state dynamic effectively adjustment buffer size, and ensure it is smooth, ensure to play stream
Make time delay small as possible under the premise of smooth.
Description of the drawings
Fig. 1 is that a kind of flow for the control method that delay variation is eliminated based on TCP live broadcast systems is shown in the embodiment of the present invention
It is intended to.
Fig. 2 is the buffering area change schematic diagram in the case of being sky there is no buffering area in the currently adjustment period.
Fig. 3 is the idiographic flow schematic diagram of step S102 shown in FIG. 1.
Fig. 4 is buffering area change schematic diagram of the current adjustment cycle memory in the case where buffering area is sky.
Fig. 5 is showing for initial buffer area data duration and the second largest linear relationships of increase function of delay variation value after increase
It is intended to.
Fig. 6 is that a kind of structure for the control device that delay variation is eliminated based on TCP live broadcast systems is shown in the embodiment of the present invention
It is intended to.
Fig. 7 is a kind of stream for the video broadcasting method that delay variation is eliminated based on TCP live broadcast systems in the embodiment of the present invention
Journey schematic diagram.
Fig. 8 is a kind of structure diagram of the audio/video live streaming reception device based on TCP in the embodiment of the present invention.
Specific embodiment
Below in conjunction with the attached drawing in the embodiment of the present invention, the technical solution in the embodiment of the present invention is carried out clear, complete
Site preparation describes, it is clear that described embodiment is only part of the embodiment of the present invention, instead of all the embodiments.It is based on
Embodiment in the present invention, those of ordinary skill in the art are obtained every other without creative efforts
Embodiment shall fall within the protection scope of the present invention.
It is a kind of control method that delay variation is eliminated based on TCP live broadcast systems provided in an embodiment of the present invention referring to Fig. 1
Flow diagram.The control method includes step：
S101, judge whether current buffer is empty；
S102, when buffering area for sky, when currently adjustment the period reach when, if currently adjustment the period in maximum buffer
The difference of data duration and minimal buffering area data duration is less than preset buffering area duration dithering threshold, and the minimal buffering
When area's data duration is more than preset minimal buffering area duration threshold value, then current buffer data duration is shortened, and shorten
Value is less than or equal to the difference between the minimal buffering area data duration and the preset minimal buffering area duration threshold value；
S103, when buffering area is sky, area to be buffered to be empty to when receiving first data packet, then by initial buffer area number
Increase according to duration, and the initial buffer area data duration after increase is the increasing function of second largest delay variation value；Wherein, it is described first
Beginning buffer data duration is that buffering area plays required buffered data duration to be empty to recovery；The second largest delay variation
Value is that buffering area is continuous received by the preset time period to the time for receiving first data packet before sky
Two data packets time difference in Second Largest Value.
The present embodiment is used to that, when receiving the data of audio/video live broadcast system to play, delay variation can be effectively eliminated.
Since video existing in video living transmission system has audio again, this implementation is described in detail by taking video living transmission system as an example in the present embodiment
The realization process of example.For video living transmission system, audio pack is smaller than the data volume of video bag, first has to ensure in live broadcast system
Interactive voice quality, therefore using VoP, rate changes to define delay variation value the present embodiment in a network
Jitter and control buffer size.
Three threads are often created in video living transmission system, are respectively：Read thread, audio thread, video thread.It reads
Line taking journey is responsible for receiving data from network, and audio, video data is respectively placed in Voice and Video buffering area.Audio thread is from sound
It fetches according to and decodes in frequency buffering area, then play sound.Video thread is fetched according to from screen buffer and is decoded, Ran Houbo
Put video.Therefore there are two buffering areas in video living transmission system, and for the sake of simplicity, the present embodiment uses unified controlling party
Method controls two buffer sizes.
Since data packet does not have synchronised clock between transmitting terminal and receiving terminal, can not calculate data transmission it is real when
Prolong, therefore the present embodiment weighs network state when designing jitter-buffer size using the time delay of receiving terminal as standard.
The video living transmission system of the present embodiment is based on Transmission Control Protocol, which is a kind of reliable protocol, and there is no networks to lose
Packet.The data packet number fluctuation arrived sequentially in a time cycle in the case of network is stable is smaller.However work as net
Under network shake serious situation, the data packet number that receiving terminal receives can decline rapidly, and buffer data is caused to reduce rapidly,
It is even empty.When network recovery is normal, receiving terminal can receive in a flash a large amount of data packet, therefore the number of buffering area
According to surge or overflow.
Assuming that there are two continuous audio/video packets, serial number is i and i+1 respectively, then agrees as follows：
(1)Ti,Ti+1：The internal time stamp of data packet i and i+1；
(2)Ri,Ri+1：Data packet i and i+1 reach the time of receiving terminal；
(3)Pi,Pi+1：Data packet i and i+1 is in the time locally played；
(4)Jitteri=Ri+1-Ri：Data packet i and i+1 reach the time difference of receiving terminal, are determined by network state.
It shakes, then has to arbitrary i and j if there is no network delay：Jitteri=JitterjAnd Ri+1-Ri=
Ti+1-Ti, in this ideal case, data packet, which does not need to any buffering, directly to be played, it is only necessary to meet Pi+1-Pi=
Jitteri.But in a practical situation, due to there are delay variation, at this time if voice distortion, video will be generated by directly playing
Show as continuous interim card, so played after a period of time must be cached to audio/video packet, can just make voice true in this way and
Video links up.It follows that the target that the present embodiment carries out jitter buffer is exactly：Make reproduction time interval of audio/video packet etc.
Interval is stabbed, and reduce interim card number and delay as far as possible in their internal time.
Buffer size is adjusted using the control method that delay variation is eliminated based on TCP live broadcast systems of the present embodiment
It is whole, it is firstly introduced into several parametric variables：
(1)min_buffer_time：Preset minimal buffering area duration threshold value, i.e. jitter-buffer minimum need to buffer
Time, buffering the time for the first time can play.But the time that buffering reaches minimum every time after not representing can be broadcast
It puts.The parameter determines the minimal time delay of normal smooth playing.
(2)max_buffer_time：During preset maximum buffer duration threshold value, the i.e. maximum cushioning of jitter-buffer
Between, the data packet beyond the time can be dropped.The parameter determines the maximum delay that can allow.
(3)cur_buffer_time：Current buffer data duration.When the variable is data in reaction current buffer
Long, this value is at every moment in variation, it is therefore desirable to the max (maximum cushionings for having got the value and current time
Area's data duration) and min (minimal buffering area data duration) be compared, so as to update max (maximum buffer data duration)
With the value of min (minimal buffering area data duration), treat just obtain final max (maximum buffers after this end cycle
Data duration) and min (minimal buffering area data duration) value.
(3.1)max：Maximum buffer data duration in one adjustment period.Value reflection is an adjustment period
The maximum value of data duration in the buffering area changed at the time of interior.An adjustment period can be obtained by way of above-mentioned (3)
Interior final max values.
(3.2)min：Minimal buffering area data duration in one adjustment period.Value reflection is an adjustment period
The minimum value of data duration in the buffering area changed at the time of interior.An adjustment period can be obtained by way of above-mentioned (3)
Interior final min values.
(4)replay_buffer_time：Initial buffer area data duration refers to that (i.e. buffering area is net cast interim card
It is empty) later required buffered data duration is played to recovery.The value is located at min_buffer_time and max_buffer_
Between time.The initial value of the value takes min_buffer_time.The size of the follow-up value is by jitter values described below
Dynamic determines.
(5)jitter：Delay variation value, i.e., continuous two audio packs reach the time difference of receiving terminal, whenever calculating
One jitter value saves its stacking.Jitter values in a period of time can reflect the dither state of network.
(6)min_jitter：Preset minimal time delay dithering threshold can consider that network jitter is very small less than this value,
Network is smooth.
(7)max_jitter：Preset maximum delay shakes threshold r values, can consider that network jitter is very strict more than the value
Evil.
(8)Th：The threshold value of preset buffering area duration dithering threshold, i.e. a cycle internal buffer duration shake, is less than
The threshold value represents network jitter very little.
After more than parametric variable is defined, then can be through this embodiment based on TCP live broadcast systems eliminate delay variation
Control method buffer size is adjusted.The control method of the present embodiment according to current network state it is good/difference is come to slow
It rushes area size and carries out different adjustment, wherein, tentatively distinguish current network state by judging whether current buffer is empty
It is preferably still poor (above-mentioned steps S101), i.e., in the case that current buffer is not empty, it is believed that network state is relatively good；Currently
In the case that buffering area is empty, it is believed that network state is poor.
Consider the relatively good situation of network state, that is, the value fluctuation very little of delay variation value Jitter, this when
The variation of buffer data duration is as shown in Figure 2：One adjust the period internal buffer fluctuation it is smaller, in Fig. 2 peak and
Minimum point is denoted as max (maximum buffer data duration) and min (minimal buffering area data duration) respectively, delays in this case
It rushes that area is practical to have buffered excessive data, leads to delay not less than min (minimal buffering area data duration).Actually this situation
Under optimal buffer size best_buffer_time=max-min.Therefore forwarding operation can be performed, discards buffering
Oldest batch of data in area, the data duration of discarding are no more than min (minimal buffering area data duration) or min (minimal bufferings
Area's data duration) difference between min_buffer_time (preset minimal buffering area duration threshold value), it will not cause in this way
Buffering area no data and interim card can reduce the delay of live streaming after opposite F.F..
With reference to figure 3, (network state compares in the case of judging current buffer not for sky in above-mentioned steps S102
It is good), adjust buffer data duration especially by following steps：
Step S1021, real-time update currently adjusts the maximum buffer data duration max and minimal buffering area number in the period
According to duration min；Specific update mode is that (frequency acquisition can be pre-set, and collection period is all far fewer than adjustment for real-time acquisition
Phase) current buffer data duration cur_buffer_time, and by current buffer data duration cur_buffer_time with
The maximum buffer data duration max and minimal buffering area data duration min got is compared, so as to real-time update
The value of maximum buffer data duration max and minimal buffering area data duration min；
Step S1022, judge whether current (complete) the adjustment period reaches, if so, performing step S1023；Otherwise
It returns and performs step S1021；
Step S1023, it (can obtain the current maximum for adjusting the period at this time to delay when reaching in current (complete) the adjustment period
Rush the value of area data duration max and minimal buffering area data duration min), calculate relevant value by the following formula (1), (2)：
d1=max-min； (1)
d2=min-min_buffer_time (2)
Wherein, max is the maximum buffer data duration in the current adjustment period, and min is adjusted in the period most to be current
Big buffer data duration, d1During currently to adjust the maximum buffer data duration max in the period with minimal buffering area data
The difference of long min；Min_buffer_time be preset minimal buffering area duration threshold value, d2For minimal buffering area data duration
Difference between min and preset minimal buffering area duration threshold value min_buffer_time.
Step S1024, the value d that judgment step S1023 is calculated1And d2Whether the following formula (3) is met, if so, holding
Row step S1025；Otherwise buffer data duration is not adjusted.
d1<Th&d2>0 (3)
Wherein, Th is preset buffering area duration dithering threshold.
Step S1025, by the current buffer data duration cur_ of current collected (adjusting period arrival moment)
Buffer_time shortens (being equivalent to F.F.) a period of time (data duration), and the data duration shortened is less equal than d2。
In formula (3), when calculating maximum buffer data duration max and minimal buffering area in the current adjustment period
The difference of data duration min represents network jitter very little when the difference is less than preset buffering area duration dithering threshold Th, because
This, is if minimal buffering area data duration min is bigger than preset minimal buffering area duration threshold value min_buffer_time this moment,
Then current buffer data duration can be shortened and (be equivalent to and perform forwarding operation), time delay can be reduced in this way.In addition, shorten
Buffer data duration (or data duration of F.F.) less than or equal to the minimal buffering area data duration min and described pre-
If minimal buffering area duration threshold value min_buffer_time between difference, buffering area no data will not be caused in this way and blocked
.
When network state is poor, the value fluctuation of delay variation value Jitter is bigger, and receiving terminal receives number
According to drastically declining, but end is played still according to the access of fixed frequency according to playing out, therefore the data of buffering area can be drastically
Decline up to being zero, cause interim card is broadcast live.When network recovery is normal, connect suddenly in the receiving terminal at that moment of recovery
A large amount of data are received, then network state is restored to situation shown in Fig. 2.As shown in figure 4, peak is denoted as max (most in Fig. 4
Big buffer data duration), buffering area is denoted as gap for sky to the time interval for having data again.In this case it is ideal slow
Size is rushed as best_buffer_time=max+gap.Lead to interim card since delay variation value Jitter fluctuations are larger,
It needs to increase initial buffer area data duration (playing required buffered data duration to recovery after net cast interim card)
Size, that is, the size of replay_buffer_time.Video living transmission system with user speech interaction due to lacking, to voice
Delay requirement unlike other simple speech communication systems it is so high, allow have higher time delay (such as 1 second).
In above-mentioned steps S103 in the case of judging current buffer for sky (network state is poor), then first
It waits for until buffering area to be empty to when receiving first data packet, is immediately increased initial buffer area data duration by calculating
Greatly, and increase after initial buffer area data duration (i.e. replay_buffer_time) be second largest delay variation value increasing
Function.
Specifically, in receiving terminal received data packet, it can be by the time difference (time delay of continuously receive two data packets
Jitter value Jitter) stacking preservation.Wherein, delay variation value Jitter is calculated in digital independent thread and is obtained, and is preserved
In a stack, therefore the delay variation value Jitter for the always nearest a period of time stored in the stack, it can reflect nearest one section
The network state of time.In the present embodiment, when judge current buffer to be empty to when receiving first data packet, Ji Jianghuan
It is described to receiving for the preset time period (1/3~1/2 time in preferably one complete adjustment period) before sky to rush area
The Second Largest Value preserved in stack in the time of first data packet is selected as second largest delay variation value Jitter in the stack
Adjustment benchmark of the second largest delay variation value Jitter as the initial buffer area data duration after increase, that is, it is initial after increasing
The increasing function of a length of second largest delay variation value Jitter during buffer data.
As shown in Figure 5, it is preferred that the initial buffer area data duration (replay_ after the present embodiment adjustment (increase)
Buffer_time) be second largest delay variation value Jitter linear increasing function.In Fig. 4, sec_jitter represents second
Long time delay jitter value jitter, if the value is more than max_jitter (desired maximum jitter values), value max_
jitter；If the value is less than min_jitter (desired minimum jitter values), value min_jitter.Fig. 4 is shown
When linear coefficient r is respectively：Initial buffer area data duration after increasing in the case of r ＜ 1,1 three kinds of r=1 and r ＞
(replay_buffer_time) with the linear relationship of second largest delay variation value sec_jitter.It is in addition, initial after increase
Buffer data duration (replay_buffer_time) value is between min_buffer_time and max_buffer_time.
I.e. when initial buffer area data duration (replay_buffer_time) after the increase is more than preset maximum buffer
During long threshold value (max_buffer_time), using the preset maximum buffer duration threshold value (max_buffer_time) as
Initial buffer area data duration after the increase；Initial buffer area data duration (the replay_ after increase
When buffer_time) less than preset minimal buffering area duration threshold value (min_buffer_time), by the preset minimum
Buffering area duration threshold value (min_buffer_time) is as the initial buffer area data duration after the increase.
It should be understood that increase after initial buffer area data duration (i.e. replay_buffer_time) and it is second largest when
The relationships of increase function for prolonging jitter value sec_jitter is not unique, and can also using piecewise linear function etc., other increase letter
Number.
As it can be seen that the control method that delay variation is eliminated based on TCP live broadcast systems of the present embodiment, in self-adapted jitter buffer
In control method, by judging whether current buffer is empty, and in the case of be/not being sky in current buffer, according to ought
Preceding network state carries out buffer data duration to shorten/increase adjustment, and the value adjusted is becomes according to network state respectively
The value of change.Therefore, can according to network state dynamic effectively adjustment buffer size, and ensure it is smooth, ensure to play it is smooth
Under the premise of make time delay small as possible.
It is a kind of knot for the control device that delay variation is eliminated based on TCP live broadcast systems in the embodiment of the present invention with reference to figure 6
Structure schematic diagram.The control device includes：
Judgment module 21, for judging whether current buffer is empty；
The first adjustment module 22 is not sky for working as buffering area, when currently the adjustment period reaches, if the currently adjustment period
The difference of interior maximum buffer data duration and minimal buffering area data duration is less than preset buffering area duration dithering threshold,
And the minimal buffering area data duration is more than preset minimal buffering area duration threshold value, then current buffer data duration contracts
It is short, and the value shortened be less than or equal to the minimal buffering area data duration and the preset minimal buffering area duration threshold value it
Between difference；And
Second adjustment module 23, for working as buffering area for sky, area to be buffered to be empty to when receiving first data packet, then
Initial buffer area data duration is increased, and the initial buffer area data duration after increase is the increasing letter of second largest delay variation value
Number is (including linear increasing function, piecewise linearity increasing function etc.)；Wherein, the initial buffer area data duration is that buffering area is arrived for sky
Restore to play required buffered data duration；The second largest delay variation value is the preset time period before buffering area is sky
It is connect in (1/3~1/2 time in preferably one complete adjustment period) to the time for receiving first data packet
Second Largest Value in the time difference of continuous two data packets received.
Preferably, the second adjustment module 23 of the present embodiment is additionally operable to：When the initial buffer area data after the increase
It is long when being more than preset maximum buffer duration threshold value, using the preset maximum buffer duration threshold value as the increase after
Initial buffer area data duration；Initial buffer area data duration after the increase is less than preset minimal buffering area duration
During threshold value, using the preset minimal buffering area duration threshold value as the initial buffer area data duration after the increase.
Preferably, the second adjustment module 23 of the present embodiment is additionally operable to：It is preset when the second largest delay variation value is less than
Minimal time delay jitter value when, using the preset minimal time delay jitter value as the second largest delay variation value；When described
When second largest delay variation value is more than preset maximum delay jitter value, using the preset maximum delay jitter value as described in
Second largest delay variation value.
Preferably, the control device of the present embodiment further includes：Stacking preserving module (not shown), for that will continuously receive
Two data packets time difference stacking preserve.
The course of work of the control device for eliminating delay variation based on TCP live broadcast systems of the present embodiment can refer to above-mentioned reality
Example is applied, details are not described herein.
It is a kind of flow diagram of the audio/video playback method based on TCP in the embodiment of the present invention with reference to figure 7.It should
Audio/video playback method includes step：
S301, received data packet and will be in the data pack buffer to buffering area；
S302, the buffering area is adjusted using the control method shown in FIG. 1 that delay variation is eliminated based on TCP live broadcast systems
Buffer data duration；
S303, adjusted based on buffer data duration after buffering area in obtain data packet and decode broadcasting.
Wherein, the specific implementation of step S301 and step S303 can be used in existing audio/video playback method
Corresponded manner is realized, and the specific implementation of step 302 can refer to the corresponding embodiments of Fig. 1, and details are not described herein.
It is a kind of structure diagram of the audio/video live broadcast device based on TCP in the embodiment of the present invention with reference to figure 8.It should
Audio/video live broadcast device includes：
Data reception module 41, and will be in the data pack buffer to buffering area for received data packet；
Control device 42, for the buffer data duration to adjusting the buffering area；And
Playing module 43 is decoded, data packet is obtained in the buffering area after being adjusted based on buffer data duration and is decoded and is broadcast
It puts.
Wherein, existing video playing can be used in the specific implementation of data reception module 41 and decoding playing module 43
Corresponded manner in device realizes, and the specific implementation of control device 42 can be used it is shown in fig. 5 based on TCP live broadcast systems
The control device of delay variation is eliminated to realize, details are not described herein.
In conclusion it is disclosed by the invention based on TCP live broadcast systems eliminate the video broadcasting method of delay variation, device and
Control method, device can effectively solve the problems, such as time delay and shake in video living transmission system, in self-adapted jitter buffer controlling party
In method, by judging whether current buffer is empty, and in the case where buffering area be sky/be sky, respectively to number of buffer
Shorten/increase adjustment according to duration, and the value adjusted is the value that changes according to network state.It therefore, can be according to network-like
State dynamically adjusts buffer size, and ensure it is smooth, ensure to play it is smooth under the premise of make time delay small as possible.
The above is the preferred embodiment of the present invention, it is noted that for those skilled in the art
For, various improvements and modifications may be made without departing from the principle of the present invention, these improvements and modifications are also considered as
Protection scope of the present invention.
Claims (16)
1. a kind of control method that delay variation is eliminated based on TCP live broadcast systems, which is characterized in that including：
Judge whether current buffering area is empty；
When buffering area for sky, when currently adjustment the period reach when, if currently adjustment the period in maximum buffer data duration
It is less than preset buffering area duration dithering threshold, and during the minimal buffering area data with the difference of minimal buffering area data duration
It is long to be more than preset minimal buffering area duration threshold value, then current buffer data duration is shortened, and the value shortened is less than or waits
Difference between the minimal buffering area data duration and the preset minimal buffering area duration threshold value；
When buffering area is sky, area to be buffered to when receiving first data packet, then increases initial buffer area data duration to be empty
Greatly, and increase after initial buffer area data duration be second largest delay variation value increasing function；Wherein, the initial buffer area
Data duration is that buffering area plays required buffered data duration to be empty to recovery；The second largest delay variation value is buffering
Area is continuous two number received in the preset time period to the time for receiving first data packet before sky
According to the Second Largest Value in the time difference of packet.
2. the control method of delay variation is eliminated based on TCP live broadcast systems as described in claim 1, which is characterized in that work as institute
It, will be described preset maximum slow when stating the initial buffer area data duration after increase more than preset maximum buffer duration threshold value
Area's duration threshold value is rushed as the initial buffer area data duration after the increase；When the initial buffer area data after the increase
It is long when being less than preset minimal buffering area duration threshold value, using the preset minimal buffering area duration threshold value as the increase after
Initial buffer area data duration.
3. the control method of delay variation is eliminated based on TCP live broadcast systems as described in claim 1, which is characterized in that work as institute
When stating second largest delay variation value less than preset minimal time delay jitter value, using the preset minimal time delay jitter value as institute
State second largest delay variation value；It, will be described pre- when the second largest delay variation value is more than preset maximum delay jitter value
If maximum delay jitter value as the second largest delay variation value.
4. the control method of delay variation is eliminated based on TCP live broadcast systems as described in claim 1, which is characterized in that also wrap
It includes：
The time difference stacking of continuously receive two data packets is preserved.
5. the control method of delay variation is eliminated based on TCP live broadcast systems as described in claim 1, which is characterized in that described
Initial buffer area data duration after increase is the linear increasing function of second largest delay variation value.
6. the control method of delay variation is eliminated based on TCP live broadcast systems as described in claim 1, which is characterized in that described
Initial buffer area data duration after increase is the piecewise linearity increasing function of second largest delay variation value.
7. the control method of delay variation is eliminated based on TCP live broadcast systems as described in claim 1, which is characterized in that described
Preset time period completely adjusts the 1/3~1/2 of the period for one.
8. a kind of control device that delay variation is eliminated based on TCP live broadcast systems, which is characterized in that including：
Judgment module, for judging whether current buffer is empty；
The first adjustment module is not sky for working as buffering area, when currently the adjustment period reaches, if in the currently adjustment period most
The difference of big buffer data duration and minimal buffering area data duration is less than preset buffering area duration dithering threshold, and described
Minimal buffering area data duration is more than preset minimal buffering area duration threshold value, then shortens current buffer data duration, and
The value of shortening is less than or equal between the minimal buffering area data duration and the preset minimal buffering area duration threshold value
Difference；
Second adjustment module, for working as buffering area for sky, area to be buffered is arrived to be empty when receiving first data packet, then will be initially
Buffer data duration increases, and the initial buffer area data duration after increase is the increasing function of second largest delay variation value；Its
In, the initial buffer area data duration is that buffering area plays required buffered data duration to be empty to recovery；Described second
Long time delay jitter value is connect in the preset time period to the time for receiving first data packet before buffering area is sky
Second Largest Value in the time difference of continuous two data packets received.
9. the control device of delay variation is eliminated based on TCP live broadcast systems as claimed in claim 8, which is characterized in that described
Second adjustment module is additionally operable to：Initial buffer area data duration after the increase is more than preset maximum buffer duration threshold
During value, using the preset maximum buffer duration threshold value as the initial buffer area data duration after the increase；When described
When initial buffer area data duration after increase is less than preset minimal buffering area duration threshold value, by the preset minimal buffering
Area's duration threshold value is as the initial buffer area data duration after the increase.
10. the control device of delay variation is eliminated based on TCP live broadcast systems as claimed in claim 8, which is characterized in that described
Second adjustment module is additionally operable to：It, will be described pre- when the second largest delay variation value is less than preset minimal time delay jitter value
If minimal time delay jitter value as the second largest delay variation value；When the second largest delay variation value be more than it is preset most
During long time delay jitter value, using the preset maximum delay jitter value as the second largest delay variation value.
11. the control device of delay variation is eliminated based on TCP live broadcast systems as claimed in claim 8, which is characterized in that also wrap
It includes：
Stacking preserving module, the time difference stacking of two data packets for that will continuously receive preserve.
12. the control device of delay variation is eliminated based on TCP live broadcast systems as claimed in claim 8, which is characterized in that described
Initial buffer area data duration after increase is the linear increasing function of second largest delay variation value.
13. the control device of delay variation is eliminated based on TCP live broadcast systems as claimed in claim 8, which is characterized in that described
Initial buffer area data duration after increase is the piecewise linearity increasing function of second largest delay variation value.
14. the control device of delay variation is eliminated based on TCP live broadcast systems as claimed in claim 8, which is characterized in that described
Preset time period completely adjusts the 1/3~1/2 of the period for one.
15. a kind of audio/video live broadcasting method based on TCP, which is characterized in that including：
Received data packet simultaneously will be in the data pack buffer to buffering area；
The buffer data duration of the buffering area is adjusted according to claim 1~7 any one of them control method；
Data packet is obtained in buffering area after being adjusted based on buffer data duration and decodes broadcasting.
16. a kind of audio/video live broadcast device based on TCP, which is characterized in that including：
Data reception module, and will be in the data pack buffer to buffering area for received data packet；
Such as claim 8~14 any one of them control device, for the buffer data duration to adjusting the buffering area；
And
Playing module is decoded, data packet is obtained in the buffering area after being adjusted based on buffer data duration and decodes broadcasting.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510483445.4A CN105142002B (en) | 2015-08-07 | 2015-08-07 | Audio/video live broadcasting method, device and control method, device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510483445.4A CN105142002B (en) | 2015-08-07 | 2015-08-07 | Audio/video live broadcasting method, device and control method, device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN105142002A CN105142002A (en) | 2015-12-09 |
| CN105142002B true CN105142002B (en) | 2018-06-22 |
Family
ID=54727187
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201510483445.4A Active CN105142002B (en) | 2015-08-07 | 2015-08-07 | Audio/video live broadcasting method, device and control method, device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN105142002B (en) |
Families Citing this family (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106658049B (en) * | 2016-12-31 | 2019-08-30 | 深圳市优必选科技有限公司 | A kind of video playing way to play for time and system |
| CN106713941A (en) * | 2017-01-04 | 2017-05-24 | 北京百度网讯科技有限公司 | Audio and video live broadcast implementation method and server |
| CN108650521A (en) * | 2018-05-17 | 2018-10-12 | 上海七牛信息技术有限公司 | A kind of live broadcasting method, device and multimedia play system |
| CN109640160B (en) * | 2018-12-28 | 2022-01-11 | 鸿视线科技(北京)有限公司 | Sustainable synchronous control system and method based on buffer control structure |
| CN110266611B (en) * | 2019-07-08 | 2023-06-23 | 腾讯科技（上海）有限公司 | Method, device and system for processing buffered data |
| CN110634511B (en) * | 2019-09-27 | 2021-09-14 | 北京西山居互动娱乐科技有限公司 | Audio data processing method and device |
| CN112822502B (en) * | 2020-12-28 | 2022-06-07 | 阿里巴巴（中国）有限公司 | Live broadcast jitter removal intelligent caching and live broadcast method, equipment and storage medium |
| CN113727186B (en) * | 2021-08-26 | 2024-05-17 | 百果园技术(新加坡)有限公司 | Video playing method and system for real-time video call |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101123571A (en) * | 2006-08-07 | 2008-02-13 | 北京三星通信技术研究有限公司 | Adjustment method for scheduling policy based on self-adapted jitter buffer |
| US7359324B1 (en) * | 2004-03-09 | 2008-04-15 | Nortel Networks Limited | Adaptive jitter buffer control |
| US7916742B1 (en) * | 2005-05-11 | 2011-03-29 | Sprint Communications Company L.P. | Dynamic jitter buffer calibration |
| CN102739503A (en) * | 2011-04-01 | 2012-10-17 | 华平信息技术股份有限公司 | Processing method of VoIP (voice over internet protocol) de-jitter buffer |
| CN103533451A (en) * | 2013-09-30 | 2014-01-22 | 广州华多网络科技有限公司 | Method and system for regulating jitter buffer |
-
2015
- 2015-08-07 CN CN201510483445.4A patent/CN105142002B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7359324B1 (en) * | 2004-03-09 | 2008-04-15 | Nortel Networks Limited | Adaptive jitter buffer control |
| US7916742B1 (en) * | 2005-05-11 | 2011-03-29 | Sprint Communications Company L.P. | Dynamic jitter buffer calibration |
| CN101123571A (en) * | 2006-08-07 | 2008-02-13 | 北京三星通信技术研究有限公司 | Adjustment method for scheduling policy based on self-adapted jitter buffer |
| CN102739503A (en) * | 2011-04-01 | 2012-10-17 | 华平信息技术股份有限公司 | Processing method of VoIP (voice over internet protocol) de-jitter buffer |
| CN103533451A (en) * | 2013-09-30 | 2014-01-22 | 广州华多网络科技有限公司 | Method and system for regulating jitter buffer |
Also Published As
| Publication number | Publication date |
|---|---|
| CN105142002A (en) | 2015-12-09 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN105142002B (en) | Audio/video live broadcasting method, device and control method, device | |
| US7830862B2 (en) | System and method for modifying speech playout to compensate for transmission delay jitter in a voice over internet protocol (VoIP) network | |
| US7359324B1 (en) | Adaptive jitter buffer control | |
| US9143810B2 (en) | Method for manually optimizing jitter, delay and synch levels in audio-video transmission | |
| EP3466001B1 (en) | Media buffering | |
| TW200849842A (en) | Method of regulating the transmission jitter within a reception terminal | |
| CN113242436B (en) | Live broadcast data processing method and device and electronic equipment | |
| CN108881931A (en) | A kind of data buffering method and the network equipment | |
| US20080101398A1 (en) | Transmission scheme dependent control of a frame buffer | |
| CN108476176A (en) | Real-time transport protocol Research of Congestion Control Techniques in visual telephone | |
| US6567428B1 (en) | Method and system for using low data rate vocorder packets to multiplex signaling packets into a CDMA traffic channel for high quality CDMA voice transmission | |
| CN104168218B (en) | Jitter buffer method and device | |
| WO2007080788A1 (en) | Teleconference control device and teleconference control method | |
| CN107979482B (en) | Information processing method, device, sending end, jitter removal end and receiving end | |
| EP2070294B1 (en) | Supporting a decoding of frames | |
| CN106331847B (en) | Audio and video playing method and apparatus | |
| CN112423074A (en) | Audio and video synchronization processing method and device, electronic equipment and storage medium | |
| EP2351371A1 (en) | Method for buffering streaming data and a terminal device | |
| JP4561301B2 (en) | Audio reproduction device and program for controlling reproduction and stop of audio | |
| CN110875860B (en) | Method and device for processing network jitter | |
| WO2016104178A1 (en) | Signal processing device, signal processing method, and program | |
| US6567392B1 (en) | Method and system for using vocorder rate determination for high quality CDMA voice transmission | |
| JP4050961B2 (en) | Packet-type voice communication terminal | |
| US8791981B2 (en) | Bit rate control apparatus and method thereof | |
| US20130142192A1 (en) | Voice communication apparatus for intermittently discarding packets |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |