CN107231328A - Method for real-time video transmission, device, equipment and system - Google Patents
Method for real-time video transmission, device, equipment and system Download PDFInfo
- Publication number
- CN107231328A CN107231328A CN201610167888.7A CN201610167888A CN107231328A CN 107231328 A CN107231328 A CN 107231328A CN 201610167888 A CN201610167888 A CN 201610167888A CN 107231328 A CN107231328 A CN 107231328A
- Authority
- CN
- China
- Prior art keywords
- key frame
- frame
- video
- rtp
- transmission
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 230000005540 biological transmission Effects 0.000 title claims abstract description 138
- 238000000034 method Methods 0.000 title claims abstract description 31
- 238000012545 processing Methods 0.000 claims abstract description 63
- 230000008859 change Effects 0.000 claims abstract description 11
- 241001269238 Data Species 0.000 claims abstract description 10
- 230000003139 buffering effect Effects 0.000 claims abstract description 6
- 239000000872 buffer Substances 0.000 claims description 95
- 238000009432 framing Methods 0.000 claims description 35
- 230000007246 mechanism Effects 0.000 claims description 8
- 238000005516 engineering process Methods 0.000 abstract description 4
- 230000003111 delayed effect Effects 0.000 abstract 1
- 210000004209 hair Anatomy 0.000 description 8
- 238000011084 recovery Methods 0.000 description 7
- 230000006835 compression Effects 0.000 description 6
- 238000007906 compression Methods 0.000 description 6
- 238000010586 diagram Methods 0.000 description 4
- 230000006870 function Effects 0.000 description 4
- 230000008569 process Effects 0.000 description 4
- 230000009286 beneficial effect Effects 0.000 description 3
- 101000741965 Homo sapiens Inactive tyrosine-protein kinase PRAG1 Proteins 0.000 description 2
- 102100038659 Inactive tyrosine-protein kinase PRAG1 Human genes 0.000 description 2
- 230000001413 cellular effect Effects 0.000 description 2
- 238000010276 construction Methods 0.000 description 2
- 238000012937 correction Methods 0.000 description 2
- 230000003068 static effect Effects 0.000 description 2
- 230000002159 abnormal effect Effects 0.000 description 1
- 238000012790 confirmation Methods 0.000 description 1
- 230000007423 decrease Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 230000002787 reinforcement Effects 0.000 description 1
- 230000035807 sensation Effects 0.000 description 1
- 238000012163 sequencing technique Methods 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
- H04L65/60—Network streaming of media packets
- H04L65/65—Network streaming protocols, e.g. real-time transport protocol [RTP] or real-time control protocol [RTCP]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
- H04L65/60—Network streaming of media packets
- H04L65/61—Network streaming of media packets for supporting one-way streaming services, e.g. Internet radio
- H04L65/612—Network streaming of media packets for supporting one-way streaming services, e.g. Internet radio for unicast
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/234—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
- H04N21/23406—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving management of server-side video buffer
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
- H04N21/44004—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving video buffer management, e.g. video decoder buffer or video display buffer
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/60—Network structure or processes for video distribution between server and client or between remote clients; Control signalling between clients, server and network components; Transmission of management data between server and client, e.g. sending from server to client commands for recording incoming content stream; Communication details between server and client
- H04N21/63—Control signaling related to video distribution between client, server and network components; Network processes for video distribution between server and clients or between remote clients, e.g. transmitting basic layer and enhancement layers over different transmission paths, setting up a peer-to-peer communication via Internet between remote STB's; Communication protocols; Addressing
- H04N21/637—Control signals issued by the client directed to the server or network components
- H04N21/6375—Control signals issued by the client directed to the server or network components for requesting retransmission, e.g. of data packets lost or corrupted during transmission from server
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/60—Network structure or processes for video distribution between server and client or between remote clients; Control signalling between clients, server and network components; Transmission of management data between server and client, e.g. sending from server to client commands for recording incoming content stream; Communication details between server and client
- H04N21/63—Control signaling related to video distribution between client, server and network components; Network processes for video distribution between server and clients or between remote clients, e.g. transmitting basic layer and enhancement layers over different transmission paths, setting up a peer-to-peer communication via Internet between remote STB's; Communication protocols; Addressing
- H04N21/643—Communication protocols
- H04N21/6437—Real-time Transport Protocol [RTP]
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Computer Networks & Wireless Communication (AREA)
- Compression Or Coding Systems Of Tv Signals (AREA)
- Detection And Prevention Of Errors In Transmission (AREA)
Abstract
The invention belongs to stream medium data network transmission technology field, and in particular to a kind of method for real-time video transmission, device, equipment and system.The method for real-time video transmission of the present invention includes：Video requency frame data is encoded to key frame or non-key frame by transmitting terminal, sends RTP packets to receiving terminal；Receiving terminal is handled RTP packets, if handling successfully, frame of display video, if processing failure, is sent and is rearranged key frame or retransmit the RTP request of data lacked in key frame to transmitting terminal；After transmitting terminal is received, sent after the transmission opportunity rearranged key frame and change key frame to receiving terminal, or the RTP data lacked in key frame are retransmitted to receiving terminal；1-3 repeat the above steps until all video requency frame datas are transmitted.This method employs the RTP data for retransmitting and being lacked in key frame in data handling, and the mode for abandoning imperfect non-key frame data ensures video quality, while using double buffering, improving data-handling efficiency, reduces the display delayed of real time video data.
Description
Technical field
The invention belongs to stream medium data network transmission technology field, and in particular to a kind of method for real-time video transmission, device, set
Standby and system.
Background technology
With the continuous development of network technology, people take the need of the network traffic of large bandwidth for real-time video traffic etc.
Ask continuous reinforcement.The way of transmission live video stream common at present is by the way that TS (Transport Stream, transport stream) is held
It is loaded on RTP (Real-time Transport protocol, RTP), and RTP bottoms use UDP (User
Datagram Protocol, UDP) it is transmitted, the wireless network such as current substantial amounts of use wifi and 3G, 4G
Network communicates to connect internet, because UDP is unreliable host-host protocol in itself, and wireless network relative instability, can not
What is avoided causes part RTP loss of datas.
In order to solve the problems, such as RTP packet losses, IETF (The Internet Engineering Task Force, Internet work
Journey task groups) having worked out FEC (forward error correction is also in forward error correction) and NACK respectively, (negotiation confirms, plays packet loss re-transmission
Effect) two kinds of algorithms solve this problem.But, the redundancy of data can be increased using FEC algorithms, Streaming Media number is increased
According to occupied bandwidth.NACK algorithms introduce confirmation and retransmitted, and real-time can be caused poor.
The video compression encoding algorithm commonly used according to prior art, video pictures are encoded as intra-coded frame, and (I frames, are also known as closed
Key frame), multiple forward predicted frames (P frames) and two-way interpolation frame (B frames) composition image sequence.Key frame is typically each
GOP (Group of Pictures, picture group) first frame,, can as the reference point of random access by moderately compressing
With as image.P frames are predicted by the P frames before it and I frames, during video data transmission, because data are lost
Lose, video can be caused can not to show or show fuzzy, interim card, if I frame data are imperfect, decoding can be caused to fail, it is impossible to
Video pictures are shown, if P LOFs influence whether that the decoding of P frames below is shown, picture mosaic, P LOFs is occurred
It is more, mosaic is more serious.During being transmitted, abandoning B frame data bag and being shown for the decoding of whole video influences
Very little.
The Chinese patent of Application No. 201410577593.8, discloses a kind of video transmission method, is passed by obtaining current network
Transmission quality information, searches corresponding video coding parameter, removes picture frame information according to video coding parameter, then adjust video
Code check and frame per second, to being transmitted after Video coding.This method have adjusted coding parameter when network quality declines, and reduce
Video quality, causes the unintelligible of video pictures.In addition, under the frequent situation of change of network quality, this method transmitting terminal can frequency
Change video coding parameter numerously, carry out corresponding coded treatment, cause the increase of the low for the treatment of effeciency and processing delay, shadow
Ring the Consumer's Experience of receiving terminal.
The content of the invention
An object of the present invention is to overcome disadvantage mentioned above, realizes in the environment of declining occurs in network quality, reduce as far as possible
The use of the network bandwidth, while ensureing the transmission quality of video.
In order to solve the above-mentioned technical problem, the invention provides a kind of method for real-time video transmission, comprise the following steps：
Step 1：Video requency frame data is encoded to key frame or non-key frame by transmitting terminal, sends RTP packets to receiving terminal；Institute
The transmission opportunity for stating key frame is determined by the spacing parameter configured；
Step 2：Receiving terminal is handled RTP packets, if handling successfully, frame of display video, if processing failure,
Send and rearrange key frame or retransmit the RTP request of data lacked in key frame to transmitting terminal；
Step 3：After transmitting terminal is received, sent after the transmission opportunity rearranged key frame and change key frame to receiving terminal, or to connecing
Receiving end retransmits the RTP data lacked in key frame；
1-3 repeat the above steps until all video requency frame datas are transmitted.
The present invention proposes dynamic key frame interval, by setting larger key frame interval in the case of network quality is normal,
So as to reduce the quantity of key frame, the total amount of data of transmission is reduced；In the case where network quality decline causes data processing failure,
The RTP data packet requests rearranged and lacked in key frame or re-transmission key frame are sent by receiving terminal.For rearranging key frame request, hair
Sending end carries out key frame coding according to request dynamic and changes key frame transmission opportunity, when reducing the wait of receiving terminal video recovery
Between, it is ensured that the timely recovery of video pictures.
Further, the step 1, in addition to：
The RTP packets sent are put into transmission data buffer zone by transmitting terminal, and the RTP that lacks in key frame is retransmitted when receiving
During request of data, the transmission data buffer zone is preferentially searched.
Data buffer zone is sent by setting, the place of searching data when transmitting terminal retransmits the RTP data lacked in key frame is improved
Efficiency is managed, processing time is shortened.
Further, step 2 receiving terminal is handled RTP packets, is specially：
RTP packets are received to reception data buffer；
Whether judge RTP packets is the data lacked in request retransmission key frame；
If not retransmitting the RTP data lacked in key frame, normal data frames processing is carried out；
If retransmitting the RTP data lacked in key frame, carry out key frame and handle again.
Further, the step 2 also includes：Receiving terminal sets pending buffer area, retransmits what key frame had been received for caching
RTP packets.
The receiving terminal of the present invention employs double buffering mechanism, and pending buffer area caching retransmits the RTP packets that key frame has been received,
Reception data buffer can continue to follow-up RTP packets, framing operation be carried out, when incomplete key frame has been received
After whole, key frame and follow-up frame of video can be retransmitted with continuous decoding display, reduced because what wait re-transmission data were caused regards in real time
Frequency delay.
Further, the receiving terminal is from the reception buffering area to read RTP packets according to the jitter time delay of setting.
By adding the problem of jitter time delay can alleviate the RTP data packet disorders due to being caused in network transmission process.
Further, the normal data frames processing, comprises the following steps：
Framing is carried out to the RTP bags in reception data buffer, and judges whether frame of video is complete；
If frame of video is complete, frame of video decoding operate is carried out, it is successfully, otherwise at mark that successfully decoded, which then identifies result,
Result is managed to fail, and transmission is rearranged key frame and asked to transmitting terminal；
Whether if frame of video is imperfect, it is key frame to determine whether frame of video, if the frame of video is key frame, will have been received
RTP packets write-in pending buffer area, mark result is failure, and sends the RTP data for retransmitting and being lacked in key frame
Ask, to transmitting terminal, to notify transmitting terminal to retransmit the RTP data lost in key frame；If the frame of video is non-key frame, abandon
The frame of video, then judge whether the non-key frame quantity abandoned between two continuous key frames reaches threshold value, if having reached
Threshold value, mark result is fails, and transmission rearranges key frame and asks, to transmitting terminal, otherwise to mark result to be successfully.
Above-mentioned technical proposal can carry out different processing in the case of frame of video is incomplete according to the type of frame of video：If
The RTP packets that key frame is then lacked in request retransmission key frame, so ensure that the integrality of video pictures；If right and wrong
The key frame and accumulative frame number that abandons is being set in threshold range, then directly abandon frame, the next frame of video of wait is so handled
Beneficial effect is to reduce the network overhead for rearranging key frame, it is ensured that the fluency of video, and the influence to video pictures only has slightly
Mosaic or less delay；If accumulative non-key frame, which abandons frame number, reaches threshold values, now send and rearrange key frame request, protect
Demonstrate,prove the definition of video pictures.
Further, the progress key frame is handled again, is comprised the following steps：
By the RTP packets write-in pending buffer area of the re-transmission key frame of reception；
Framing is carried out to the RTP bags in pending buffer area, and judges whether the re-transmission key frame in pending buffer area is complete；
If it is complete to retransmit key frame, frame of video decoding operate is carried out, it is successfully, otherwise to mark that successfully decoded, which then identifies result,
Result is known to fail, and transmission is rearranged key frame and asked to transmitting terminal；
If it is imperfect to retransmit key frame, further determines whether to wait time-out, have not timed out, continue waiting for, otherwise if waiting
Reception data buffer and pending buffer area are emptied, mark result is fails, and transmission is rearranged key frame and asked to transmitting terminal.
Above-mentioned technical proposal waits overtime threshold by setting, it is ensured that in the case where network quality declines, receiving terminal will not be unlimited
RTP packets in the key frame of request retransmission to be received such as always of system and cause the interim card of video pictures.
Correspondingly, present invention also offers a kind of real-time video transmission device, including：
Sending module one, for performing step 1：Video requency frame data is encoded to key frame or non-key frame by transmitting terminal, sends RTP
Packet is to receiving terminal；The transmission opportunity of the key frame is determined by the spacing parameter configured；
Receiving module, for performing step 2：Receiving terminal is handled RTP packets, if handling successfully, shows video
Frame, if processing failure, sends and rearranges key frame or retransmit the RTP request of data lacked in key frame to transmitting terminal；
Sending module two, for performing step 3：After transmitting terminal is received, after the transmission opportunity rearranged key frame and change key frame
Sent to receiving terminal, or the RTP data lacked in key frame are retransmitted to receiving terminal；
Loop module, for the 1-3 that repeats the above steps until all video requency frame datas are transmitted.
Further, the receiving module, including：
Receiving unit, for receiving RTP packets to reception data buffer；
Judging unit, for judging whether RTP packets are the data that lack in request retransmission key frame；
First processing units, if not for retransmitting the RTP data lacked in key frame, carry out normal data frames processing；
Second processing unit, if for retransmitting the RTP data lacked in key frame, carry out key frame and handle again.
Further, the receiving module, in addition to：
Pending buffer unit, pending buffer area is set for receiving terminal, and caching retransmits the RTP packets that key frame has been received.
Further, the first processing units, including：
First judges component, for carrying out framing to the RTP bags in reception data buffer, and judges whether frame of video is complete；
First decoding mechanism, if complete for frame of video, carries out frame of video decoding operate, successfully decoded then identifies result
For success, result is otherwise identified to fail, and transmission is rearranged key frame and asked to transmitting terminal；
First processing component, whether if imperfect for frame of video, it is key frame to determine whether frame of video, if the frame of video
For key frame, the RTP packets received are write into pending buffer area, mark result is failure, and it is crucial to send re-transmission
The RTP request of data lacked in frame notifies transmitting terminal to retransmit the RTP data lost in key frame to transmitting terminal；If the frame of video
For non-key frame, then the frame of video is abandoned, then judge whether the non-key frame quantity abandoned between two continuous key frames reaches
To threshold value, if having reached threshold value, mark result is fails, and transmission rearranges key frame and asks, to transmitting terminal, otherwise to mark
Remember that result is successfully.
Further, the second processing unit, including：
Second receiving member, for the RTP packets of the re-transmission key frame of reception to be write into pending buffer area；
Second judges component, for carrying out framing to the RTP bags in pending buffer area, and judges that the re-transmission in pending buffer area is crucial
Whether frame is complete；
Second decoding mechanism, if complete for retransmitting key frame, carries out frame of video decoding operate, then mark is handled successfully decoded
As a result it is successfully, otherwise to identify result to fail, and transmission is rearranged key frame and asked to transmitting terminal；
Second processing component, if imperfect for retransmitting key frame, further determine whether to wait time-out, is had not timed out if waiting,
Then continue waiting for, otherwise empty reception data buffer and pending buffer area, mark result is failure, and pass is rearranged in transmission
Key frame is asked to transmitting terminal.
Correspondingly, present invention also offers a kind of equipment for real-time video transmission, the equipment for real-time video transmission
To send equipment or receiving device,
The transmission equipment is used to video requency frame data being encoded to key frame or non-key frame, sends RTP packets to receiving device；
The transmission opportunity of the key frame is determined by the spacing parameter configured；The transmission equipment is additionally operable to rearrange key frame and changes key
Sent after the transmission opportunity of frame to receiving device, or the RTP data lacked in the re-transmission key frame of request are sent to receiving device；
The receiving device is used to handle RTP packets, if handling successfully, frame of display video, if processing failure,
Then send to rearrange key frame or retransmit the RTP request of data lacked in key frame and give transmission equipment.
Correspondingly, present invention also offers a kind of RTP Transport System for Real-time, including transmitting terminal and receiving terminal, the transmitting terminal is used for
Video requency frame data is encoded to key frame or non-key frame, RTP packets are sent to receiving terminal；The transmission opportunity of the key frame
Determined by the spacing parameter configured；The transmitting terminal is additionally operable to after transmission opportunity for rearranging key frame and changing key frame to receiving terminal
Send, or the RTP data lacked in the re-transmission key frame of request are sent to receiving terminal；
The receiving terminal is used to handle RTP packets, if handling successfully, frame of display video, if processing failure,
Send and rearrange key frame or retransmit the RTP request of data lacked in key frame to transmitting terminal.
Further, the transmitting terminal, which is set, sends data buffer zone and is used to cache the RTP packets sent, when receiving weight
When passing the RTP request of data lacked in key frame, the transmission data buffer zone is preferentially searched.
Further, the receiving terminal sets reception data buffer and pending buffer area, and the reception data buffer is used to delay
The RTP packets received are deposited, the pending buffer area, which is used to cache, retransmits the RTP packets that key frame has been received.
In summary, the beneficial effect of technical solution of the present invention has:
1. the present invention proposes dynamic key frame interval, by setting larger crucial interframe in the case of network quality is normal
Every, so that the quantity of key frame is reduced, the total amount of data of reduction transmission；Declining in network quality causes the feelings of data processing failure
Under condition, the RTP data packet requests rearranged and lacked in key frame or re-transmission key frame are sent by receiving terminal.Please for rearranging key frame
Ask, transmitting terminal carries out key frame coding according to request dynamic and changed key frame transmission opportunity, reduces receiving terminal video recovery
Stand-by period, it is ensured that the timely recovery of video pictures；
2. transmitting terminal, which is set, sends data buffer zone, the processing of searching data when improving the RTP data for retransmitting and being lacked in key frame
Efficiency, shortens processing time.Receiving terminal sets pending buffer area and reception data buffer, can simultaneously and concurrently handle RTP numbers
According to bag, the stand-by period is down at least caused by the RTP data that will be lacked in re-transmission key frame；
3. in the case of frame of video is incomplete, different processing can be carried out according to the type of frame of video：If key frame then please
The RTP data for retransmitting and being lacked in key frame are sought, the integrality of video pictures is so ensure that；If non-key frame and accumulative
Abandon frame number setting in threshold range, then directly abandon frame, wait next frame of video, can so reduce and rearrange key frame
Network overhead, it is ensured that the fluency of video, and the influence to video pictures only has slight mosaic or less delay；If even
Add up non-key frame discarding frame number between continuous two key frames and reach threshold values, now send and rearrange key frame request, it is ensured that video pictures
Definition；
4. receiving terminal is by adding the problem of jitter time delay alleviates the RTP data packet disorders due to being caused in network transmission process, together
When pass through to set and wait overtime threshold, it is ensured that in the case where network quality declines, receiving terminal will not be unconfined always etc. waiting
Receive the key frame RTP packets retransmitted and cause the interim card of video pictures.
Brief description of the drawings
Fig. 1 is a kind of method for real-time video transmission flow chart of steps of the embodiment of the present invention.
Fig. 2 is that a kind of receiving terminal of the embodiment of the present invention carries out process step flow chart to RTP packets.
Fig. 3 is a kind of normal data frames process step flow chart of the embodiment of the present invention.
Fig. 4 is a kind of key frame weight process step flow chart of the embodiment of the present invention.
Fig. 5 is a kind of real-time video transmission apparatus structure frame diagram of the embodiment of the present invention.
Fig. 6 is a kind of receiving module structural framing figure of the embodiment of the present invention.
Fig. 7 is a kind of first processing units structural framing figure of the embodiment of the present invention.
Fig. 8 is a kind of second processing cellular construction frame diagram of the embodiment of the present invention.
Fig. 9 is a kind of transmission equipment structure chart for real-time video transmission of the embodiment of the present invention.
Figure 10 is another receiving device structure chart for real-time video transmission of the embodiment of the present invention.
Figure 11 is a kind of RTP Transport System for Real-time structural framing figure of the embodiment of the present invention.
Embodiment
Below in conjunction with the accompanying drawing in the embodiment of the present invention, the technical scheme in the embodiment of the present invention is clearly and completely retouched
State, it is clear that described embodiment is only a part of embodiment of the invention, rather than whole embodiments.Based on the present invention
In embodiment, the every other embodiment that those of ordinary skill in the art are obtained under the premise of creative work is not made,
Belong to the scope of protection of the invention.
Such as Fig. 1, it is a kind of method for real-time video transmission flow chart of steps of the embodiment of the present invention, comprises the following steps：
Step 1：Video requency frame data is encoded to key frame or non-key frame by transmitting terminal, sends RTP packets to receiving terminal；Institute
The transmission opportunity for stating key frame is determined by the spacing parameter configured；
It is well known that video be exactly the principle that is persisted using human vision by playing a series of picture, human eye is produced motion
Sensation, the static picture of each of which, we term it frame of video.Video file is in network transmission and storage for convenience,
Need to be compressed video coding, reduce file size.
Motion Picture Experts Group's compression standard second edition (MPEG-2, Moving Picture Experts Group common at present
Compression Standard Version 2), H.264/AVC (MPEG-4 advanced video codings, Advanced Video
Coding), Video Codec -1 (Video Codec 1, VC-1), digital audio/video encoding and decoding standard (AVS, Audio Coding
) etc. Standard in video and audio compress encryption algorithm, video pictures be encoded as by intra-coded frame (I frames, also known as key frame),
Multiple forward predicted frames (P frames) and the image sequence of two-way interpolation frame (B frames) composition.I frames are typically each GOP (Group of
Pictures, picture group) first frame, by moderately compressing, as the reference point of random access, it can be understood as this
The complete reservation of frame picture, this frame data is only needed to during decoding can just complete；P frames represent this frame with an I frame before
The difference of (or P frames), needs to be superimposed with the difference of this frame definition with the picture cached before during decoding, generates final picture,
P LOFs above influence whether that the decoding of P frames below is shown, picture mosaic is occurred；B frame recordings are this frames with before
The difference of frame, in other words, will decode B frames afterwards, the caching picture before not only obtaining, the picture after also decoding, and lead to
Cross front and rear picture and obtain final picture with being superimposed for this frame data.
Inside three class frames, I frames compression ratio is minimum, P frames compression ratio secondly, B frame compression ratio highests, to video pictures quality
From the point of view of influence, the influence of I frames is maximum, and secondly, B frames influence very little to P frames.In technical scheme, what the key frame referred to
It is I frames, the non-key frame includes P frames and B frames, under heterogeneous networks quality environment, the technical program is crucial using setting
Frame period parameter controls the quantity of key frame, so as to control the data volume of whole transmission video, the spacing parameter can be according to net
Network quality condition sets different values, and larger spacing parameter is set as far as possible in the case where network quality allows.For example,
In the poor embodiment of one network environment, the spacing parameter is set to send a width key frame data every 5 seconds codings；Another
In the preferable embodiment of network environment, spacing parameter could be arranged to 10 seconds.
Step 2：Receiving terminal is handled RTP packets, if handling successfully, frame of display video, if processing failure,
Send and rearrange key frame or retransmit the RTP request of data lacked in key frame to transmitting terminal；
During transmission of video, receiving terminal is mainly responsible for receiving RTP packets, and framing, then root are carried out to RTP packets
Still image frame is decoded into using corresponded manner according to the different type of frame of video, finally image is shown.Passed in network data
During defeated, it will usually run into two kinds of abnormal conditions：The first, unstable due to network causes the packet of still image frame
Lose, receiving terminal can not obtain complete image frame data, now receiving terminal can influence size according on image quality, and decision is
It is no to transmitting terminal request retransmission RTP packets；Second, image frame data is complete, but occurs mistake in decoding process,
Now receiving terminal will ask transmitting terminal to recompile key frame.Simultaneously as video is made up of a series of orderly image frames,
When RTP packets retransmit, follow-up frame of video can carry out framing processing, but have to wait for the reception of retransmission of video frame
Completely, after the completion of framing, decoding, display, the follow-up frame of video of decoding display could be continued；When frame of video is rearranged in request
Wait, receiving terminal has to wait for rearranging after data frame receipt, framing, decoding, display, then handles subsequent image frames successively.
Step 3：After transmitting terminal is received, sent after the transmission opportunity rearranged key frame and change key frame to receiving terminal, or to connecing
Receiving end retransmits the RTP data lacked in key frame；
Although the key frame interval parameter larger by setting can greatly reduce video data transmission amount, the fluency of video is improved,
But when decoded packet data failure is sent, the recovery time of video pictures can be caused longer, to solve the problem, this hair
Bright technical scheme permission receiving terminal is sent when decoded packet data fails rearranges key frame request, and transmitting terminal, which is received, to be rearranged
After key frame request, the picture frame for requiring to rearrange is encoded to after key frame is sent to receiving terminal as next frame immediately, realized dynamic
State adjusts the transmission opportunity of key frame.When receiving terminal finds data-bag lost, the RTP data for retransmitting and being lacked in key frame are sent
Request, transmitting terminal is received after request, is searched the RTP Data Concurrents lacked in the key frame for requiring to retransmit and is given receiving terminal.It is logical
Cross fixed intervals encoded key frames and being combined according to receiving terminal both modes of dynamic requests encoded key frames, it is possible to achieve protect
On the premise of demonstrate,proving video pictures quality, video data transmission amount is reduced, fluency is improved.
For example, in a specific embodiment, in the case where key frame transmission frequency is fixed, the crucial interframe that transmitting terminal is set
It is 10 seconds every parameter, sent a key frame at interval of 10 seconds in the case of transmission data are normal, it is preferable that video is just opened
Begin that during transmission specially treated can be carried out, what transmitting terminal video data started transmission is encoded into key in the 0th second, the 2nd second, the 4th second
Frame, can so solve transmitting terminal and the receiving terminal processing time inconsistent the beginning part data loss problem caused (due to just opening
Originate and send the time interval of key frame short, even if receiving terminal is not successfully received the key frame of above one or two frame, video reality is influenceed
Less)；Meanwhile, if having and receiving the request for rearranging key frame for first 4 seconds, in order to avoid frequently sending key frame, transmitting terminal is not
Handled.Since after the 4th second, transmitting terminal at interval of 10 seconds send a key frame, i.e., the 14th second, the 24th second ... according to
This analogizes.The request of key frame is rearranged if receiving receiving terminal, the picture frame for requiring to rearrange is encoded to after key frame done immediately
Receiving terminal is sent to for next frame, from being sent the key frame that this is rearranged, then next key frame was sent at interval of 10 seconds again.
For example, after the 24th second encoded key frames of transmitting terminal, being received at the 28th second and rearranging key frame request, then immediately rearranged request
Picture frame be encoded to key frame, and be sent to receiving terminal；Then key frame was sent next time at the 38th second.
When transmitting terminal receives re-transmission key frame request, the image frame data being transmitted across can be sent to receiving terminal again,
Preferably, the transmitting terminal in technical solution of the present invention also can be set and send data buffer zone, cache the RTP packets sent, when
When receiving the RTP data packet requests for retransmitting key frame missing, the transmission data buffer zone is preferentially searched.Under normal circumstances,
It is the firm data sent soon to need the key frame retransmitted, sends data buffer zone by setting, it is possible to reduce from original video
The probability of data file searching data, improves search efficiency.For example, in the less embodiment of network delay, can be set and send
Data buffer zone size has sent RTP data in nearest 2 seconds to preserve；In the larger embodiment of network delay, hair can be set
Data buffer zone size is sent to send RTP data within nearest 5 seconds to preserve.
1-3 repeat the above steps until all video requency frame datas are transmitted.Transmitting terminal and receiving terminal cooperate, until whole
All reception, framing, decoding, display are completed all picture frames that video bag contains in sequence.
It is that a kind of receiving terminal of the embodiment of the present invention carries out process step flow chart, including following step to RTP packets such as Fig. 2
Suddenly：
Step 1: receiving terminal is by the RTP data pack buffers received to reception data buffer；
All reception data are all stored in reception data buffer by receiving terminal, then data are carried out with framing, decoding, display
Processing, due to the continuity of video frame image, it is necessary to after current video frame is shown, could show next frame video image.
In a preferred embodiment, except reception data buffer, receiving terminal also sets up pending buffer area, is closed in request retransmission
In the case of the RTP data lacked in key frame, incomplete frame data are received for caching, the beneficial effect so handled is,
Reception data buffer can continue to follow-up RTP packets, carry out the framing processing of subsequent video frame, key frame to be retransmitted
Display subsequent video frame can be decoded immediately after the completion of complete, framing, decoding, display by receiving, and reduced and retransmitted after key frame processing
Recovery time.
Step 2: judging whether RTP packets are to retransmit the RTP data lacked in key frame；Receiving terminal is according to received data packet
The follow-up processing path of type selecting, do different processing；
If not Step 3: retransmitting the RTP data lacked in key frame, carry out normal data frames processing；
It is a kind of normal data frames process step flow chart of the embodiment of the present invention, step is as follows such as Fig. 3：
Framing is carried out to the RTP bags in reception data buffer, and judges whether frame of video is complete；
Because in video data network transmission process, width static state video requency frame data is often broken up into multiple RTP packets hairs
Send, each RTP bags have network sequence number to be used for the sequencing for identifying data, receiving terminal is received after RTP packets,
According to RTP header packet informations, by multiple RTP data packet groups into original video requency frame data.
In a preferred embodiment, receiving terminal is carried out before framing to RTP packets, is from reception according to the jitter time delay of setting
Buffering area reads RTP packets, the problem of addition jitter time delay can alleviate RTP data packet disorders in network transmission process.
If frame of video is complete, frame of video decoding operate is carried out, it is successfully, otherwise at mark that successfully decoded, which then identifies result,
It is failure to manage result；Decoding herein is that different processing are done according to the type of data frame：If I frames, it is only necessary to this frame number
According to can just reduce video pictures；If P frames are, it is necessary to be superimposed with the difference that this frame data is defined with the picture cached before,
Generate final picture；If B frames, the caching picture before not only obtaining, the picture after also decoding, by front and rear
Picture with being superimposed for this frame data obtains final picture.If successfully decoded, the frame of video is shown, if decoding failure, is needed
Send and rearrange key frame request.
If frame of video is imperfect, determine whether, if the frame of video is key frame, RTP packets are write into pending buffer area
So that receipt of subsequent retransmits framing, and mark result to be failure again after data.Due to key frame on image quality influence compared with
Greatly, in this case, receiving terminal will all initiate to retransmit the RTP request of data lacked in key frame；
If the frame of video is imperfect and is non-key frame, because a small amount of non-key frame missing influences smaller to video pictures quality,
So the technical program employs the mode for directly abandoning the frame of video, so there is no need to carry out the re-transmission of data, network is reduced
The occupancy of bandwidth.But, if the non-key frame directly abandoned between two continuous key frames reaches certain quantity, also can
Situations such as causing video pictures interim card or mosaic, so the technical program abandons non-pass between may be provided at two continuous key frames
The upper limit threshold of key frame, if having reached the upper limit threshold, mark result is failure, and request is rearranged key frame, otherwise marked
Remember that result is successfully, not ask to rearrange key frame.For example, in one embodiment, permitting between two continuous key frames
The upper limit threshold for being permitted to abandon non-key frame is set to 10, when receiving terminal key frame per treatment, will all abandon non-key frame
Aggregate-value zero setting, when receiving terminal abandons a non-key frame, plus 1, if aggregate-value is not more than at 10, continuation by aggregate-value
Subsequent video frame is managed, if aggregate-value is more than 10, request transmitting terminal rearranges key frame, while by aggregate-value again zero setting.
If Step 4: retransmitting the RTP data lacked in key frame, carry out key frame and handle again.
It is a kind of key frame weight process step flow chart of the embodiment of the present invention such as Fig. 4, step is as follows：
By the re-transmission key frame RTP packets write-in pending buffer area of reception；Before buffered inside pending buffer area
The partial data received, as long as the follow-up RTP packets by re-transmission write pending buffer area.
Framing is carried out to the RTP bags in pending buffer area, and judges whether the re-transmission key frame data in pending buffer area is complete；This
The framing operation at place, it is consistent with the framing mode of operation in normal data frames processing.
If it is complete to retransmit key frame, frame of video decoding operate is carried out, it is successfully, otherwise to mark that successfully decoded, which then identifies result,
It is failure to know result；Decoding process mode is consistent with normal frame processing mode herein, the carry out picture of successfully decoded frame of video
It has been shown that, if decoding failure rearranges key frame request, it is necessary to send.
If it is imperfect to retransmit key frame, now need to be judged according to the timeout threshold of setting, if the stand-by period is not less than threshold value,
Continue waiting for, if the stand-by period alreadys exceed threshold value, at this time empty reception data buffer and pending buffer area simultaneously, identify
Result is failure, and key frame request is rearranged in transmission.
It is a kind of real-time video transmission apparatus structure frame diagram of the embodiment of the present invention such as Fig. 5, including：
Sending module one, for performing step 1：Video requency frame data is encoded to key frame or non-key frame by transmitting terminal, sends RTP
Packet is to receiving terminal；The transmission opportunity of the key frame is determined by the spacing parameter configured；
Receiving module, for performing step 2：Receiving terminal is handled RTP packets, if handling successfully, shows video
Frame, if processing failure, sends and rearranges key frame or retransmit the RTP request of data lacked in key frame to transmitting terminal；
Sending module two, for performing step 3：After transmitting terminal is received, after the transmission opportunity rearranged key frame and change key frame
Sent to receiving terminal, or the RTP data lacked in key frame are retransmitted to receiving terminal；
Loop module, for the 1-3 that repeats the above steps until all video requency frame datas are transmitted.
It is a kind of receiving module structural framing figure of the embodiment of the present invention such as Fig. 6, including：
Receiving unit, for receiving RTP packets to reception data buffer；
Pending buffer unit, pending buffer area is set for receiving terminal, and caching retransmits the RTP packets that key frame has been received；
Judging unit, for judging whether RTP packets are the data that lack in request retransmission key frame；
First processing units, if not for retransmitting the RTP data lacked in key frame, carry out normal data frames processing；
Second processing unit, if for retransmitting the RTP data lacked in key frame, carry out key frame and handle again.
It is a kind of first processing units structural framing figure of the embodiment of the present invention such as Fig. 7, including：
First judges component, for carrying out framing to the RTP bags in reception data buffer, and judges whether frame of video is complete；
First decoding mechanism, if complete for frame of video, carries out frame of video decoding operate, successfully decoded then identifies result
For success, result is otherwise identified to fail, and transmission is rearranged key frame and asked to transmitting terminal；
First processing component, whether if imperfect for frame of video, it is key frame to determine whether frame of video, if the frame of video
For key frame, the RTP packets received are write into pending buffer area, mark result is failure, and it is crucial to send re-transmission
The RTP request of data lacked in frame notifies transmitting terminal to retransmit the RTP data lost in key frame to transmitting terminal；If the frame of video
For non-key frame, then the frame of video is abandoned, then judge whether the non-key frame quantity abandoned between two continuous key frames reaches
To threshold value, if having reached threshold value, mark result is fails, and transmission rearranges key frame and asks, to transmitting terminal, otherwise to mark
Remember that result is successfully.
It is a kind of second processing cellular construction frame diagram of the embodiment of the present invention such as Fig. 8, including：
Second receiving member, for the RTP packets of the re-transmission key frame of reception to be write into pending buffer area；
Second judges component, for carrying out framing to the RTP bags in pending buffer area, and judges that the re-transmission in pending buffer area is crucial
Whether frame is complete；
Second decoding mechanism, if complete for retransmitting key frame, carries out frame of video decoding operate, then mark is handled successfully decoded
As a result it is successfully, otherwise to identify result to fail, and transmission is rearranged key frame and asked to transmitting terminal；
Second processing component, if imperfect for retransmitting key frame, further determine whether to wait time-out, is had not timed out if waiting,
Then continue waiting for, otherwise empty reception data buffer and pending buffer area, mark result is failure, and pass is rearranged in transmission
Key frame is asked to transmitting terminal.
Meanwhile, the invention provides a kind of equipment for real-time video transmission, the equipment can be receiving device or send equipment,
Can be specifically the equipment that mobile phone, ladder-port machine, doorway machine, indoor set etc. have transmission of video function.Wherein, equipment is sent to use
In video requency frame data is encoded into key frame or non-key frame, RTP packets are sent to receiving device；The transmission opportunity of key frame
Determined by the spacing parameter configured；Connect backward in addition, sending equipment and being additionally operable to rearrange key frame and change transmission opportunity of key frame
Receiving unit is sent, or the RTP data lacked in the re-transmission key frame of request are sent to receiving device；
Receiving device is used to handle RTP packets, if handling successfully, frame of display video, if processing failure, sends out
Send to rearrange key frame or retransmit the RTP request of data lacked in key frame and give transmission equipment.It is the one of the embodiment of the present invention such as Fig. 9
The transmission equipment structure chart for real-time video transmission is planted, passes through the sending module in the real-time video transmission device comprising the present invention
First, sending module two and loop module, can be achieved the function of real-time video transmission transmitting terminal.Wherein, sending module one, uses
In video requency frame data is encoded into key frame or non-key frame, RTP packets are sent to receiving terminal；During the transmission of the key frame
Machine is determined by the spacing parameter configured；Sending module two, for after transmission opportunity for rearranging key frame and changing key frame to reception
End is sent, or the RTP data lacked in key frame are retransmitted to receiving terminal；Loop module, for the He of circular treatment sending module one
The step of sending module two, is until all video requency frame datas are transmitted.
It is another receiving device structure chart for real-time video transmission of the embodiment of the present invention, by including this hair such as Figure 10
Receiving module in bright real-time video transmission device, can be achieved the function of real-time video transmission receiving terminal.Wherein, receiving module
For handling RTP packets, if handling successfully, frame of display video, if processing failure, key frame is rearranged in transmission
Or the RTP request of data lacked in key frame is retransmitted to transmitting terminal.
In actual applications, by the collaborative work of above-mentioned sending ending equipment and receiving device, real-time video transmission is realized
Function.For example, in visiting guest's video transmitting procedure of gate control system, transmitting terminal is doorway machine, receiving terminal is indoor set.
When the real-time video transmission device of doorway machine and the indoor set application present invention carry out real-time video transmission,
The sending module one of doorway machine, for visiting guest's video requency frame data of collection to be encoded into key frame or non-key frame, and
RTP packets are split into indoor set；Wherein, the transmission opportunity of key frame is determined by the spacing parameter configured；
The receiving module of indoor set, the RTP packets for receiving doorway machine transmission carry out framing, then root to RTP packets
Still image frame is decoded into using corresponded manner according to the different type of frame of video, finally included image on the screen of indoor set.
In processing procedure, if the data of frame of video are imperfect, now indoor set can influence size according on image quality, and decision is
The no packet lacked to doorway machine request retransmission；If image frame data is complete, but occurs mistake in decoding process, this
When indoor set by ask doorway machine rearrange key frame.
The sending module two of doorway machine, for receive rearrange key frame request when, rearrange key frame and change the hair of key frame
Send after opportunity and to be sent to indoor set；Or when receiving the RTP request of data for retransmitting key frame missing, missing is retransmitted to indoor set
RTP packets；
The loop module of doorway machine, for repeating the above steps until all video requency frame datas are sent to indoor set.
And for example, in monitoring system camera video acquisition point and control centre video data transmission, the clothes in video-on-demand applications
The transmission of video being engaged between device and cell-phone customer terminal, can use apparatus of the present invention.In a word, needed between two or more equipment into
, can be using device of the invention during row real-time video transmission.
It is a kind of RTP Transport System for Real-time structural framing figure of the embodiment of the present invention such as Figure 11, including transmitting terminal and receiving terminal, hair
Sending end and receiving terminal are by network connection, and transmitting terminal is used to video requency frame data being encoded to key frame or non-key frame, wherein crucial
Frame is I frames, and non-key frame includes P frames and B frames, and video requency frame data is split into multiple RTP packets is sent to receiving terminal；
Transmitting terminal determines key frame according to the spacing parameter of configuration；Meanwhile, transmitting terminal is additionally operable to rearrange key frame and changes the hair of key frame
Send after opportunity and to be sent to receiving terminal, or the RTP data lacked in key frame are retransmitted to receiving terminal；
Receiving terminal is mainly responsible for receiving RTP packets, carries out framing to RTP packets, the different type further according to frame of video is adopted
Still image frame is decoded into corresponded manner, finally image is shown.Meanwhile, when in a certain still image frame transmission process
When appearance data-bag lost causes data imperfect, receiving terminal determines the need for request retransmission according on image quality influence situation
The RTP data lacked in key frame；Or in decoding process when appearance mistake, key frame request is rearranged in transmission.
In a preferred embodiment, the settable data buffer zone that sends of the transmitting terminal of RTP Transport System for Real-time has been sent out for caching
The RTP packets sent, when receiving the RTP request of data for retransmitting and being lacked in key frame, preferentially search the transmission data and delay
Area is rushed, improves and searches speed, shortens processing time.
In an additional preferred embodiment, reception data buffer and pending buffer area can be set in the receiving terminal of RTP Transport System for Real-time,
Reception data buffer is used to cache the RTP packets received, and it is received that pending buffer area is used for caching re-transmission key frame
RTP packets, by double buffering mechanism, pending buffer area preserves incomplete frame data, and reception data buffer can be after
Follow-up RTP packets are received in continued access, carry out the framing processing of subsequent video frame, key frame to be retransmitted receive completely, framing, decoding,
Display subsequent video frame can be decoded after the completion of display immediately, the recovery time retransmitted after key frame processing is reduced.
Claims (16)
1. a kind of method for real-time video transmission, it is characterised in that comprise the following steps：
Step 1：Video requency frame data is encoded to key frame or non-key frame by transmitting terminal, sends RTP packets to receiving terminal；Institute
The transmission opportunity for stating key frame is determined by the spacing parameter configured；
Step 2：Receiving terminal is handled RTP packets, if handling successfully, frame of display video, if processing failure,
Send and rearrange key frame or retransmit the RTP request of data lacked in key frame to transmitting terminal；
Step 3：After transmitting terminal is received, sent after the transmission opportunity rearranged key frame and change key frame to receiving terminal, or to connecing
Receiving end retransmits the RTP data lacked in key frame；
1-3 repeat the above steps until all video requency frame datas are transmitted.
2. method for real-time video transmission as claimed in claim 1, it is characterised in that the step 1, in addition to：
The RTP packets sent are put into transmission data buffer zone by transmitting terminal, and the RTP that lacks in key frame is retransmitted when receiving
During request of data, the transmission data buffer zone is preferentially searched.
3. method for real-time video transmission as claimed in claim 1, it is characterised in that step 2 receiving terminal is to RTP data
Bag is handled, and is specially：
RTP packets are received to reception data buffer；
Whether judge RTP packets is the data lacked in request retransmission key frame；
If not retransmitting the RTP data lacked in key frame, normal data frames processing is carried out；
If retransmitting the RTP data lacked in key frame, carry out key frame and handle again.
4. method for real-time video transmission as claimed in claim 3, it is characterised in that the step 2 also includes：Receiving terminal is set
Pending buffer area is put, the RTP packets that key frame has been received are retransmitted for caching.
5. method for real-time video transmission as claimed in claim 3, it is characterised in that the receiving terminal is the shake according to setting
Delay reads RTP packets from the reception buffering area.
6. method for real-time video transmission as claimed in claim 4, it is characterised in that the normal data frames processing, including with
Lower step：
Framing is carried out to the RTP bags in reception data buffer, and judges whether frame of video is complete；
If frame of video is complete, frame of video decoding operate is carried out, it is successfully, otherwise at mark that successfully decoded, which then identifies result,
Result is managed to fail, and transmission is rearranged key frame and asked to transmitting terminal；
Whether if frame of video is imperfect, it is key frame to determine whether frame of video, if the frame of video is key frame, will have been received
RTP packets write-in pending buffer area, mark result is failure, and sends the RTP data for retransmitting and being lacked in key frame
Ask, to transmitting terminal, to notify transmitting terminal to retransmit the RTP data lost in key frame；If the frame of video is non-key frame, abandon
The frame of video, then judge whether the non-key frame quantity abandoned between two continuous key frames reaches threshold value, if having reached
Threshold value, mark result is fails, and transmission rearranges key frame and asks, to transmitting terminal, otherwise to mark result to be successfully.
7. method for real-time video transmission as claimed in claim 4, it is characterised in that the carry out key frame is handled again, including
Following steps：
By the RTP packets write-in pending buffer area of the re-transmission key frame of reception；
Framing is carried out to the RTP bags in pending buffer area, and judges whether the re-transmission key frame in pending buffer area is complete；
If it is complete to retransmit key frame, frame of video decoding operate is carried out, it is successfully, otherwise to mark that successfully decoded, which then identifies result,
Result is known to fail, and transmission is rearranged key frame and asked to transmitting terminal；
If it is imperfect to retransmit key frame, further determines whether to wait time-out, have not timed out, continue waiting for, otherwise if waiting
Reception data buffer and pending buffer area are emptied, mark result is fails, and transmission is rearranged key frame and asked to transmitting terminal.
8. a kind of real-time video transmission device, it is characterised in that including：
Sending module one, for performing step 1：Video requency frame data is encoded to key frame or non-key frame by transmitting terminal, sends RTP
Packet is to receiving terminal；The transmission opportunity of the key frame is determined by the spacing parameter configured；
Receiving module, for performing step 2：Receiving terminal is handled RTP packets, if handling successfully, shows video
Frame, if processing failure, sends and rearranges key frame or retransmit the RTP request of data lacked in key frame to transmitting terminal；
Sending module two, for performing step 3：After transmitting terminal is received, after the transmission opportunity rearranged key frame and change key frame
Sent to receiving terminal, or the RTP data lacked in key frame are retransmitted to receiving terminal；
Loop module, for the 1-3 that repeats the above steps until all video requency frame datas are transmitted.
9. real-time video transmission device as claimed in claim 8, it is characterised in that the receiving module, including：
Receiving unit, for receiving RTP packets to reception data buffer；
Judging unit, for judging whether RTP packets are the data that lack in request retransmission key frame；
First processing units, if not for retransmitting the RTP data lacked in key frame, carry out normal data frames processing；
Second processing unit, if for retransmitting the RTP data lacked in key frame, carry out key frame and handle again.
10. real-time video transmission device as claimed in claim 9, it is characterised in that the receiving module, in addition to：
Pending buffer unit, pending buffer area is set for receiving terminal, and caching retransmits the RTP packets that key frame has been received.
11. real-time video transmission device as claimed in claim 10, it is characterised in that the first processing units, including：
First judges component, for carrying out framing to the RTP bags in reception data buffer, and judges whether frame of video is complete；
First decoding mechanism, if complete for frame of video, carries out frame of video decoding operate, successfully decoded then identifies result
For success, result is otherwise identified to fail, and transmission is rearranged key frame and asked to transmitting terminal；
First processing component, whether if imperfect for frame of video, it is key frame to determine whether frame of video, if the frame of video
For key frame, the RTP packets received are write into pending buffer area, mark result is failure, and it is crucial to send re-transmission
The RTP request of data lacked in frame notifies transmitting terminal to retransmit the RTP data lost in key frame to transmitting terminal；If the frame of video
For non-key frame, then the frame of video is abandoned, then judge whether the non-key frame quantity abandoned between two continuous key frames reaches
To threshold value, if having reached threshold value, mark result is fails, and transmission rearranges key frame and asks, to transmitting terminal, otherwise to mark
Remember that result is successfully.
12. real-time video transmission device as claimed in claim 10, it is characterised in that the second processing unit, including：
Second receiving member, for the RTP packets of the re-transmission key frame of reception to be write into pending buffer area；
Second judges component, for carrying out framing to the RTP bags in pending buffer area, and judges that the re-transmission in pending buffer area is crucial
Whether frame is complete；
Second decoding mechanism, if complete for retransmitting key frame, carries out frame of video decoding operate, then mark is handled successfully decoded
As a result it is successfully, otherwise to identify result to fail, and transmission is rearranged key frame and asked to transmitting terminal；
Second processing component, if imperfect for retransmitting key frame, further determine whether to wait time-out, is had not timed out if waiting,
Then continue waiting for, otherwise empty reception data buffer and pending buffer area, mark result is failure, and pass is rearranged in transmission
Key frame is asked to transmitting terminal.
13. a kind of equipment for real-time video transmission, it is characterised in that the equipment for real-time video transmission is transmission
Equipment or receiving device,
The transmission equipment is used to video requency frame data being encoded to key frame or non-key frame, sends RTP packets to receiving device；
The transmission opportunity of the key frame is determined by the spacing parameter configured；The transmission equipment is additionally operable to rearrange key frame and changes key
Sent after the transmission opportunity of frame to receiving device, or the RTP data lacked in the re-transmission key frame of request are sent to receiving device；
The receiving device is used to handle RTP packets, if handling successfully, frame of display video, if processing failure,
Then send to rearrange key frame or retransmit the RTP request of data lacked in key frame and give transmission equipment.
14. a kind of RTP Transport System for Real-time, it is characterised in that including transmitting terminal and receiving terminal, the transmitting terminal is used for video
Frame data are encoded to key frame or non-key frame, send RTP packets to receiving terminal；The transmission opportunity of the key frame is by configuring
Spacing parameter determine；The transmitting terminal is sent after being additionally operable to transmission opportunity for rearranging key frame and changing key frame to receiving terminal,
Or the RTP data lacked in the re-transmission key frame of request are sent to receiving terminal；
The receiving terminal is used to handle RTP packets, if handling successfully, frame of display video, if processing failure,
Send and rearrange key frame or retransmit the RTP request of data lacked in key frame to transmitting terminal.
15. RTP Transport System for Real-time as claimed in claim 14, it is characterised in that the transmitting terminal sets transmission data to delay
Rush area be used for cache the RTP packets that have sent, when receiving the RTP request of data lacked in re-transmission key frame, preferentially look into
Look for the transmission data buffer zone.
16. RTP Transport System for Real-time as claimed in claim 14, it is characterised in that the receiving terminal sets reception data to delay
Area and pending buffer area are rushed, the reception data buffer is used to cache the RTP packets received, and the pending buffer area is used
The RTP packets that key frame has been received are retransmitted in caching.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610167888.7A CN107231328B (en) | 2016-03-23 | 2016-03-23 | Real-time video transmission method, device, equipment and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610167888.7A CN107231328B (en) | 2016-03-23 | 2016-03-23 | Real-time video transmission method, device, equipment and system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN107231328A true CN107231328A (en) | 2017-10-03 |
| CN107231328B CN107231328B (en) | 2020-08-28 |
Family
ID=59931630
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610167888.7A Active CN107231328B (en) | 2016-03-23 | 2016-03-23 | Real-time video transmission method, device, equipment and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107231328B (en) |
Cited By (23)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107846475A (en) * | 2017-12-06 | 2018-03-27 | 中国水利水电科学研究院 | Intelligent water conservancy information investigating method and measure and control device |
| CN107948142A (en) * | 2017-11-15 | 2018-04-20 | 广州杏雨信息科技有限公司 | A kind of point-to-point live transmission method of 3D HD videos and system |
| CN108366306A (en) * | 2017-12-13 | 2018-08-03 | 天津天地伟业机器人技术有限公司 | A kind of control method of the bit stream based on embedded device |
| CN109361935A (en) * | 2018-12-24 | 2019-02-19 | 广州微算互联信息技术有限公司 | Picture stream video transmission method and system |
| CN109862400A (en) * | 2019-02-18 | 2019-06-07 | 苏州长风航空电子有限公司 | A kind of flow-medium transmission method, device and its system |
| CN110062003A (en) * | 2019-04-30 | 2019-07-26 | 北京金山云网络技术有限公司 | Video data transmitting method, device, electronic equipment and storage medium |
| CN110225348A (en) * | 2019-06-24 | 2019-09-10 | 北京大米科技有限公司 | Restorative procedure, device, electronic equipment and the storage medium of video data |
| CN110225347A (en) * | 2019-06-24 | 2019-09-10 | 北京大米科技有限公司 | Method of transmitting video data, device, electronic equipment and storage medium |
| CN110557677A (en) * | 2019-09-27 | 2019-12-10 | 北京西山居互动娱乐科技有限公司 | Video transmission method and device |
| CN110769380A (en) * | 2019-10-31 | 2020-02-07 | 联想(北京)有限公司 | Video distribution method and device |
| CN111163362A (en) * | 2019-12-30 | 2020-05-15 | 北京佳讯飞鸿电气股份有限公司 | Video receiving method and system capable of self-adapting retransmission waiting time |
| CN111326176A (en) * | 2018-12-14 | 2020-06-23 | 中移（杭州）信息技术有限公司 | Detection method, device and medium of RTP packet based on OPUS coding |
| CN111953612A (en) * | 2020-07-21 | 2020-11-17 | 西安万像电子科技有限公司 | Data transmission control method and device |
| CN112291523A (en) * | 2020-10-29 | 2021-01-29 | 合肥安迅精密技术有限公司 | Image data receiving system and method of chip mounter |
| CN112449190A (en) * | 2019-09-05 | 2021-03-05 | 曙光网络科技有限公司 | Decoding method of concurrent video session IPB frame image group |
| CN112689160A (en) * | 2020-11-27 | 2021-04-20 | 烟台艾睿光电科技有限公司 | Video transmission method and device applied to image acquisition equipment |
| CN113452953A (en) * | 2020-03-26 | 2021-09-28 | 浙江宇视科技有限公司 | Video stream transmission control method, device, equipment and medium |
| CN113612962A (en) * | 2021-07-15 | 2021-11-05 | 深圳市捷视飞通科技股份有限公司 | Video conference processing method, system and device |
| CN114567796A (en) * | 2022-03-04 | 2022-05-31 | 北京字节跳动网络技术有限公司 | Frame loss method, device, server and medium |
| CN115102927A (en) * | 2022-04-29 | 2022-09-23 | 厦门立林科技有限公司 | SIP (Session initiation protocol) talkback method, system and storage device for keeping video clear |
| CN115208864A (en) * | 2022-08-12 | 2022-10-18 | 阿波罗智联(北京)科技有限公司 | Data transmission method, device, equipment, vehicle and storage medium |
| CN116112697A (en) * | 2022-11-28 | 2023-05-12 | 长沙千视电子科技有限公司 | NDI-based real-time video recording method |
| WO2023206910A1 (en) * | 2022-04-29 | 2023-11-02 | 厦门立林科技有限公司 | Sip intercom method and system based on local area network and wide area network, and storage medium |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7174561B2 (en) * | 2001-04-13 | 2007-02-06 | Emc Corporation | MPEG dual-channel decoder data and control protocols for real-time video streaming |
| CN101115196A (en) * | 2006-07-27 | 2008-01-30 | 腾讯科技（深圳）有限公司 | Method, system and terminal for adjusting video quality |
| US20090303309A1 (en) * | 2008-06-04 | 2009-12-10 | Pantech Co., Ltd. | Mobile terminal and method for transmitting video data in video telephony system |
| CN101860733A (en) * | 2010-06-11 | 2010-10-13 | 深圳市黄河数字技术有限公司 | 3g network video monitoring system and monitoring method |
| CN101909210A (en) * | 2009-12-17 | 2010-12-08 | 新奥特（北京）视频技术有限公司 | Network streaming media server and low-bandwidth high-quality solution thereof |
| CN101990087A (en) * | 2010-09-28 | 2011-03-23 | 深圳中兴力维技术有限公司 | Wireless video monitoring system and method for dynamically regulating code stream according to network state |
| CN101998101A (en) * | 2009-08-31 | 2011-03-30 | 中兴通讯股份有限公司 | Video data receiving and transmitting systems and video data processing method for video telephone |
| CN103533387A (en) * | 2013-10-21 | 2014-01-22 | 腾讯科技（深圳）有限公司 | Live video control method, equipment and system |
| CN104869461A (en) * | 2015-05-22 | 2015-08-26 | 南京创维信息技术研究院有限公司 | Video data processing system and method |
-
2016
- 2016-03-23 CN CN201610167888.7A patent/CN107231328B/en active Active
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7174561B2 (en) * | 2001-04-13 | 2007-02-06 | Emc Corporation | MPEG dual-channel decoder data and control protocols for real-time video streaming |
| CN101115196A (en) * | 2006-07-27 | 2008-01-30 | 腾讯科技（深圳）有限公司 | Method, system and terminal for adjusting video quality |
| US20090303309A1 (en) * | 2008-06-04 | 2009-12-10 | Pantech Co., Ltd. | Mobile terminal and method for transmitting video data in video telephony system |
| CN101998101A (en) * | 2009-08-31 | 2011-03-30 | 中兴通讯股份有限公司 | Video data receiving and transmitting systems and video data processing method for video telephone |
| CN101909210A (en) * | 2009-12-17 | 2010-12-08 | 新奥特（北京）视频技术有限公司 | Network streaming media server and low-bandwidth high-quality solution thereof |
| CN101860733A (en) * | 2010-06-11 | 2010-10-13 | 深圳市黄河数字技术有限公司 | 3g network video monitoring system and monitoring method |
| CN101990087A (en) * | 2010-09-28 | 2011-03-23 | 深圳中兴力维技术有限公司 | Wireless video monitoring system and method for dynamically regulating code stream according to network state |
| CN103533387A (en) * | 2013-10-21 | 2014-01-22 | 腾讯科技（深圳）有限公司 | Live video control method, equipment and system |
| CN104869461A (en) * | 2015-05-22 | 2015-08-26 | 南京创维信息技术研究院有限公司 | Video data processing system and method |
Cited By (31)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107948142A (en) * | 2017-11-15 | 2018-04-20 | 广州杏雨信息科技有限公司 | A kind of point-to-point live transmission method of 3D HD videos and system |
| CN107846475A (en) * | 2017-12-06 | 2018-03-27 | 中国水利水电科学研究院 | Intelligent water conservancy information investigating method and measure and control device |
| CN108366306A (en) * | 2017-12-13 | 2018-08-03 | 天津天地伟业机器人技术有限公司 | A kind of control method of the bit stream based on embedded device |
| CN111326176A (en) * | 2018-12-14 | 2020-06-23 | 中移（杭州）信息技术有限公司 | Detection method, device and medium of RTP packet based on OPUS coding |
| CN109361935A (en) * | 2018-12-24 | 2019-02-19 | 广州微算互联信息技术有限公司 | Picture stream video transmission method and system |
| CN109862400A (en) * | 2019-02-18 | 2019-06-07 | 苏州长风航空电子有限公司 | A kind of flow-medium transmission method, device and its system |
| CN109862400B (en) * | 2019-02-18 | 2021-08-31 | 苏州长风航空电子有限公司 | Streaming media transmission method, device and system |
| CN110062003A (en) * | 2019-04-30 | 2019-07-26 | 北京金山云网络技术有限公司 | Video data transmitting method, device, electronic equipment and storage medium |
| CN110225348A (en) * | 2019-06-24 | 2019-09-10 | 北京大米科技有限公司 | Restorative procedure, device, electronic equipment and the storage medium of video data |
| CN110225347A (en) * | 2019-06-24 | 2019-09-10 | 北京大米科技有限公司 | Method of transmitting video data, device, electronic equipment and storage medium |
| CN112449190A (en) * | 2019-09-05 | 2021-03-05 | 曙光网络科技有限公司 | Decoding method of concurrent video session IPB frame image group |
| CN110557677A (en) * | 2019-09-27 | 2019-12-10 | 北京西山居互动娱乐科技有限公司 | Video transmission method and device |
| CN110769380A (en) * | 2019-10-31 | 2020-02-07 | 联想(北京)有限公司 | Video distribution method and device |
| CN111163362A (en) * | 2019-12-30 | 2020-05-15 | 北京佳讯飞鸿电气股份有限公司 | Video receiving method and system capable of self-adapting retransmission waiting time |
| CN111163362B (en) * | 2019-12-30 | 2021-12-24 | 北京佳讯飞鸿电气股份有限公司 | Video receiving method and system capable of self-adapting retransmission waiting time |
| CN113452953B (en) * | 2020-03-26 | 2022-06-14 | 浙江宇视科技有限公司 | Video stream transmission control method, device, equipment and medium |
| CN113452953A (en) * | 2020-03-26 | 2021-09-28 | 浙江宇视科技有限公司 | Video stream transmission control method, device, equipment and medium |
| CN111953612A (en) * | 2020-07-21 | 2020-11-17 | 西安万像电子科技有限公司 | Data transmission control method and device |
| CN111953612B (en) * | 2020-07-21 | 2024-02-23 | 西安万像电子科技有限公司 | Data transmission control method and device |
| CN112291523B (en) * | 2020-10-29 | 2023-12-05 | 合肥安迅精密技术有限公司 | Image data receiving system and method of chip mounter equipment |
| CN112291523A (en) * | 2020-10-29 | 2021-01-29 | 合肥安迅精密技术有限公司 | Image data receiving system and method of chip mounter |
| CN112689160A (en) * | 2020-11-27 | 2021-04-20 | 烟台艾睿光电科技有限公司 | Video transmission method and device applied to image acquisition equipment |
| CN113612962A (en) * | 2021-07-15 | 2021-11-05 | 深圳市捷视飞通科技股份有限公司 | Video conference processing method, system and device |
| CN114567796A (en) * | 2022-03-04 | 2022-05-31 | 北京字节跳动网络技术有限公司 | Frame loss method, device, server and medium |
| CN115102927A (en) * | 2022-04-29 | 2022-09-23 | 厦门立林科技有限公司 | SIP (Session initiation protocol) talkback method, system and storage device for keeping video clear |
| CN115102927B (en) * | 2022-04-29 | 2023-10-27 | 厦门立林科技有限公司 | SIP intercom method, system and storage device for keeping video clear |
| WO2023206910A1 (en) * | 2022-04-29 | 2023-11-02 | 厦门立林科技有限公司 | Sip intercom method and system based on local area network and wide area network, and storage medium |
| CN115208864B (en) * | 2022-08-12 | 2024-02-06 | 阿波罗智联(北京)科技有限公司 | Data transmission method, device, equipment, vehicle and storage medium |
| CN115208864A (en) * | 2022-08-12 | 2022-10-18 | 阿波罗智联(北京)科技有限公司 | Data transmission method, device, equipment, vehicle and storage medium |
| CN116112697B (en) * | 2022-11-28 | 2023-08-11 | 长沙千视电子科技有限公司 | NDI-based real-time video recording method |
| CN116112697A (en) * | 2022-11-28 | 2023-05-12 | 长沙千视电子科技有限公司 | NDI-based real-time video recording method |
Also Published As
| Publication number | Publication date |
|---|---|
| CN107231328B (en) | 2020-08-28 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107231328A (en) | Method for real-time video transmission, device, equipment and system | |
| CN109729439B (en) | Real-time video transmission method | |
| Shan | Cross-layer techniques for adaptive video streaming over wireless networks | |
| US8971415B2 (en) | Video communication system, device and method based on feedback reference frames | |
| US9306708B2 (en) | Method and apparatus for retransmission decision making | |
| US8711929B2 (en) | Network-based dynamic encoding | |
| KR101242663B1 (en) | Packet transmission apparatus, communication system and computer-readable recording medium | |
| US8499212B2 (en) | Method and apparatus for adaptive forward error correction with merged automatic repeat request for reliable multicast in wireless local area networks | |
| US20110228845A1 (en) | Systems and methods for transmitting and receiving data streams with feedback information over a lossy network | |
| US20130003579A1 (en) | Method and apparatus for parsing a network abstraction-layer for reliable data communication | |
| Miao et al. | Expected run-time distortion based scheduling for delivery of scalable media | |
| CN102088640A (en) | Adaptive selection retransmission method based on video content | |
| CN108769735A (en) | Optimize Spice methods in a kind of data compression of network transmission | |
| US20230291497A1 (en) | Radio access network configuration for video approximate semantic communications | |
| JPH10126772A (en) | Dynamic image data transfer system | |
| CN101192903B (en) | Data frame coding and decoding control method | |
| Chen et al. | Multi-stages hybrid ARQ with conditional frame skipping and reference frame selecting scheme for real-time video transport over wireless LAN | |
| CN106792265A (en) | A kind of network real-time flow-medium transmission method and system | |
| US10700818B2 (en) | System and method for improving efficiency of wirelessly transmitting video packets | |
| CN106230553A (en) | Data transmission method and method of reseptance, data transmission terminal and reception terminal | |
| CN113542685B (en) | Real-time ultra-high definition video transmission method based on reliable UDP | |
| Jin et al. | Performance evaluation of a hybrid FEC/ARQ for wireless media streaming | |
| JPH10200897A (en) | Moving picture transmission device | |
| WO2024141075A1 (en) | Adaptive method and apparatus for video stream bitrate, and computer device and storage medium | |
| Sun et al. | Quality improvement of real-time video in mobile network |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20200722 Address after: Cangshan District of Fuzhou City, Fujian province 350000 Jinshan Road No. 618 juyuanzhou Ruijie Science Park building 19-22 Applicant after: FUJIAN STAR-NET COMMUNICATION Co.,Ltd. Address before: Cangshan District of Fuzhou City, Fujian province 350000 Jinshan Road No. 618 juyuanzhou Ruijie Science Park building 19-22 Applicant before: FUJIAN STAR-NET COMMUNICATION Co.,Ltd. |
|
| GR01 | Patent grant | ||
| GR01 | Patent grant |