CN114173207B - Method and system for video frame sequential transmission - Google Patents
Method and system for video frame sequential transmission Download PDFInfo
- Publication number
- CN114173207B CN114173207B CN202111349397.1A CN202111349397A CN114173207B CN 114173207 B CN114173207 B CN 114173207B CN 202111349397 A CN202111349397 A CN 202111349397A CN 114173207 B CN114173207 B CN 114173207B
- Authority
- CN
- China
- Prior art keywords
- video frame
- video
- screen end
- time
- decoding
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 230000005540 biological transmission Effects 0.000 title claims abstract description 40
- 238000000034 method Methods 0.000 title claims abstract description 17
- 102100037812 Medium-wave-sensitive opsin 1 Human genes 0.000 description 3
- 230000000153 supplemental effect Effects 0.000 description 2
- 230000005856 abnormality Effects 0.000 description 1
- 230000006835 compression Effects 0.000 description 1
- 238000007906 compression Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 239000013589 supplement Substances 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
- H04N21/85—Assembly of content; Generation of multimedia applications
- H04N21/854—Content authoring
- H04N21/8547—Content authoring involving timestamps for synchronizing content
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
- H04N19/127—Prioritisation of hardware or computational resources
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Computer Security & Cryptography (AREA)
- Computing Systems (AREA)
- Theoretical Computer Science (AREA)
- Compression Or Coding Systems Of Tv Signals (AREA)
Abstract
The invention relates to a multi-screen sharing technology, and discloses a method and a system for video frame sequential transmission, which are applied to multi-screen network transmission, wherein the method comprises the following steps of: the main screen end analyzes the video source to obtain a video frame, and the timestamp corresponding to the video frame is the display time; adding the time stamp corresponding to the video frame into the extension information of the video frame, and modifying the time stamp corresponding to the video frame into decoding time; auxiliary screen end: transmitting the encoded video frame to a secondary screen end through network transmission; the time stamp corresponding to the video frame obtained by the secondary screen end is the decoding time; analyzing the expansion information of the video frames to obtain the display time of the video frames, and sending the video frames to a decoder at the auxiliary screen end according to the sequence of the decoding time; the secondary screen displays the video content according to the order of decoding time. The sequential transmission of video frames is ensured by adding extension information conforming to the corresponding coding format to the code stream of the video frames.
Description
Technical Field
The invention relates to a multi-screen sharing technology, in particular to a method and a system for sequentially transmitting video frames in multi-screen network transmission.
Background
In the application of multi-screen sharing, video content of a main screen needs to be transmitted to each auxiliary screen through a network transmission protocol, in the current traditional technology, most of time stamps of the video are placed in a protocol header of the network transmission protocol for transmission, frames are ordered according to the size of the time stamps in a receiving end, the mode has no problem on a part of videos (decoding time of the video frames is equal to display time), but as video definition becomes higher, decoding time and display time of many videos are inconsistent with each other with development of video coding technology, a player outputs video frames according to decoding sequence through analysis of a video stream, and time stamps corresponding to the video frames are display time, the sequence has no influence on a decoder of the main screen, but on the auxiliary screen end, since the video frames are ordered according to the time stamps after being received, the frames are ordered according to display time, if video data are sent to the decoder of the auxiliary screen according to the sequence, decoding abnormality is caused, and correct results are not obtained.
Disclosure of Invention
Aiming at the problem that in the prior art, video cannot be decoded normally after being transmitted through a network or the decoding time and the display time of encoded video frames are inconsistent, the invention provides a method and a system for transmitting video frames in sequence.
In order to solve the technical problems, the invention is solved by the following technical scheme:
a video frame sequential transmission method is applied to network transmission and comprises a main screen end and at least 1 auxiliary screen end; the method includes the steps of,
main screen end:
the method comprises the steps that a video frame is obtained, a main screen end analyzes a video source to obtain the video frame, and a time stamp corresponding to the video frame is display time;
encoding video frame information, namely adding a time stamp corresponding to the video frame into the extension information of the video frame, and modifying the time stamp corresponding to the video frame into decoding time;
auxiliary screen end:
acquiring video frame information, and transmitting the encoded video frame to a secondary screen end through network transmission; the time stamp corresponding to the video frame obtained by the secondary screen end is the decoding time;
decoding the video frame information, analyzing the expansion information of the video frame to obtain the display time of the video frame, and transmitting the video frame to a decoder at the auxiliary screen end according to the sequence of the decoding time;
the secondary screen displays the video content according to the order of decoding time.
Preferably, the network transmission mode includes, but is not limited to, RTP real-time transmission mode.
Preferably, the video frame information is encoded,
and modifying the data stream, adding a data unit to the modified data stream, and modifying the corresponding time stamp of the video frame.
Preferably, decoding the video frame information analyzes the data unit of the video frame to obtain the display time of the video frame.
In order to solve the technical problem, the invention also provides a system for transmitting video frames sequentially, which comprises a main screen end and at least one group of auxiliary screen ends in network transmission; the device comprises a main screen end video frame acquisition module, a video frame information encoding module, a secondary screen end video frame information acquisition module, a video frame information decoding module and a video frame playing module;
main screen end:
the main screen end analyzes the video source to obtain a video frame, and the timestamp corresponding to the video frame is the display time;
the encoding module of the video frame information adds the time stamp corresponding to the video frame into the extension information of the video frame, and modifies the time stamp corresponding to the video frame into decoding time;
auxiliary screen end:
the secondary screen end video frame information acquisition module is used for transmitting the encoded video frames to the secondary screen end through network transmission; the time stamp corresponding to the video frame obtained by the secondary screen end is the decoding time;
the decoding module of the video frame information analyzes the expansion information of the video frame to obtain the display time of the video frame, and sends the video frame to the decoder at the auxiliary screen end according to the sequence of the decoding time;
and the video playing module is used for displaying video contents by the secondary screen end according to the sequence of the decoding time.
Preferably, the network transmission mode includes, but is not limited to, RTP real-time transmission mode.
Preferably, the video frame information encoding module is used for modifying the video data stream, adding a data unit to the modified data stream, and modifying the timestamp corresponding to the video frame.
Preferably, the decoding module of the video frame information analyzes the data unit of the video frame to obtain the display time of the video frame.
The invention has the remarkable technical effects due to the adoption of the technical scheme:
the invention uses the actual video time stamp as the information of the code stream, and adds the expansion information conforming to the corresponding coding format to the code stream of the video frame, thereby ensuring that the video stream after network transmission is transmitted to the decoder of the auxiliary screen end in the input sequence consistent with the main screen end.
The decoding time of the encoded video frames is consistent with the display time, so that the sequential transmission of the video frames is further ensured.
Drawings
FIG. 1 is a flow chart of an embodiment of the present invention.
Fig. 2 is a GOP sequence diagram of the present invention.
RTP: real-time Transport Protocol, a network transport protocol, real-time transport protocol;
NAL: network Abstraction Layer, a network abstraction layer;
nal_unit_type: the type of NAL unit;
SEI: supplemental Enhancement Information, supplemental enhancement information;
DTS: decoding Time Stamp, decoding the timestamp;
PTS: presentation Time Stamp, displaying the timestamp;
i frame: intra-coded picture Intra-coding an image frame;
p frame: predictive-coded picture Predictive-coding an image frame;
b frame: bidirectionallypredicted picture bi-directionally predictively encoding the image frames;
GOP: group of pictures refers to the distance between two I frames.
Detailed Description
The present invention will be described in further detail with reference to the accompanying drawings and examples.
Example 1
A video frame sequential transmission method is applied to network transmission and comprises a main screen end and at least 1 auxiliary screen end; the method includes the steps of,
for the home screen end:
the method comprises the steps that a video frame is obtained, a main screen end analyzes a video source to obtain the video frame, and a time stamp corresponding to the video frame is display time;
encoding video frame information, namely adding a time stamp corresponding to the video frame into the extension information of the video frame, and modifying the time stamp corresponding to the video frame into decoding time;
for the secondary screen end:
acquiring video frame information, and transmitting the encoded video frame to a secondary screen end through network transmission; the time stamp corresponding to the video frame obtained by the secondary screen end is the decoding time;
decoding the video frame information, analyzing the expansion information of the video frame to obtain the display time of the video frame, and transmitting the video frame to a decoder at the auxiliary screen end according to the sequence of the decoding time;
the secondary screen displays the video content according to the order of decoding time.
Network transport includes, but is not limited to, RTP real-time transport.
The encoding of the video frame information is performed,
and modifying the data stream, adding a data unit to the modified data stream, and modifying the corresponding time stamp of the video frame.
Decoding the video frame information, analyzing the data unit of the video frame, and obtaining the display time of the video frame.
Example 2
Based on embodiment 1, a system for sequentially transmitting video frames in this embodiment includes a main screen end and at least one group of auxiliary screen ends in network transmission; the device comprises a main screen end video frame acquisition module, a video frame information encoding module, a secondary screen end video frame information acquisition module, a video frame information decoding module and a video frame playing module;
for the home screen end:
the main screen end analyzes the video source to obtain a video frame, and the timestamp corresponding to the video frame is the display time;
the encoding module of the video frame information adds the time stamp corresponding to the video frame into the extension information of the video frame, and modifies the time stamp corresponding to the video frame into decoding time;
for the secondary screen end:
the secondary screen end video frame information acquisition module is used for transmitting the encoded video frames to the secondary screen end through network transmission; the time stamp corresponding to the video frame obtained by the secondary screen end is the decoding time;
the decoding module of the video frame information analyzes the expansion information of the video frame to obtain the display time of the video frame, and sends the video frame to the decoder at the auxiliary screen end according to the sequence of the decoding time;
and the video playing module is used for displaying video contents by the secondary screen end according to the sequence of the decoding time.
Network transport includes, but is not limited to, RTP real-time transport.
The video frame information coding module is used for modifying the video data stream, adding data units to the modified data stream and modifying the corresponding time stamps of the video frames.
And the decoding module of the video frame information analyzes the data unit of the video frame to obtain the display time of the video frame.
Example 3
Based on the above embodiment, the video of this embodiment has an encoding format of H264, and one frame of H264 data is composed of a plurality of NAL units; the network transmission protocol adopts RTP protocol; the main screen end analyzes to obtain H264 stream data, and the timestamp corresponding to the frame is the display time; modifying H264 stream data, adding an SEI Unit, wherein the SEI Unit supplements enhancement information which is one Type of NAL_Unit_Type, and can carry out custom editing on information in the NAL Unit, wherein the SEI Unit information comprises display time, and simultaneously, the timestamp corresponding to the frame is modified into decoding time;
transmitting the frame to a secondary screen end through an RTP protocol, wherein the timestamp corresponding to the frame is decoding time;
analyzing an SEI unit of the frame to obtain a display time corresponding to the frame; the H264 frame data with the display time obtained again is sent to a decoder according to the sequence of the decoding time;
the secondary screen displays the video content according to the display time sequence.
Example 4
Based on the above embodiments, this embodiment can be seen from fig. 2, where for an H264 encoded GOP, I frames are taken as an example: intra-coded picture (Intra-frame coded picture), not referring to other picture frames, is coded using only the information of the present frame, P-frame: predictive-coded picture (Predictive-coded picture frame), inter-frame Predictive coding is performed by using a previous I-frame or P-frame and by using a motion prediction method, B-frame: bidirectionallypredicted picture (bi-predictive coded image frames), which provide the highest compression ratio, require both the previous image frame (I-frame or P-frame) and the subsequent image frame (P-frame), use motion prediction for inter-frame bi-predictive coding,
before using the invention, the frame sequence sent to the transmission module by the main screen end is as follows: IPBBPBBPBBPBBPB, the frame with a time stamp of PTS (this is also the frame order that the primary screen side sent to the decoder), so that the frame order received at the secondary screen side becomes ibbpbbpbbppbp, which is sent to the decoder in this order, can cause decoding anomalies (since the B frame needs to refer to the P frame following it to decode, but the P frame has not yet arrived); after the invention is used, the frame sequence received by the auxiliary screen end is still IPBBPBBPBBPBBPB, so that the decoding can be normally performed.
Claims (4)
1. A video frame sequential transmission method is applied to multi-screen network transmission and comprises a main screen end and at least 1 auxiliary screen end; characterized in that the method comprises the steps of,
main screen end:
the method comprises the steps that a video frame is obtained, a main screen end analyzes a video source to obtain the video frame, and a time stamp corresponding to the video frame is display time;
encoding video frame information, namely adding a time stamp corresponding to the video frame into the extension information of the video frame, and modifying the time stamp corresponding to the video frame into decoding time; modifying the data stream, adding a data unit to the modified data stream, and modifying a time stamp corresponding to the video frame;
auxiliary screen end:
acquiring video frame information, and transmitting the encoded video frame to a secondary screen end through network transmission; the time stamp corresponding to the video frame obtained by the secondary screen end is the decoding time;
decoding the video frame information, analyzing the expansion information of the video frame to obtain the display time of the video frame, and transmitting the video frame to a decoder at the auxiliary screen end according to the sequence of the decoding time; analyzing the data unit of the video frame to obtain the display time of the video frame;
the secondary screen displays the video content according to the order of decoding time.
2. A method for sequential transmission of video frames according to claim 1, wherein the network transmission means comprises, but is not limited to, RTP real-time transmission means.
3. A video frame sequential transmission system comprises a main screen end and at least one group of auxiliary screen ends in multi-screen network transmission; the video frame playing system is characterized by comprising a main screen end video frame acquisition module, a video frame information encoding module, a secondary screen end video frame information acquisition module, a video frame information decoding module and a video frame playing module;
main screen end:
the main screen end analyzes the video source to obtain a video frame, and the timestamp corresponding to the video frame is the display time;
the encoding module of the video frame information adds the time stamp corresponding to the video frame into the extension information of the video frame, and modifies the time stamp corresponding to the video frame into decoding time; the coding module of the video frame information is used for modifying the video data stream, adding a data unit to the modified data stream, and modifying the corresponding time stamp of the video frame;
auxiliary screen end:
the secondary screen end video frame information acquisition module is used for transmitting the encoded video frames to the secondary screen end through network transmission; the time stamp corresponding to the video frame obtained by the secondary screen end is the decoding time;
the decoding module of the video frame information analyzes the expansion information of the video frame to obtain the display time of the video frame, and sends the video frame to the decoder at the auxiliary screen end according to the sequence of the decoding time; the decoding module of the video frame information analyzes the data unit of the video frame to obtain the display time of the video frame;
and the video playing module is used for displaying video contents by the secondary screen end according to the sequence of the decoding time.
4. A system for sequential transmission of video frames according to claim 3, wherein the network transmission means includes, but is not limited to, RTP real time transmission means.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111349397.1A CN114173207B (en) | 2021-11-15 | 2021-11-15 | Method and system for video frame sequential transmission |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111349397.1A CN114173207B (en) | 2021-11-15 | 2021-11-15 | Method and system for video frame sequential transmission |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114173207A CN114173207A (en) | 2022-03-11 |
| CN114173207B true CN114173207B (en) | 2024-02-23 |
Family
ID=80479033
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111349397.1A Active CN114173207B (en) | 2021-11-15 | 2021-11-15 | Method and system for video frame sequential transmission |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114173207B (en) |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1751517A (en) * | 2003-02-18 | 2006-03-22 | 诺基亚有限公司 | Picture decoding method |
| CN101605252A (en) * | 2009-07-17 | 2009-12-16 | 深圳创维数字技术股份有限公司 | Program stream is converted to the method and system of transport stream |
| CN102665103A (en) * | 2012-04-13 | 2012-09-12 | 烽火通信科技股份有限公司 | Audio and video packaging method applicable to streaming media services |
| CN102685588A (en) * | 2011-03-11 | 2012-09-19 | 汤姆森特许公司 | Decoder and method at the decoder for synchronizing rendering of contents received through different networks |
| CN105338426A (en) * | 2015-10-30 | 2016-02-17 | 北京数码视讯科技股份有限公司 | Correction method for time stamp in transport stream and device thereof |
| CN105959786A (en) * | 2016-04-28 | 2016-09-21 | 乐视控股（北京）有限公司 | Audio and video playing method and device |
| CN110225416A (en) * | 2019-05-31 | 2019-09-10 | 杭州涂鸦信息技术有限公司 | A kind of transmission method of video, the network terminal, intelligent terminal and storage device |
| CN110753202A (en) * | 2019-10-30 | 2020-02-04 | 广州河东科技有限公司 | Audio and video synchronization method, device, equipment and storage medium of video intercom system |
| CN112653904A (en) * | 2020-12-16 | 2021-04-13 | 杭州当虹科技股份有限公司 | Rapid video clipping method based on PTS and DTS modification |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8437619B2 (en) * | 2010-12-20 | 2013-05-07 | General Instrument Corporation | Method of processing a sequence of coded video frames |
| KR102227661B1 (en) * | 2014-01-08 | 2021-03-15 | 삼성전자주식회사 | Method for screen mirroring and apparatus thereof |
| US20170105010A1 (en) * | 2015-10-09 | 2017-04-13 | Microsoft Technology Licensing, Llc | Receiver-side modifications for reduced video latency |
| CN105491414B (en) * | 2015-11-19 | 2017-05-17 | 深圳市鹰硕技术有限公司 | Synchronous display method and device of images |
-
2021
- 2021-11-15 CN CN202111349397.1A patent/CN114173207B/en active Active
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1751517A (en) * | 2003-02-18 | 2006-03-22 | 诺基亚有限公司 | Picture decoding method |
| CN101605252A (en) * | 2009-07-17 | 2009-12-16 | 深圳创维数字技术股份有限公司 | Program stream is converted to the method and system of transport stream |
| CN102685588A (en) * | 2011-03-11 | 2012-09-19 | 汤姆森特许公司 | Decoder and method at the decoder for synchronizing rendering of contents received through different networks |
| CN102665103A (en) * | 2012-04-13 | 2012-09-12 | 烽火通信科技股份有限公司 | Audio and video packaging method applicable to streaming media services |
| CN105338426A (en) * | 2015-10-30 | 2016-02-17 | 北京数码视讯科技股份有限公司 | Correction method for time stamp in transport stream and device thereof |
| CN105959786A (en) * | 2016-04-28 | 2016-09-21 | 乐视控股（北京）有限公司 | Audio and video playing method and device |
| CN110225416A (en) * | 2019-05-31 | 2019-09-10 | 杭州涂鸦信息技术有限公司 | A kind of transmission method of video, the network terminal, intelligent terminal and storage device |
| CN110753202A (en) * | 2019-10-30 | 2020-02-04 | 广州河东科技有限公司 | Audio and video synchronization method, device, equipment and storage medium of video intercom system |
| CN112653904A (en) * | 2020-12-16 | 2021-04-13 | 杭州当虹科技股份有限公司 | Rapid video clipping method based on PTS and DTS modification |
Non-Patent Citations (1)
| Title |
|---|
| 基于H.264标准的视频编解码器软终端的一种实现;寇毅;朱志祥;;西安邮电学院学报(第05期);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114173207A (en) | 2022-03-11 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11070833B2 (en) | Method and system for encoding video with overlay | |
| TWI280059B (en) | Picture coding method | |
| TWI407796B (en) | An image signal decoding apparatus, an image signal decoding method, an image signal decoding method, an image signal coding method, and a program | |
| US7436454B2 (en) | Method and apparatus for transmitting encoded information based upon piority data in the encoded information | |
| US6980594B2 (en) | Generation of MPEG slow motion playout | |
| KR100711635B1 (en) | Picture coding method | |
| CN102986218B (en) | Video for stream video data switches | |
| CN100568964C (en) | Picture decoding method | |
| CN101252696B (en) | For the method and apparatus of Video coding | |
| CN101682762B (en) | Method for realizing random access in compressed code stream using multi-reference images and decoder | |
| US10187646B2 (en) | Encoding device, encoding method, transmission device, decoding device, decoding method, and reception device | |
| RU2283543C2 (en) | Method for processing error code of compressed image during transmission | |
| MXPA05011533A (en) | Picture coding method. | |
| CN110392284A (en) | Video coding, video data handling procedure, device, computer equipment and storage medium | |
| JP2001025007A (en) | Mpeg stream switching method | |
| CN114173207B (en) | Method and system for video frame sequential transmission | |
| CN101888545B (en) | Compression coding method for signal source with low code rate | |
| CN113973207A (en) | Real-time control and playback method for IOS remote real machine WEB end based on H264 code stream | |
| JPH10262228A (en) | Communication system, multi-point controller and video information display method | |
| WO2010076769A1 (en) | Embedding of addressable and context based messages in encoded video streams | |
| US9219930B1 (en) | Method and system for timing media stream modifications | |
| CN118632004A (en) | Screen image encoding method, decoding method, system, device and medium | |
| CN118158420A (en) | Image encoding method, image decoding device, and computer storage medium | |
| JP2000152249A (en) | Image coder | |
| CN116916090A (en) | Video stream data processing method and device, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |