CN114173207A - Method and system for transmitting video frames sequentially - Google Patents
Method and system for transmitting video frames sequentially Download PDFInfo
- Publication number
- CN114173207A CN114173207A CN202111349397.1A CN202111349397A CN114173207A CN 114173207 A CN114173207 A CN 114173207A CN 202111349397 A CN202111349397 A CN 202111349397A CN 114173207 A CN114173207 A CN 114173207A
- Authority
- CN
- China
- Prior art keywords
- video frame
- screen end
- video
- time
- information
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 15
- 230000005540 biological transmission Effects 0.000 claims abstract description 33
- 102100037812 Medium-wave-sensitive opsin 1 Human genes 0.000 description 3
- 230000002457 bidirectional effect Effects 0.000 description 3
- 230000002159 abnormal effect Effects 0.000 description 1
- 230000005856 abnormality Effects 0.000 description 1
- 230000006835 compression Effects 0.000 description 1
- 238000007906 compression Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000000153 supplemental effect Effects 0.000 description 1
- 230000001502 supplementing effect Effects 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
- H04N21/85—Assembly of content; Generation of multimedia applications
- H04N21/854—Content authoring
- H04N21/8547—Content authoring involving timestamps for synchronizing content
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
- H04N19/127—Prioritisation of hardware or computational resources
Abstract
The invention relates to a multi-screen sharing technology, and discloses a method and a system for sequentially transmitting video frames, which are applied to multi-screen network transmission, wherein the method comprises the following steps that: the main screen terminal analyzes the video source to obtain a video frame, and a timestamp corresponding to the video frame is display time; adding a timestamp corresponding to the video frame into the extended information of the video frame, and modifying the timestamp corresponding to the video frame into decoding time; and (4) secondary screen end: transmitting the encoded video frame to the secondary screen end through network transmission; the time stamp corresponding to the video frame obtained by the auxiliary screen end is decoding time; analyzing the extended information of the video frame to obtain the display time of the video frame, and sending the video frame to a decoder at the auxiliary screen end according to the sequence of the decoding time; and the secondary screen end displays the video content according to the sequence of the decoding time. By adding the extended information conforming to the corresponding coding format to the code stream of the video frame, the sequential transmission of the video frame is ensured.
Description
Technical Field
The invention relates to a multi-screen sharing technology, in particular to a method and a system for sequentially transmitting video frames in multi-screen network transmission.
Background
In the multi-screen sharing application, the video content of the primary screen needs to be transmitted to each secondary screen through a network transmission protocol, in the current conventional technology, most of timestamps of videos are transmitted in a protocol header of the network transmission protocol, frames are sorted according to the size of the timestamps in a receiving end, the mode is not problematic for a part of videos (the decoding time of the video frames is equal to the display time), but as the definition of the videos becomes higher, the decoding time and the display time of many videos are inconsistent with the development of the video coding technology, a player analyzes a video stream and outputs the video frames according to the decoding sequence, the timestamps corresponding to the video frames are the display time, the sequence has no influence on a decoder of the primary screen, but at the secondary screen end, due to the fact that the video frames are sorted according to the timestamps after being received, the display time will be ordered, and if the video data is sent to the decoder of the secondary screen in this order, it will cause an abnormal decoding and fail to obtain the correct result.
Disclosure of Invention
The invention provides a method and a system for sequentially transmitting video frames, aiming at the problems that videos cannot be normally decoded after being transmitted through a network or the decoding time and the display time of coded video frames are inconsistent in the prior art.
In order to solve the technical problem, the invention is solved by the following technical scheme:
a video frame sequential transmission method is applied to network transmission and comprises a main screen end and at least 1 auxiliary screen end; the method comprises the steps of (1) carrying out,
the main screen end:
acquiring a video frame, analyzing a video source by a main screen end to obtain the video frame, wherein a timestamp corresponding to the video frame is display time;
coding video frame information, namely adding a timestamp corresponding to a video frame into extended information of the video frame, and modifying the timestamp corresponding to the video frame into decoding time;
and (4) secondary screen end:
acquiring video frame information, and transmitting the encoded video frame to an auxiliary screen end through network transmission; the time stamp corresponding to the video frame obtained by the auxiliary screen end is decoding time;
decoding video frame information, analyzing the extended information of the video frame to obtain the display time of the video frame, and sending the video frame to a decoder at an auxiliary screen end according to the sequence of the decoding time;
and the secondary screen end displays the video content according to the sequence of the decoding time.
Preferably, the network transmission mode includes, but is not limited to, an RTP real-time transmission mode.
Preferably, the encoding of the video frame information,
and modifying the data stream, namely adding a data unit to the modified data stream, and modifying the corresponding time stamp of the video frame.
Preferably, the decoding of the video frame information analyzes the data unit of the video frame to obtain the display time of the video frame.
In order to solve the above technical problem, the present invention further provides a system for sequentially transmitting video frames, which comprises a main screen end and at least one group of auxiliary screen ends in network transmission; the system comprises a main screen video frame acquisition module, a video frame information coding module, an auxiliary screen video frame information acquisition module, a video frame information decoding module and a video frame playing module;
the main screen end:
the main screen end video frame acquisition module analyzes a video source to obtain a video frame, and a timestamp corresponding to the video frame is display time;
the video frame information encoding module is used for adding the timestamp corresponding to the video frame into the extended information of the video frame and modifying the timestamp corresponding to the video frame into decoding time;
and (4) secondary screen end:
the acquisition module of the video frame information of the auxiliary screen end transmits the encoded video frame to the auxiliary screen end through network transmission; the time stamp corresponding to the video frame obtained by the auxiliary screen end is decoding time;
the decoding module of the video frame information analyzes the extended information of the video frame to obtain the display time of the video frame, and sends the video frame to a decoder at the auxiliary screen end according to the sequence of the decoding time;
and the video frame playing module is used for displaying the video content at the auxiliary screen end according to the sequence of the decoding time.
Preferably, the network transmission mode includes, but is not limited to, an RTP real-time transmission mode.
Preferably, the video frame information encoding module is configured to modify a video data stream, add a data unit to the modified data stream, and modify a timestamp corresponding to the video frame.
Preferably, the decoding module of the video frame information analyzes the data unit of the video frame to obtain the display time of the video frame.
Due to the adoption of the technical scheme, the invention has the remarkable technical effects that:
the invention ensures that the video stream transmitted by the network keeps the input sequence consistent with the input sequence of the main screen end and is transmitted to the decoder of the auxiliary screen end by taking the actual video time stamp as the information of the code stream and adding the extended information conforming to the corresponding coding format to the code stream of the video frame.
The decoding time and the display time of the coded video frame are consistent, and the sequential transmission of the video frame is further ensured.
Drawings
FIG. 1 is a flow chart of an embodiment of the present invention.
Fig. 2 is a GOP sequence diagram of the present invention.
RTP: real-time Transport Protocol, a network Transport Protocol, Real-time Transport Protocol;
NAL: network Abstraction Layer, Network Abstraction Layer;
NAL _ Unit _ Type: type of NAL unit;
SEI: supplemental Enhancement Information;
DTS: decoding Time Stamp;
PTS: presentation Time Stamp, displaying a timestamp;
i frame: intra-coded picture Intra-coded image frames;
p frame: predictive-coded picture prediction coding image frame;
b frame: bidirectional predicted picture frames;
GOP: group of pictures (Group of pictures), which refers to the distance between two I-frames.
Detailed Description
The present invention will be described in further detail with reference to the accompanying drawings and examples.
Example 1
A video frame sequential transmission method is applied to network transmission and comprises a main screen end and at least 1 auxiliary screen end; the method comprises the steps of (1) carrying out,
for the main screen end:
acquiring a video frame, analyzing a video source by a main screen end to obtain the video frame, wherein a timestamp corresponding to the video frame is display time;
coding video frame information, namely adding a timestamp corresponding to a video frame into extended information of the video frame, and modifying the timestamp corresponding to the video frame into decoding time;
for the secondary screen end:
acquiring video frame information, and transmitting the encoded video frame to an auxiliary screen end through network transmission; the time stamp corresponding to the video frame obtained by the auxiliary screen end is decoding time;
decoding video frame information, analyzing the extended information of the video frame to obtain the display time of the video frame, and sending the video frame to a decoder at an auxiliary screen end according to the sequence of the decoding time;
and the secondary screen end displays the video content according to the sequence of the decoding time.
The network transmission mode includes but is not limited to RTP real-time transmission mode.
The encoding of the video frame information is performed,
and modifying the data stream, namely adding a data unit to the modified data stream, and modifying the corresponding time stamp of the video frame.
And decoding the video frame information, and analyzing the data unit of the video frame to obtain the display time of the video frame.
Example 2
On the basis of embodiment 1, the system for sequentially transmitting video frames in this embodiment includes a main screen end and at least one group of auxiliary screen ends in network transmission; the system comprises a main screen video frame acquisition module, a video frame information coding module, an auxiliary screen video frame information acquisition module, a video frame information decoding module and a video frame playing module;
for the main screen end:
the main screen end video frame acquisition module analyzes a video source to obtain a video frame, and a timestamp corresponding to the video frame is display time;
the video frame information encoding module is used for adding the timestamp corresponding to the video frame into the extended information of the video frame and modifying the timestamp corresponding to the video frame into decoding time;
for the secondary screen end:
the acquisition module of the video frame information of the auxiliary screen end transmits the encoded video frame to the auxiliary screen end through network transmission; the time stamp corresponding to the video frame obtained by the auxiliary screen end is decoding time;
the decoding module of the video frame information analyzes the extended information of the video frame to obtain the display time of the video frame, and sends the video frame to a decoder at the auxiliary screen end according to the sequence of the decoding time;
and the video frame playing module is used for displaying the video content at the auxiliary screen end according to the sequence of the decoding time.
The network transmission mode includes but is not limited to RTP real-time transmission mode.
And the video frame information coding module is used for modifying the video data stream, adding a data unit to the modified data stream and modifying the corresponding timestamp of the video frame.
And the decoding module of the video frame information analyzes the data unit of the video frame to obtain the display time of the video frame.
Example 3
On the basis of the above embodiments, the encoding format of the video of this embodiment is H264, and a frame of H264 data is composed of a plurality of NAL units; the network transmission protocol adopts an RTP protocol; the main screen terminal analyzes to obtain H264 stream data, and the timestamp corresponding to the frame is the display time; modifying H264 streaming data, adding an SEI Unit, supplementing enhancement information by the SEI Unit, wherein the SEI Unit is one Type of NAL _ Unit _ Type, and can perform custom editing on information in the NAL Unit to obtain SEI Unit information including display time and modify a timestamp corresponding to the frame into decoding time;
the frame is transmitted to the auxiliary screen end through an RTP protocol, and the time stamp corresponding to the frame is the decoding time;
analyzing the SEI unit of the frame to obtain the display time corresponding to the frame; sending the H264 frame data with the display time obtained again to a decoder according to the sequence of decoding time;
and the auxiliary screen end displays the video contents according to the sequence of the display time.
Example 4
Based on the above embodiment, as can be seen from fig. 2, for an H264 encoded GOP as an example, I frame: intra-coded picture (Intra-frame coded picture frame), which is coded using only the information of the present frame without referring to other picture frames, P frame: predictive-coded picture (Predictive-coded picture frame), inter-frame Predictive coding is performed by using the previous I frame or P frame in a motion prediction manner, and a B frame: bidirectional predictive coded picture, providing the highest compression ratio, which requires both previous image frames (I-frames or P-frames) and subsequent image frames (P-frames), inter-frame bidirectional predictive coding using motion prediction,
before the invention is used, the frame sequence sent to the transmission module by the main screen end is as follows: ipbbpbbbpbbpb, the timestamp of the frame is PTS (this is also the frame sequence sent to the decoder by the primary screen), so the frame sequence received at the secondary screen end will become IBBPBBPBBPBBPBP, and sending to the decoder in such sequence will cause decoding abnormality (because B frame needs to refer to the P frame after it to decode, but the P frame has not arrived at this time); after the invention is used, the sequence of the frames received by the auxiliary screen end is still IPBBPBBPBBPBBPB, thereby ensuring that the decoding can be normally carried out.
Claims (8)
1. A video frame sequential transmission method is applied to multi-screen network transmission and comprises a main screen end and at least 1 auxiliary screen end; which is characterized in that the method comprises the following steps,
the main screen end:
acquiring a video frame, analyzing a video source by a main screen end to obtain the video frame, wherein a timestamp corresponding to the video frame is display time;
coding video frame information, namely adding a timestamp corresponding to a video frame into extended information of the video frame, and modifying the timestamp corresponding to the video frame into decoding time;
and (4) secondary screen end:
acquiring video frame information, and transmitting the encoded video frame to an auxiliary screen end through network transmission; the time stamp corresponding to the video frame obtained by the auxiliary screen end is decoding time;
decoding video frame information, analyzing the extended information of the video frame to obtain the display time of the video frame, and sending the video frame to a decoder at an auxiliary screen end according to the sequence of the decoding time;
and the secondary screen end displays the video content according to the sequence of the decoding time.
2. The method of claim 1, wherein the network transport comprises but is not limited to RTP real-time transport.
3. The method of claim 1, wherein the encoding of the video frame information,
and modifying the data stream, namely adding a data unit to the modified data stream, and modifying the corresponding time stamp of the video frame.
4. The method of claim 1, wherein the decoding of the video frame information parses the data units of the video frame to obtain the display time of the video frame.
5. A video frame sequential transmission system comprises a main screen end and at least one group of auxiliary screen ends in multi-screen network transmission; the device is characterized by comprising a main screen video frame acquisition module, a video frame information coding module, an auxiliary screen video frame information acquisition module, a video frame information decoding module and a video frame playing module;
the main screen end:
the main screen end video frame acquisition module analyzes a video source to obtain a video frame, and a timestamp corresponding to the video frame is display time;
the video frame information encoding module is used for adding the timestamp corresponding to the video frame into the extended information of the video frame and modifying the timestamp corresponding to the video frame into decoding time;
and (4) secondary screen end:
the acquisition module of the video frame information of the auxiliary screen end transmits the encoded video frame to the auxiliary screen end through network transmission; the time stamp corresponding to the video frame obtained by the auxiliary screen end is decoding time;
the decoding module of the video frame information analyzes the extended information of the video frame to obtain the display time of the video frame, and sends the video frame to a decoder at the auxiliary screen end according to the sequence of the decoding time;
and the video frame playing module is used for displaying the video content at the auxiliary screen end according to the sequence of the decoding time.
6. The system of claim 5, wherein the network transport comprises but is not limited to RTP real-time transport.
7. The system of claim 5, wherein the video frame information encoding module is configured to modify the video data stream, add data units to the modified video data stream, and modify timestamps corresponding to the video frames.
8. The system of claim 5, wherein the video frame information decoding module parses the data unit of the video frame to obtain the display time of the video frame.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111349397.1A CN114173207B (en) | 2021-11-15 | 2021-11-15 | Method and system for video frame sequential transmission |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111349397.1A CN114173207B (en) | 2021-11-15 | 2021-11-15 | Method and system for video frame sequential transmission |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114173207A true CN114173207A (en) | 2022-03-11 |
| CN114173207B CN114173207B (en) | 2024-02-23 |
Family
ID=80479033
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111349397.1A Active CN114173207B (en) | 2021-11-15 | 2021-11-15 | Method and system for video frame sequential transmission |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114173207B (en) |
Citations (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1751517A (en) * | 2003-02-18 | 2006-03-22 | 诺基亚有限公司 | Picture decoding method |
| CN101605252A (en) * | 2009-07-17 | 2009-12-16 | 深圳创维数字技术股份有限公司 | Program stream is converted to the method and system of transport stream |
| US20120155833A1 (en) * | 2010-12-20 | 2012-06-21 | General Instrument Corporation | Method of Processing a Sequence of Coded Video Frames |
| CN102665103A (en) * | 2012-04-13 | 2012-09-12 | 烽火通信科技股份有限公司 | Audio and video packaging method applicable to streaming media services |
| CN102685588A (en) * | 2011-03-11 | 2012-09-19 | 汤姆森特许公司 | Decoder and method at the decoder for synchronizing rendering of contents received through different networks |
| US20150195601A1 (en) * | 2014-01-08 | 2015-07-09 | Samsung Electronics Co., Ltd. | Method and device for screen mirroring |
| CN105338426A (en) * | 2015-10-30 | 2016-02-17 | 北京数码视讯科技股份有限公司 | Correction method for time stamp in transport stream and device thereof |
| CN105959786A (en) * | 2016-04-28 | 2016-09-21 | 乐视控股（北京）有限公司 | Audio and video playing method and device |
| US20170105010A1 (en) * | 2015-10-09 | 2017-04-13 | Microsoft Technology Licensing, Llc | Receiver-side modifications for reduced video latency |
| US20180247550A1 (en) * | 2015-11-19 | 2018-08-30 | Shenzhen Eaglesoul Technology Co., Ltd. | Image synchronous display method and device |
| CN110225416A (en) * | 2019-05-31 | 2019-09-10 | 杭州涂鸦信息技术有限公司 | A kind of transmission method of video, the network terminal, intelligent terminal and storage device |
| CN110753202A (en) * | 2019-10-30 | 2020-02-04 | 广州河东科技有限公司 | Audio and video synchronization method, device, equipment and storage medium of video intercom system |
| CN112653904A (en) * | 2020-12-16 | 2021-04-13 | 杭州当虹科技股份有限公司 | Rapid video clipping method based on PTS and DTS modification |
-
2021
- 2021-11-15 CN CN202111349397.1A patent/CN114173207B/en active Active
Patent Citations (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1751517A (en) * | 2003-02-18 | 2006-03-22 | 诺基亚有限公司 | Picture decoding method |
| CN101605252A (en) * | 2009-07-17 | 2009-12-16 | 深圳创维数字技术股份有限公司 | Program stream is converted to the method and system of transport stream |
| US20120155833A1 (en) * | 2010-12-20 | 2012-06-21 | General Instrument Corporation | Method of Processing a Sequence of Coded Video Frames |
| CN102685588A (en) * | 2011-03-11 | 2012-09-19 | 汤姆森特许公司 | Decoder and method at the decoder for synchronizing rendering of contents received through different networks |
| CN102665103A (en) * | 2012-04-13 | 2012-09-12 | 烽火通信科技股份有限公司 | Audio and video packaging method applicable to streaming media services |
| US20150195601A1 (en) * | 2014-01-08 | 2015-07-09 | Samsung Electronics Co., Ltd. | Method and device for screen mirroring |
| US20170105010A1 (en) * | 2015-10-09 | 2017-04-13 | Microsoft Technology Licensing, Llc | Receiver-side modifications for reduced video latency |
| CN105338426A (en) * | 2015-10-30 | 2016-02-17 | 北京数码视讯科技股份有限公司 | Correction method for time stamp in transport stream and device thereof |
| US20180247550A1 (en) * | 2015-11-19 | 2018-08-30 | Shenzhen Eaglesoul Technology Co., Ltd. | Image synchronous display method and device |
| CN105959786A (en) * | 2016-04-28 | 2016-09-21 | 乐视控股（北京）有限公司 | Audio and video playing method and device |
| CN110225416A (en) * | 2019-05-31 | 2019-09-10 | 杭州涂鸦信息技术有限公司 | A kind of transmission method of video, the network terminal, intelligent terminal and storage device |
| CN110753202A (en) * | 2019-10-30 | 2020-02-04 | 广州河东科技有限公司 | Audio and video synchronization method, device, equipment and storage medium of video intercom system |
| CN112653904A (en) * | 2020-12-16 | 2021-04-13 | 杭州当虹科技股份有限公司 | Rapid video clipping method based on PTS and DTS modification |
Non-Patent Citations (1)
| Title |
|---|
| 寇毅;朱志祥;: "基于H.264标准的视频编解码器软终端的一种实现", 西安邮电学院学报, no. 05 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114173207B (en) | 2024-02-23 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11943480B2 (en) | Constraints and unit types to simplify video random access | |
| US11070833B2 (en) | Method and system for encoding video with overlay | |
| KR100711635B1 (en) | Picture coding method | |
| TWI280059B (en) | Picture coding method | |
| US7436454B2 (en) | Method and apparatus for transmitting encoded information based upon piority data in the encoded information | |
| US9894381B1 (en) | Managing multi-reference picture buffers for video data coding | |
| TWI407796B (en) | An image signal decoding apparatus, an image signal decoding method, an image signal decoding method, an image signal coding method, and a program | |
| EP3158752B1 (en) | Dependent random access point pictures | |
| US9819955B2 (en) | Carriage systems encoding or decoding JPEG 2000 video | |
| US10187646B2 (en) | Encoding device, encoding method, transmission device, decoding device, decoding method, and reception device | |
| US20130176381A1 (en) | System, method, and apparatus for displaying pictures | |
| CN114930829A (en) | Sub-picture signaling in video coding | |
| US11843822B2 (en) | Transmission apparatus, transmission method, encoding apparatus, encoding method, reception apparatus, and reception method | |
| CN114173207B (en) | Method and system for video frame sequential transmission | |
| WO2018037985A1 (en) | Transmission apparatus, transmission method, reception apparatus, and reception method | |
| JP2018033194A (en) | Encoding device | |
| JP6614281B2 (en) | Receiving apparatus and receiving method | |
| US9219930B1 (en) | Method and system for timing media stream modifications | |
| JP6350638B2 (en) | Transmitting apparatus, transmitting method, receiving apparatus, and receiving method | |
| JP6257448B2 (en) | Encoding device, decoding device, encoding method, and decoding method | |
| WO2004114666A1 (en) | Constant stream compression processing method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |