CN112449138A - Low-delay video transmission system and method - Google Patents
Low-delay video transmission system and method Download PDFInfo
- Publication number
- CN112449138A CN112449138A CN202011294245.1A CN202011294245A CN112449138A CN 112449138 A CN112449138 A CN 112449138A CN 202011294245 A CN202011294245 A CN 202011294245A CN 112449138 A CN112449138 A CN 112449138A
- Authority
- CN
- China
- Prior art keywords
- video
- fpga
- low
- data
- frame
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000005540 biological transmission Effects 0.000 title claims abstract description 38
- 238000000034 method Methods 0.000 title claims abstract description 20
- 230000000903 blocking effect Effects 0.000 claims abstract description 13
- 238000005516 engineering process Methods 0.000 claims abstract description 7
- 238000012545 processing Methods 0.000 claims description 23
- 238000005259 measurement Methods 0.000 claims description 7
- 238000007906 compression Methods 0.000 description 10
- 230000006835 compression Effects 0.000 description 10
- 238000006243 chemical reaction Methods 0.000 description 2
- 238000013144 data compression Methods 0.000 description 2
- 230000006872 improvement Effects 0.000 description 2
- 238000000638 solvent extraction Methods 0.000 description 2
- 230000003044 adaptive effect Effects 0.000 description 1
- 230000004075 alteration Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 239000002131 composite material Substances 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 239000000203 mixture Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
- 238000004904 shortening Methods 0.000 description 1
- 230000001360 synchronised effect Effects 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N5/00—Details of television systems
- H04N5/76—Television signal recording
- H04N5/765—Interface circuits between an apparatus for recording and another apparatus
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N5/00—Details of television systems
- H04N5/222—Studio circuitry; Studio devices; Studio equipment
- H04N5/262—Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects
- H04N5/268—Signal distribution or switching
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Compression Or Coding Systems Of Tv Signals (AREA)
Abstract
The invention discloses a low-delay video transmission system and a low-delay video transmission method. The low-delay video transmission system comprises a camera, an FPGA, a main processor and an antenna; the output end of the camera is connected with the FPGA, the FPGA is bidirectionally connected with the main controller, and the FPGA is connected with the ground station through an antenna; the FPGA includes a video blocking module. On the basis of a general H264 algorithm, the serial work of the whole video transmission system is converted into parallel work through a video blocking technology, and the low-delay transmission of the video is quickly realized.
Description
Technical Field
The invention relates to the field of video processing, in particular to a low-delay video transmission system and a low-delay video transmission method.
Background
Along with unmanned aerial vehicle's application is more and more, also higher and more high to unmanned aerial vehicle's requirement, for example in the video transmission field, need unmanned aerial vehicle when high-speed flight, can in time provide clear reliable video image. Compressed data of the unmanned aerial vehicle airborne video processing system is subjected to composite processing with various data such as system task equipment, remote measurement and the like, and then transmitted to a ground station through a wireless channel for processing. Therefore, the designed video processing system of the unmanned aerial vehicle can constantly output data to a data link as far as possible, the time for transmitting the video data to the ground station is as short as possible, the composition of various data such as system task equipment and remote measurement and the like is facilitated, the video of the monitoring site of the unmanned aerial vehicle is rapidly displayed, and the purpose of rapid detection is realized.
Under general conditions, video acquisition, format conversion and telemetering transmission all adopt a serial working mode, namely when the FPGA performs video acquisition, other work is in a pause state, the utilization rate of each link is not fully used, so that the delay of the whole system is increased, and the method is not suitable for acquiring data by the unmanned aerial vehicle flying at high speed.
Disclosure of Invention
The invention aims to overcome the defect of large delay of a video transmission system in a serial working mode in the prior art, and provides a low-delay video transmission system and a low-delay video transmission method.
In order to achieve the above purpose, the invention provides the following technical scheme:
a low-delay video transmission system comprises a camera, an FPGA, a main processor and an antenna; the output end of the camera is connected with the FPGA, the FPGA is bidirectionally connected with the main controller, and the FPGA is connected with the ground station through an antenna; the FPGA includes a video blocking module.
Preferably, the FPGA is configured to receive a video transmitted by the camera in a line mode, the video partitioning module is configured to perform cross cutting on the video, divide a frame of the video into a plurality of subframes, and perform subsequent processing on the transmitted data every time a frame of the video is acquired.
Preferably, the FPGA is Zynq-7000 series of Xinlinx.
Preferably, the host processor is an IMX6Q multi-core processor of the IMX family of Feichalcar.
A low-delay video transmission method comprises the following steps:
the camera collects video data and sends the video data to the FPGA; the FPGA packages and sends the sub-frames to a main processor by utilizing a video blocking technology every time the sub-frames with fixed length are collected; the plurality of sub-frames form a whole video frame;
and the main processor compresses the sub-frames, and then the video is transmitted to the upper computer through remote measurement.
Preferably, the format of the video data collected by the camera is 1080p, and 1280 lines of data are provided; the FPGA slices the entire frame of video into 40 sub-frames with a resolution of 1920 x 32 per sub-frame.
Preferably, the GOP value of the key frame interval when the main processor compresses the video is 30-40.
Compared with the prior art, the invention has the beneficial effects that: on the basis of a general H264 algorithm, the invention combines a video blocking technology with a method for increasing the interval of key frames in H264 to ensure that the serial work of the whole video transmission system is converted into parallel work, thereby quickly realizing low-delay video transmission.
Description of the drawings:
fig. 1 is a system block diagram of a low-latency video transmission system according to exemplary embodiment 1 of the present invention;
fig. 2 is a flowchart of a low-latency video transmission method according to an exemplary embodiment 2 of the present invention.
Detailed Description
The present invention will be described in further detail with reference to test examples and specific embodiments. It should be understood that the scope of the above-described subject matter is not limited to the following examples, and any techniques implemented based on the disclosure of the present invention are within the scope of the present invention.
Example 1
As shown in fig. 1, the present embodiment provides a low latency video transmission system, which includes a camera, an FPGA, a main processor, and an antenna, wherein an output end of the camera is connected to the FPGA, the FPGA is bidirectionally connected to a main controller, and the FPGA is connected to a ground station via the antenna; the FPGA includes a video blocking module.
The existing video transmission mode adopts a serial working mode, namely, a camera outputs a video according to frames, after the FPGA receives one frame of data, the video format is converted, and then the video is transmitted to a main processor; and the main processor compresses the video and then transmits the data to the FPGA to be transmitted to the ground station through the antenna, so that the video transmission is completed. In the existing video transmission mode, when a camera collects 1 frame of data, other modules do not work, and the work among a plurality of modules needs to be started after the previous module finishes processing, so that part of modules are in a busy state at some time and in an idle state at some time, the time distribution is unreasonable, and the delay time is long; according to the method and the device, the video of one frame is blocked through the FPGA to obtain a plurality of subframes, the data volume of single transmission is reduced, the subsequent processing module can receive data faster, the subsequent module can occupy the processing time of a task with short subsequent processing time when receiving the task with long processing time to process the task, and the processing processes of other modules are not influenced through the balance of sampling time and the like, so that the delay time is reduced. The divided subframes occupy small space, the front end can periodically and continuously output data video streams, each link at the rear end of the front end is in a busy state, and an efficient work assembly line is formed, namely, video acquisition, a video compression module, remote measurement output, upper computer decoding and display work simultaneously, and then parallel work is realized. The data volume of the sub-frame is small, the time for processing the rear end is short, when the FPGA sends the last video sub-frame, the rear end finishes the processing work of all the previous sub-frames, and the purpose of shortening the time delay is achieved.
The FPGA is used for receiving videos sent by the cameras in a line mode, the video partitioning module is used for transversely cutting the videos, one frame of video is divided into a plurality of subframes, and data are sent for subsequent processing after each frame of video is collected.
Wherein, the FPGA is Zynq-7000 series of Xinlinx; the main processor is an IMX6Q multi-core processor of the FSK family of IMXs. The FPGA is used for video acquisition, video format conversion, data transmission and other operations; the main processor IMX6Q is used for video compression and the like. The IMX6Q has the operating frequency reaching 1.2GHz, integrates a 2D and 3D graphics processor and a video coding and decoding module, wherein the video coding and decoding module supports multiple video coding and decoding standards of H263 and H264.
Example 2
The embodiment provides a low-delay video transmission method, which comprises the following steps:
the camera collects video data and sends the video data to the FPGA; the FPGA packages and sends the sub-frames to a main processor by utilizing a video blocking technology every time the sub-frames with fixed length are collected; the plurality of sub-frames form a whole video frame;
and the main processor compresses the sub-frames, and then the video is transmitted to the upper computer through remote measurement.
In this embodiment, a video whole frame is divided into a plurality of video subframes with fixed lengths, the FPGA encapsulates the subframes into BT1120 format and sends the BT1120 format to the back end for subsequent processing, each link of the system is in a working stage by a blocking technique, the data volume is reduced, the time processing time of each link in the system is short, and parallel work of the system is realized.
The format of video data collected by the camera is 1080p, and 1280 lines of data are provided; the FPGA slices the entire frame of video into 40 sub-frames with a resolution of 1920 x 32 per sub-frame. The processing time of each module is reasonably distributed, and the parallel work of the system is realized.
Wherein the GOP value of the key frame interval when the main processor compresses the video is 30 to 40. The key frame interval is adjusted to ensure the data volume, and the time consumed by the key frame for synchronization can be reduced, the GOP value is adaptive to the block size, the transmission efficiency is further improved, and the time delay is reduced.
Example 3
The present embodiment better shows the improvement in low latency by comparing the prior art with the solution described in embodiment 1 through a detailed example. The camera video format is exemplified by 1080P25 frames, which are connected to the FPGA through a CameraLink interface.
The specific work content and the consumed time of each link in the existing technical scheme of the serial mode are as follows:
video acquisition time: the camera with 1080P25 frames has the output interval of 40Ms every frame, adopts Cameralink interface Base mode, has the clock frequency of 30MHz, and consumes about 20Ms of time for sending one frame of data volume (1920 x 1080 x 2).
Video transcoding time: after receiving one frame of data through Cameralink, the FPGA needs to convert YUV422 data into BT1120, which consumes 2 ms.
Communication time between FPGA and IMX 6Q: the FPGA adopts 108M clock, 16-bit data bandwidth, and transmits BT1120 data to IMX6Q through a CSI interface (universal system interface) of IMX6Q, so that the consumption time is very small and is ignored.
IMX6Q acquisition, processing and compression: IMX6Q receives video data through GSTREAMER (which is an open source multimedia framework for constructing streaming media applications), GSTREAMER receives video source code data through a V4L2 driver, and the video source code needs to be converted into a format (data in YUV422 format needs to be converted into YUV420 format data) and then sent to a VPU for compression, and the above operation takes about 20 ms.
Telemetry transmission to ground station: the telemetry system adopts 4M bandwidth to transmit data to a ground station, and the biggest problem in the stage is that the video streams are different in length and different in frame type, the consumed time when an I frame is transmitted is 400ms at most, and the consumed time of a P frame and a B frame is about 10 ms.
And (3) video display: after the upper computer receives the H264 video stream, the data stream is decoded, and the decoded data is downloaded to the display program, the upper computer generally performs ffmpeg soft decoding, the time consumed by the ffmpeg soft decoding is about 50ms, and the display period of the display screen is 60hz, that is, the refresh rate of each picture is about 30 ms.
From the above description, the delay time of the entire video system can be calculated to be about 600 ms. Through the work content and the consumed time of each stage of the unmanned aerial vehicle video system, when the system is in a video acquisition stage, other stages do not work and are in an idle state, such as: the video compression module is idle, the telemetering output is idle, and the upper computer is idle for decoding and displaying, so that the resources of the whole system are greatly wasted.
In the technical solution described in embodiment 1, the specific work content and the consumed time of each link are as follows:
the video of the camera sends the video in a line mode; 1080p camera has 1280 row data, and FPGA transects 40 blocks of whole video, and the resolution of each block is 1920 x 32, and FPGA receives 32 row data each time, just loses data to next stage, and FPGA continues to gather other 32 rows of video data, and so on in proper order. With the blocking technique, the period of the video output from 20ms is shortened to 500 us. After the video volume is reduced, IMX6Q reduces the video processing and compression from 20ms to about 5ms, the data volume becomes smaller, the telemetry transmission is reduced at the same time, and the video decoding and display are greatly reduced. Through the blocking technology, each link of the system is in a working stage, the data volume is reduced, the time processing time of each link in the system is short, an efficient working assembly line is formed, namely, the video acquisition, the video compression module, the remote measurement output, the upper computer decoding and the display work simultaneously, and the parallel work of the system is realized. The technology can shorten the delay of the system from 600ms to 180ms or so.
And the system encodes to generate IDR frames, P frames and B frames. Where an IDR frame is a key frame, which is an I frame that encapsulates video coding information. The unmanned aerial vehicle data link channel receives the interference easily when the battlefield, and video data receives noise interference back, and ground station decoding module can't decode, needs to utilize the key frame to carry out the synchronization again, and the key frame is many, just can search for the I frame very fast when the system is synchronous, but can bring the increase of compressed data volume, leads to the delay time of video to improve, consequently, rationally solves the contradiction between key frame synchronization and the compression coding data, also can improve transmission efficiency. The number of I frames determines the processing time for subsequent encoded transmissions, and is related to the adjustment of the video block size. The method described in embodiment 1 adjusts parameters of the GOP, and appropriately reduces the interval between key frames on the premise of meeting the real-time requirement, so as to match the size of the video block, thereby achieving the optimal working state.
In this embodiment, through a large number of experiments, the optimal GOP parameter is selected. When the GOP is set to different parameters of 1, 5, 10, 20, 30, 50, the data amount size comparison before and after H264 compression encoding (unit M):
| before data compression | After data compression | GOP value size | Compression ratio |
| 290 | 16.5 | 1 | 17.58:1 |
| 290 | 8.0 | 5 | 36.25:1 |
| 290 | 7.0 | 10 | 41.42:1 |
| 290 | 6.7 | 20 | 44.62:1 |
| 290 | 6.4 | 30 | 45.31:1 |
| 290 | 6.1 | 50 | 47.54:1 |
If the value of GOP is set to be too large, I frames of the video sequence are fewer, the synchronization time of the decoding module is long, the real-time performance is reduced, so that the GOP cannot be set to be too large, the data size can be ensured, and the time consumed by the key frame for synchronization again can be reduced by adopting the GOP value of 30-40.
The foregoing is merely a detailed description of specific embodiments of the invention and is not intended to limit the invention. Various alterations, modifications and improvements will occur to those skilled in the art without departing from the spirit and scope of the invention.
Claims (7)
1. A low-delay video transmission system is characterized by comprising a camera, an FPGA, a main processor and an antenna; the output end of the camera is connected with the FPGA, the FPGA is bidirectionally connected with the main controller, and the FPGA is connected with the ground station through an antenna; the FPGA includes a video blocking module.
2. The low latency video transmission system according to claim 1, wherein the FPGA is configured to receive a video transmitted by the camera in a line manner, the video blocking module is configured to perform a cross cutting on the video, divide a frame of the video into a plurality of sub-frames, and perform a subsequent processing on the transmitted data after each acquisition of a frame of the video.
3. The low latency video transmission system according to claim 1, wherein the FPGA is the Zynq-7000 series of Xinlinx.
4. The low latency video transmission system of claim 1, wherein the main processor is an IMX6Q multi-core processor of the swiekal IMX family.
5. A low-latency video transmission method is characterized by comprising the following steps:
the camera collects video data and sends the video data to the FPGA; the FPGA packages and sends the sub-frames to a main processor by utilizing a video blocking technology every time the sub-frames with fixed length are collected; the plurality of sub-frames form a whole video frame;
and the main processor compresses the sub-frames, and then the video is transmitted to the upper computer through remote measurement.
6. The low-latency video transmission method according to claim 5, wherein the format of the video data collected by the camera is 1080p, and 1280 lines of data are available; the FPGA slices the entire frame of video into 40 sub-frames with a resolution of 1920 x 32 per sub-frame.
7. The method for transmitting the low-latency video according to claim 6, wherein a GOP value of a key inter-frame distance when the main processor compresses the video is 30-40.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011294245.1A CN112449138A (en) | 2020-11-18 | 2020-11-18 | Low-delay video transmission system and method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011294245.1A CN112449138A (en) | 2020-11-18 | 2020-11-18 | Low-delay video transmission system and method |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112449138A true CN112449138A (en) | 2021-03-05 |
Family
ID=74737458
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011294245.1A Pending CN112449138A (en) | 2020-11-18 | 2020-11-18 | Low-delay video transmission system and method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112449138A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113766235A (en) * | 2021-08-30 | 2021-12-07 | 聚好看科技股份有限公司 | Panoramic video transmission method and device |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101764797A (en) * | 2009-12-17 | 2010-06-30 | 哈尔滨侨航通信设备有限公司 | Time division multi-channel LAPD processor and designing method thereof |
| CN104902193A (en) * | 2015-05-19 | 2015-09-09 | 上海集成电路研发中心有限公司 | Method for performing segmentation processing and display for image data based on FPGA |
| CN108833932A (en) * | 2018-07-19 | 2018-11-16 | 湖南君瀚信息技术有限公司 | A kind of method and system for realizing the ultralow delay encoding and decoding of HD video and transmission |
| CN110996120A (en) * | 2019-12-13 | 2020-04-10 | 湖南君瀚信息技术有限公司 | Video stream transmitting and receiving method |
| CN111601078A (en) * | 2020-05-12 | 2020-08-28 | 西安创腾星泰电子科技有限公司 | Satellite-borne video compression system and method for video data direct transmission to ground |
| CN211791821U (en) * | 2020-05-12 | 2020-10-27 | 西安创腾星泰电子科技有限公司 | Satellite-borne video compression device for directly transmitting video data to ground |
-
2020
- 2020-11-18 CN CN202011294245.1A patent/CN112449138A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101764797A (en) * | 2009-12-17 | 2010-06-30 | 哈尔滨侨航通信设备有限公司 | Time division multi-channel LAPD processor and designing method thereof |
| CN104902193A (en) * | 2015-05-19 | 2015-09-09 | 上海集成电路研发中心有限公司 | Method for performing segmentation processing and display for image data based on FPGA |
| CN108833932A (en) * | 2018-07-19 | 2018-11-16 | 湖南君瀚信息技术有限公司 | A kind of method and system for realizing the ultralow delay encoding and decoding of HD video and transmission |
| CN110996120A (en) * | 2019-12-13 | 2020-04-10 | 湖南君瀚信息技术有限公司 | Video stream transmitting and receiving method |
| CN111601078A (en) * | 2020-05-12 | 2020-08-28 | 西安创腾星泰电子科技有限公司 | Satellite-borne video compression system and method for video data direct transmission to ground |
| CN211791821U (en) * | 2020-05-12 | 2020-10-27 | 西安创腾星泰电子科技有限公司 | Satellite-borne video compression device for directly transmitting video data to ground |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113766235A (en) * | 2021-08-30 | 2021-12-07 | 聚好看科技股份有限公司 | Panoramic video transmission method and device |
| CN113766235B (en) * | 2021-08-30 | 2023-10-17 | 聚好看科技股份有限公司 | Panoramic video transmission method and equipment |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN100423539C (en) | Image compression method, image compression device, image transmission system, data compression pre-processing device, and computer program | |
| US20200236323A1 (en) | Method and system for combining multiple area-of-interest video codestreams into a combined video codestream | |
| CN103037205B (en) | Video transmission method and system | |
| US20110032995A1 (en) | Video encoding and decoding device | |
| CN103327335B (en) | For the FPGA coded method of unmanned plane image transmission, system | |
| CN102404578A (en) | Multi-channel video transmitting system and method | |
| US20230007282A1 (en) | Image transmission method and apparatus | |
| CN105025347B (en) | A kind of method of sending and receiving of GOP images group | |
| CN110099280A (en) | A kind of video service quality Enhancement Method under wireless self-organization network Bandwidth-Constrained | |
| CN102905128A (en) | Code rate controlling method of coding and decoding processor in wireless video transmission process | |
| CN114902676A (en) | Reducing latency in wireless virtual and augmented reality systems | |
| CN207560231U (en) | A kind of LED display network controller | |
| CN102457727A (en) | Multi-channel transmission method of H.264 video frame for 3G network | |
| CN112449138A (en) | Low-delay video transmission system and method | |
| CN108924502A (en) | A kind of portable image transmission system and its figure transmission method | |
| CN106412384A (en) | Video processing system | |
| CN114339263A (en) | Lossless processing method for video data | |
| CN102123275B (en) | Video component data information acquiring and extracting method | |
| CN107360380B (en) | Double-code-rate compression camera device and upper computer decoding device thereof | |
| CN113747191A (en) | Video live broadcast method, system, equipment and storage medium based on unmanned aerial vehicle | |
| CN103428493A (en) | Light wireless transmission system for high-definition videos | |
| CN102547284A (en) | Portable wireless high-definition image transmission system and method thereof | |
| CN101686382B (en) | Video signal processing method and video signal system | |
| CN107241325A (en) | A kind of ultralow delay transport method and device of IP videos | |
| CN102857735A (en) | Real-time video synchronous automatic detection and recovery method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20210305 |