CN108111859B - H.264 video coding and decoding method based on Jetson TX1 platform - Google Patents
H.264 video coding and decoding method based on Jetson TX1 platform Download PDFInfo
- Publication number
- CN108111859B CN108111859B CN201711497612.6A CN201711497612A CN108111859B CN 108111859 B CN108111859 B CN 108111859B CN 201711497612 A CN201711497612 A CN 201711497612A CN 108111859 B CN108111859 B CN 108111859B
- Authority
- CN
- China
- Prior art keywords
- video
- stream
- code stream
- data
- decoding
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/42—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Compression Or Coding Systems Of Tv Signals (AREA)
Abstract
The invention relates to an H.264 video coding and decoding method and a device based on a Jetson TX1 platform, comprising the following steps: step S1, preprocessing the video stream collected by the video; step S2, carrying out H.264 coding compression on the preprocessed video data to generate a compressed code stream, and sending the compressed code stream through a network after packaging; and step S3, receiving the compressed code stream, decoding and converting the format of the code stream, and displaying the collected video in the corresponding display unit. The invention overcomes the defect that GPU acceleration coding and decoding can not be carried out on a Jetson TX1 platform; the real-time processing task of the video is realized, the real-time performance of video transmission and storage is greatly improved, the delay time of video transmission is reduced, and the delay time of a video data stream video transmission system with video resolutions of 640x480, 1280x720 and 1920x1080 can be controlled within 200ms for 30 frames per second.
Description
Technical Field
The invention relates to the technical field of image processing, in particular to an H.264 video coding and decoding method and device based on a Jetson TX1 platform.
Background
Jetson TX1 is an embedded GPU development platform promoted by NVIDIA, has an advanced embedded operating system, ultrahigh computing performance and GPU acceleration technology although only the size of a credit card is large, and is the most ideal embedded solution for intelligent unmanned aerial vehicles and robots.
When the visual processing task is implemented on the JetsonTX1 platform, the following disadvantages exist:
1) due to the large amount of video data and the limitation of transmission bandwidth, the original image directly transmitted cannot meet the real-time requirement, the transmission delay is large, and the frame loss is serious;
2) the two classic development libraries of X264 and FFMPEG of Jetson TX1 are not designed perfectly aiming at the acceleration of coding and decoding of the GPU, firstly, the two classic development libraries do not support the bottom hardware drive of the GPU, and secondly, a program developed based on the classic development libraries is difficult to meet the real-time requirement.
The h.264 standard is a new generation digital Video coding standard organized by JVT (Joint Video Team), aiming at formulating a new Video coding standard to achieve high compression ratio, high image quality, and good network adaptability of Video. H.264 realizes higher coding efficiency, can be applied to different occasions, can use different transmission and playing rates according to different environments, provides rich error processing tools, and can well control or eliminate packet loss and error codes.
Therefore, how to overcome the disadvantage of the JetsonTX1 platform is a problem to be solved urgently in order to realize h.264 video encoding and decoding on the JetsonTX1 platform.
Disclosure of Invention
In view of the above analysis, the present invention aims to provide an h.264 video codec design method based on the JetsonTX1 platform, which implements h.264 video codec on the JetsonTX1 platform, and implements video signal transmission with high compression ratio, high image quality, and good network adaptability.
The purpose of the invention is mainly realized by the following technical scheme:
an H.264 video coding and decoding method based on a Jetson TX1 platform comprises the following steps:
step S1, acquiring input video stream through video acquisition, and preprocessing the acquired video stream;
step S2, carrying out H.264 coding compression on the preprocessed video data at the sending end to generate a compressed code stream, and sending the video data through the network after packaging;
and step S3, the receiving end receives the compressed code stream through the network, and displays the collected video in the corresponding display unit after decoding and format conversion are carried out on the code stream.
Further, step 1 comprises the following substeps:
s101, acquiring input video stream data and providing a video source for video coding;
step S102, converting the video format of the acquired video stream data into a YUV format;
and step S103, automatically adjusting the code rate of the video stream according to the resolution of the image in the YUV format video stream data.
Further, the frame rate of the input video stream is 30 frames per second, and the video resolution includes 640x480, 1280x720 and 1920x 1080;
setting the code rate to be 3-3.5M for an input video stream with the resolution of 1280x 720;
setting the code rate of an input video stream with the resolution of 1920x1080 as 8-9M;
the code rate of an input video stream with the resolution of 640x480 is set to be 1.5-2M.
Further, step 2 comprises the following substeps:
step S201, carrying out H.264 coding on the preprocessed video code stream;
step S202, real-time transport protocol RTP packing is carried out on the H.264 code stream;
and step S203, sending the packed H.264 compressed code stream through the network.
Further, the packing is carried out, and image information including image frame number, bit width and channel number of input video stream data corresponding to the code stream data is added into each H.264 code stream data packet;
the buffer sent by the network is set to be less than or equal to 2097152 bytes.
Further, step 3 comprises the following substeps:
s301, receiving a compressed code stream at a network receiving end for decoding;
step S302, converting the format of the decoded video frame data into a format which can be directly used for video display;
and step S303, smoothly displaying the acquired video by calling a GPU image acceleration module of the embedded Jetson TX1 platform.
Furthermore, the display memory configured by the GPU image acceleration module is configured to be 6-8G, and the number of GPU acceleration cores is set to be the maximum number of cores.
An H.264 video coding and decoding device based on a Jetson TX1 platform comprises a video coding board and a video decoding board;
the video coding board receives a video stream transmitted by the camera and records image information including image resolution, image frame number, bit width and channel number in the video stream; after the video stream is correspondingly preprocessed and H.264 video coded, an output code stream is formed and sent out through the Ethernet;
the video decoding board receives H.264 compressed code stream data sent by the video coding board through a network, decodes the data, converts the format of the data, outputs a video signal, and realizes smooth display of the acquired video by adopting video acceleration.
Further, the video coding board comprises a preprocessing module, an H.264 video coding module and a network sending module;
the preprocessing module converts the video format of the acquired video stream data into a YUV format; automatically adjusting the code rate of the video stream according to the resolution of the image in the video stream data in the YUV format;
the H.264 video coding module carries out H.264 coding compression and real-time transport protocol RTP packing on the preprocessed video data;
the network sending module sends the packed H.264 compressed code stream through the Ethernet, and the network sending module corresponds to the network receiving module and sends the packed H.264 compressed code stream through a user datagram UDP protocol.
Further, the video decoding board comprises a network receiving module, an H.264 video decoding module and a display module;
the network receiving module receives H.264 compressed code stream data sent by the video coding board, corresponds to the network sending module and receives the data through a User Datagram Protocol (UDP);
the H.264 video decoding module carries out real-time transport protocol (RTP) unpacking and H.264 coding and decoding on the received compressed code stream data;
the display module converts the decoded video code stream into a format which can be directly used for video display according to a display format supported by the connected display equipment, and the GPU image acceleration module of the embedded Jetson TX1 platform is called to accelerate display, so that the collected video is smoothly displayed.
The invention has the following beneficial effects: the invention realizes GPU acceleration coding and decoding on a Jetson TX1 platform; setting corresponding code rates for input videos with different resolutions, so that the transmission of the videos meets the transmission bandwidth and transmission rate of a Jetson TX1 platform and also meets the requirement on the definition of the videos; the real-time processing of the video is realized, the real-time performance of video transmission and storage is greatly improved, the delay time of video transmission is reduced, and the delay time of a video data stream video transmission system with video resolutions of 640x480, 1280x720 and 1920x1080 can be controlled within 200ms for 30 frames per second.
Drawings
The drawings are only for purposes of illustrating particular embodiments and are not to be construed as limiting the invention, wherein like reference numerals are used to designate like parts throughout.
FIG. 1 is a flow chart of a method for encoding and decoding H.264 video;
fig. 2 is a block diagram of an h.264 video codec device.
Detailed Description
The preferred embodiments of the present invention will now be described in detail with reference to the accompanying drawings, which form a part hereof, and which together with the embodiments of the invention serve to explain the principles of the invention.
The invention discloses a specific embodiment of an H.264 video coding and decoding method based on a Jetson TX1 platform; as shown in fig. 1, the method comprises the following steps:
step S1, acquiring input video stream through video acquisition, and preprocessing the acquired video stream;
s101, acquiring input video stream data and providing a video source for video coding;
the video stream is acquired through video acquisition, the frame rate of the video stream is 30 frames per second, the video resolution comprises 640x480, 1280x720 and 1920x1080, and no special requirement is imposed on the video format; when the video is collected, the image information including the image resolution, the image frame number, the bit width and the channel number in the video stream is recorded.
Step S102, converting the video format of the acquired video stream data into a YUV format;
the format of the input data required by the H.264 video coding adopted by the embodiment is a YUV format; converting the format of input video stream data into a YUV format before encoding; and converting the video format of the acquired video stream data into a YUV format by calling format factory software.
Step S103, automatically adjusting the code rate of the video stream according to the resolution of the image in the YUV format video stream data;
setting the code rate according to the requirements of video transmission bandwidth, transmission rate and video definition;
the higher the code rate is, the higher the corresponding compression rate is, the smaller the compressed file is, the faster the transmission is, but the poorer the definition of the final video display result is, the more fuzzy the display is;
the lower the code rate is, the lower the corresponding compression rate is, the larger the compressed file is, the slower the transmission is, but the better the definition of the final video display result is, the clearer the display is;
in this embodiment, by combining the data transmission bandwidth of the JetsonTX1 platform and the requirement on the definition of a video, the code rate of an input video with the resolution of 1280x720 is set to 3-3.5M;
the method comprises the steps that for an input video with a resolution of 1920x1080, a code rate is set to be 8-9M;
the code rate of an input video with the resolution of 640x480 is set to be 1.5-2M.
Step S2, carrying out H.264 coding compression on the preprocessed video data at the sending end to generate a compressed code stream, and sending the video data through the network after packaging;
step S201, carrying out H.264 coding on the preprocessed video code stream;
in this embodiment, h.264 encoding is implemented by invoking an h.264 encoder in a Gstreamer, and h.264 compressed code stream information including a video sequence parameter set and an image parameter set is generated.
Step S202, real-time transport protocol RTP packing is carried out on the H.264 code stream;
when packing, adding image information including image frame number, bit width and channel number of input video stream data corresponding to the stream data into each H.264 stream data packet.
Step S203, sending the packed H.264 compressed code stream through the network;
the sending carries out network transmission by setting a user datagram UDP protocol, and network transmission parameters needing to be set comprise an IP address, a port number and the size of a sending buffer area, wherein the IP address is a fixed value and cannot be changed after being manually set; the port number is set arbitrarily in 1024-65535; the size of a sending buffer is selected from 65536-1073741824 bytes;
in this embodiment, according to the memory size of the GPU JetsonTX1 platform, the send buffer for network transmission is set to be 2097152 bytes or less.
Step S3, the receiving end receives the compressed code stream through the network, and after decoding and format conversion are carried out on the code stream, the collected video is displayed in the corresponding display unit;
s301, receiving a compressed code stream at a network receiving end for decoding;
receiving the H.264 compressed code stream sent by the sending end at the receiving end through a user datagram UDP protocol, then decoding the sent H.264 compressed code stream data by using a real-time transport protocol RTP, and obtaining the following data after decoding:
code stream information including video sequence parameter set and image parameter set for subsequent video format conversion;
image information including image frame number, bit width and channel number for subsequent video image display.
Step S302, converting the format of the decoded video frame data into a format which can be directly used for video display;
and calling a format conversion instruction carried by the embedded platform GPU Jetson TX1 in a receiving end to perform real-time data conversion, and converting the data into a displayable video format, such as AVI and MP4 formats.
Step S303, smoothly displaying the acquired video by calling a GPU image acceleration module of an embedded Jetson TX1 platform;
the aim of acceleration is achieved by setting parameters of the GPU image acceleration module including GPU display memory setting and the number of GPU acceleration cores; the display memory configuration is 6-8G, and the number of the GPU acceleration cores is set to be the maximum number of the cores.
An h.264 video coding and decoding device based on the Jetson TX1 platform is shown in fig. 2, and comprises a video coding board and a video decoding board;
the video coding board receives a video stream with a certain format transmitted by a camera and records image information including image resolution, image frame number, bit width and channel number in the video stream; after the video stream is correspondingly preprocessed and H.264 video coded, an output code stream is formed and sent out through the Ethernet at a certain code rate;
the format of the video stream is: the frame rate is 30 frames per second; video resolutions include 640x480, 1280x720, and 1920x 1080;
the video coding board comprises a preprocessing module, an H.264 video coding module and a network sending module;
the preprocessing module converts the video format of the acquired video stream data into a YUV format; automatically adjusting the code rate of the video stream according to the resolution of the image in the video stream data in the YUV format;
the H.264 video coding module carries out H.264 coding compression and real-time transport protocol RTP packing on the preprocessed video data;
the network sending module sends the packed H.264 compressed code stream through the Ethernet, and the network sending module corresponds to the network receiving module and can send the compressed code stream in a wireless mode or a wired mode through a user datagram UDP protocol.
The video decoding board receives H.264 compressed code stream data sent by the video coding board through a network, decodes the data, converts the format of the data and outputs a video signal, and a video acceleration technology is adopted to realize smooth display of the acquired video;
the video decoding board comprises a network receiving module, an H.264 video decoding module and a display module.
The network receiving module receives H.264 compressed code stream data sent by the video coding board, corresponds to the network sending module, and can receive the data in a wireless mode or a wired mode through a User Datagram Protocol (UDP);
the H.264 video decoding module carries out real-time transport protocol (RTP) unpacking and H.264 coding and decoding on the received compressed code stream data;
the display module converts the decoded video code stream into a format which can be directly used for video display, such as AVI and MP4 format, according to the display format supported by the connected display equipment;
the conversion is realized by calling a self-contained format conversion instruction on an embedded platform GPU Jetson TX 1;
the display module carries out display acceleration by calling a GPU image acceleration module of an embedded Jetson TX1 platform, so that the collected video is smoothly displayed;
the display memory is configured to be 6-8G in the display acceleration, and the number of GPU acceleration cores is set as the maximum number of cores.
In summary, the h.264 video coding and decoding method and device based on the JetsonTX1 platform according to the embodiments of the present invention implement GPU accelerated coding and decoding on the JetsonTX1 platform; setting corresponding code rates for input videos with different resolutions, so that the transmission of the videos meets the transmission bandwidth and transmission rate of a Jetson TX1 platform and also meets the requirement on the definition of the videos; the real-time processing of the video is realized, the real-time performance of video transmission and storage is greatly improved, the delay time of video transmission is reduced, and the delay time of a video data stream video transmission system with video resolutions of 640x480, 1280x720 and 1920x1080 can be controlled within 200ms for 30 frames per second.
Those skilled in the art will appreciate that all or part of the flow of the method implementing the above embodiments may be implemented by a computer program, which is stored in a computer readable storage medium, to instruct related hardware. The computer readable storage medium is a magnetic disk, an optical disk, a read-only memory or a random access memory.
The above description is only for the preferred embodiment of the present invention, but the scope of the present invention is not limited thereto, and any changes or substitutions that can be easily conceived by those skilled in the art within the technical scope of the present invention are included in the scope of the present invention.
Claims (1)
1. An H.264 video coding and decoding method based on a Jetson TX1 platform is characterized by comprising the following steps:
step S1, acquiring input video stream through video acquisition, and preprocessing the acquired video stream;
step S2, carrying out H.264 coding compression on the preprocessed video data at the sending end to generate a compressed code stream, and sending the video data through the network after packaging;
step S3, the receiving end receives the compressed code stream through the network, and after decoding and format conversion are carried out on the code stream, the collected video is displayed in the corresponding display unit;
step S1 includes the following substeps:
s101, acquiring input video stream data and providing a video source for video coding;
step S102, converting the video format of the acquired video stream data into a YUV format;
step S103, automatically adjusting the code rate of the video stream according to the resolution of the image in the YUV format video stream data;
the frame rate of the input video stream is 30 frames per second, and the video resolution comprises 640x480, 1280x720 and 1920x 1080;
setting the code rate to be 3-3.5M for an input video stream with the resolution of 1280x 720;
setting the code rate of an input video stream with the resolution of 1920x1080 as 8-9M;
setting the code rate of an input video stream with the resolution of 640x480 to be 1.5-2M;
step S2 includes the following substeps:
step S201, carrying out H.264 coding on the preprocessed video code stream;
step S202, real-time transport protocol RTP packing is carried out on the H.264 code stream;
step S203, sending the packed H.264 compressed code stream through the network;
the packaging is carried out, and input video stream data corresponding to the code stream data is added into each H.264 code stream data packet; the input video stream data comprises image information including image frame number, bit width and channel number;
the sending buffer zone set by the network is less than or equal to 2097152 bytes;
step S3 includes the following substeps:
s301, receiving a compressed code stream at a network receiving end for decoding;
step S302, converting the format of the decoded video frame data into a format which can be directly used for video display;
receiving the H.264 compressed code stream sent by the sending end at the receiving end through a user datagram UDP protocol, then decoding the sent H.264 compressed code stream data by using a real-time transport protocol RTP, and obtaining the following data after decoding:
code stream information including video sequence parameter set and image parameter set for subsequent video format conversion;
image information including image frame number, bit width and channel number for subsequent video image display;
step S303, smoothly displaying the acquired video by calling a GPU image acceleration module of an embedded Jetson TX1 platform;
the display memory configuration of the GPU image acceleration module is 6-8G, and the number of the GPU acceleration cores is set to be the maximum number of the cores.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711497612.6A CN108111859B (en) | 2017-12-29 | 2017-12-29 | H.264 video coding and decoding method based on Jetson TX1 platform |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711497612.6A CN108111859B (en) | 2017-12-29 | 2017-12-29 | H.264 video coding and decoding method based on Jetson TX1 platform |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108111859A CN108111859A (en) | 2018-06-01 |
| CN108111859B true CN108111859B (en) | 2020-10-09 |
Family
ID=62215535
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201711497612.6A Active CN108111859B (en) | 2017-12-29 | 2017-12-29 | H.264 video coding and decoding method based on Jetson TX1 platform |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108111859B (en) |
Families Citing this family (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109274973A (en) * | 2018-09-26 | 2019-01-25 | 江苏航天大为科技股份有限公司 | Fast video coding/decoding method on embedded-type ARM platform |
| CN109803144B (en) * | 2018-12-25 | 2021-05-18 | 北京凯视达科技股份有限公司 | Video encoding and decoding method, device and system and electronic equipment |
| CN111508038A (en) * | 2020-04-17 | 2020-08-07 | 北京百度网讯科技有限公司 | Image processing method, image processing device, electronic equipment and computer readable storage medium |
| CN112468819A (en) * | 2020-11-27 | 2021-03-09 | 武汉高德红外股份有限公司 | Video image plug flow system and equipment |
| CN114882612A (en) * | 2022-05-27 | 2022-08-09 | 六安智梭无人车科技有限公司 | Unmanned vehicle monitoring system and method |
| CN115695856B (en) * | 2022-11-01 | 2023-05-09 | 北京小鸟科技股份有限公司 | Video multicast code stream fast switching device, method, electronic device and storage medium |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103957389A (en) * | 2014-05-13 | 2014-07-30 | 重庆大学 | 3G video transmission method and system based on compression sensing |
| DE102013001814A1 (en) * | 2013-02-04 | 2014-08-07 | Silicon Software Gmbh | Embedded system for use in video distribution system for processing video in betting shop, has hardware unit that is designed so as to compress and decompress continuous video data stream with specific frames per second |
| CN105072506A (en) * | 2015-08-20 | 2015-11-18 | 湖南亿谷科技发展股份有限公司 | Method and system for transmitting teaching live video |
-
2017
- 2017-12-29 CN CN201711497612.6A patent/CN108111859B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| DE102013001814A1 (en) * | 2013-02-04 | 2014-08-07 | Silicon Software Gmbh | Embedded system for use in video distribution system for processing video in betting shop, has hardware unit that is designed so as to compress and decompress continuous video data stream with specific frames per second |
| CN103957389A (en) * | 2014-05-13 | 2014-07-30 | 重庆大学 | 3G video transmission method and system based on compression sensing |
| CN105072506A (en) * | 2015-08-20 | 2015-11-18 | 湖南亿谷科技发展股份有限公司 | Method and system for transmitting teaching live video |
Non-Patent Citations (2)
| Title |
|---|
| A real-time focused SAR algorithm on the Jetson TK1 board;K.Radecki,et al.;《Image and Signal Processing for Remote Sensing XXII》;20161018;第10004卷;全文 * |
| 基于RTP的H.264视频流实时打包传输的研究;李校林 等;《计算机工程与科学》;20120531;第34卷(第5期);168-171 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN108111859A (en) | 2018-06-01 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108111859B (en) | H.264 video coding and decoding method based on Jetson TX1 platform | |
| US11871014B2 (en) | Method for signaling a step-wise temporal sub-layer access sample | |
| US9532099B2 (en) | Distributed media stream synchronization control | |
| CN109074161B (en) | Hybrid graphics and pixel domain architecture for 360 degree video | |
| US20090322784A1 (en) | System and method for virtual 3d graphics acceleration and streaming multiple different video streams | |
| US10454986B2 (en) | Video synchronous playback method, apparatus, and system | |
| CN111339015B (en) | Control instruction transmission method, system and equipment | |
| KR101667970B1 (en) | Server device, terminal, thin client system, screen transmission method and program | |
| CN108200482B (en) | Cross-platform high-definition audio and video playing method, system and client | |
| CN112601096B (en) | Video decoding method, device, equipment and readable storage medium | |
| EP2866140A1 (en) | System and method for forwarding an application user interface | |
| JPWO2011142311A1 (en) | Remote mobile communication system, server device, and remote mobile communication system control method | |
| CN110891084A (en) | Thin client remote desktop control system based on autonomous HVDP protocol | |
| CN104185082A (en) | Audio/video live broadcast method and system, client and server | |
| EP2804143A1 (en) | System and method for forwarding a graphics command stream | |
| CN114339382A (en) | Video playing method, device, equipment and computer storage medium | |
| WO2018072337A1 (en) | Wireless transmission method, device, terminal and head-mounted display device for virtual reality | |
| CN111885346A (en) | Picture code stream synthesis method, terminal, electronic device and storage medium | |
| CN112788431A (en) | Video playing method, device, system, medium and browser based on HTML5 | |
| WO2013030166A2 (en) | Method for transmitting video signals from an application on a server over an ip network to a client device | |
| CN111263164A (en) | High frame frequency video parallel coding and recombination method | |
| CN108259993B (en) | Method for projecting large screen on wall based on network combined video stream desktop and display system thereof | |
| EP2312859A2 (en) | Method and system for communicating 3D video via a wireless communication link | |
| EP2851797A2 (en) | System and method for forwarding an application user interface | |
| US9838463B2 (en) | System and method for encoding control commands |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |