CN111541941A - Method for accelerating coding of multiple encoders at mobile terminal - Google Patents
Method for accelerating coding of multiple encoders at mobile terminal Download PDFInfo
- Publication number
- CN111541941A CN111541941A CN202010377900.3A CN202010377900A CN111541941A CN 111541941 A CN111541941 A CN 111541941A CN 202010377900 A CN202010377900 A CN 202010377900A CN 111541941 A CN111541941 A CN 111541941A
- Authority
- CN
- China
- Prior art keywords
- coding
- instance
- control module
- video
- encoding
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
- H04N19/146—Data rate or code amount at the encoder output
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
- H04N19/156—Availability of hardware or computational resources, e.g. encoding based on power-saving criteria
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/433—Content storage operation, e.g. storage operation in response to a pause request, caching operations
- H04N21/4331—Caching operations, e.g. of an advertisement for later insertion during playback
Abstract
The invention discloses a method for accelerating coding of multiple encoders at a mobile terminal. The method specifically comprises the following steps: inputting the decoded video frame into a buffer queue of an encoding control module, wherein the buffer queue limits the maximum length and controls the rate of video decoding; the coding control module starts a first coding example and distributes the input video frame to an input buffer of the coding example; if the coding control module detects that the input buffer of the current coding instance is full, the coding control module starts a new coding instance and increases the length of a buffer queue, and distributes a new video frame to the new coding instance; and outputting the compressed video frames obtained after coding and the corresponding timestamps to a compressed data queue, performing video encapsulation in an independent thread, and connecting the output of each coding instance according to a time sequence to obtain final output. The invention has the beneficial effects that: self-adaptation is realized, the occupancy rate of the coding resources is improved to the maximum extent, and the derivation speed is accelerated.
Description
Technical Field
The present invention relates to the field of video processing technologies, and in particular, to a method for accelerating encoding by multiple encoders at a mobile terminal.
Background
In the context of video editing, video derivation typically involves decoding, processing, and encoding. The time consumed by decoding, processing and encoding is different according to the complexity of video editing.
In a general scheme, each frame is decoded, processed (rendered) and encoded, and in this case, if the time consumption of video processing is long, one encoder instance can suffice; but if the video decoding/processing is time consuming, the encoding part becomes a bottleneck, in which case it is highly probable that the encoding resources are not full.
Generally, the time consumption of processing and encoding is relatively long, and especially in the case of pure transcoding, the bottleneck of video derivation is mainly in the encoding flow.
Disclosure of Invention
The invention provides a mobile terminal multi-encoder accelerated coding method for improving the occupancy rate of coding resources in order to overcome the defects in the prior art.
In order to achieve the purpose, the invention adopts the following technical scheme:
a method for accelerating coding of multiple encoders at a mobile terminal specifically comprises the following steps:
(1) inputting the decoded video frame into a buffer queue of an encoding control module, wherein the buffer queue limits the maximum length and controls the rate of video decoding;
(2) the coding control module starts a first coding example and distributes the input video frame to an input buffer of the coding example;
(3) if the coding control module detects that the input buffer of the current coding instance is full, namely the coding speed of the coding instance is lower than the decoding processing speed, the coding control module determines to start a new coding instance and increase the length of a buffer queue according to the current memory state of the mobile terminal and the hardware capability of the equipment, and allocates a new video frame to the new coding instance; if the coding control module detects that the input buffer of the current coding instance is not full, directly entering the next step;
(4) and outputting the compressed video frames obtained after coding and the corresponding timestamps to a compressed data queue, performing video encapsulation in an independent thread, and connecting the output of each coding instance according to a time sequence to obtain final output.
In the case of video processing and video encoding, if the video processing speed is higher than the video encoding speed, a new encoding instance can be automatically started; otherwise, if the video processing speed is lower than the video encoding speed, only one encoding instance is used (the encoder instance overhead is saved); the method has the main characteristic of self-adaption, and in a scene (mobile equipment) without determining the video processing performance and the coding performance, the occupancy rate of coding resources is improved to the greatest extent, so that the derivation speed is accelerated.
Preferably, in step (1), since the memory resource that can be occupied by a single process on the mobile terminal is limited, the length of the buffer queue cannot be infinitely long, and the length of the buffer queue needs to be determined according to the current memory state, the code size, and the gop length of the mobile terminal.
Preferably, in step (3), the encoding control module further acquires the encoding instance currently in the idle state, and allocates a new video frame to the encoding instance in the idle state.
Preferably, in step (4), when the video frames in the input buffer are consumed by the encoding instance, the encoding instance is in an idle state again, and the encoding control module can be reallocated next time.
Preferably, in step (4), the encoding control module continuously queries whether each encoding instance is in an idle state for a long time, and if it is found that any encoding instance is in an idle state for a long time, the encoding control module destroys the current encoding instance and reduces the length of the buffer queue.
Preferably, wherein the first frame of each coding instance must be a key frame and contain header information, if the encoder type of each coding instance is the same and the encoding parameters are also the same, ignoring keeping duplicate header information; the timestamp information corresponding to each frame of code stream is also correspondingly stored in the cache queue.
Preferably, in step (4), after all gops output by a certain encoding instance are written into the file, the encoding queue corresponding to the next gop is searched, and if the next encoding queue does not start outputting the code stream, the video encapsulation thread waits.
The invention has the beneficial effects that: self-adapting, in the scene (mobile equipment) without determining the video processing performance and the coding performance, the occupancy rate of the coding resources is improved to the greatest extent, and the derivation speed is further accelerated.
Drawings
FIG. 1 is a block flow diagram of the method of the present invention;
fig. 2 is a flow chart of the method of the present invention.
Detailed Description
The invention is further described with reference to the following figures and detailed description.
In the embodiment shown in fig. 1 and fig. 2, a method for accelerated coding of multiple encoders at a mobile terminal specifically includes the following steps:
(1) inputting the decoded video frame into a buffer queue of an encoding control module, wherein the buffer queue limits the maximum length and controls the rate of video decoding; because a single process on the mobile terminal can occupy limited memory resources, the length of the cache queue cannot be infinitely long, and the length of the cache queue needs to be determined according to the current memory state, the coding size and the gop length of the mobile terminal. Such as: 1920x1080 video is coded, the gop length is 30 frames, the buffer queue buffers RGB32 textures, and the memory occupied by one gop is about 250M.
(2) The coding control module starts a first coding example and distributes the input video frame to an input buffer of the coding example;
(3) if the coding control module detects that the input buffer of the current coding instance is full, namely the coding speed of the coding instance is lower than the decoding processing speed, the coding control module determines to start a new coding instance and increase the length of a buffer queue according to the current memory state of the mobile terminal and the hardware capability of the equipment, and allocates a new video frame to the new coding instance; if the coding control module detects that the input buffer of the current coding instance is not full, directly entering the next step; the encoding control module also obtains an encoding instance currently in an idle state and assigns a new video frame to the encoding instance in the idle state. Such as: 1920x1080 video is coded, the gop length is 30 frames, and when the free memory of the device is larger than 250M, the current buffer queue length can be increased and a new coding instance is enabled. If hardware coding is adopted, whether the GPU capability of the equipment can support the starting of the new coding instance or not needs to be comprehensively considered. If the hardware capability of the current equipment cannot support the new coding instance, the decoding thread enters waiting, and the decoding rate is reduced.
(4) And outputting the compressed video frames obtained after coding and the corresponding timestamps to a compressed data queue, performing video encapsulation in an independent thread, and connecting the output of each coding instance according to a time sequence to obtain final output. When the video frames in the input buffer are consumed by the coding instance, the coding instance is in an idle state again, and the coding control module can be redistributed next time. The coding control module can continuously inquire whether each coding instance is in an idle state for a long time, and if the coding instances are found to be in the idle state for a long time, the coding control module can destroy the current coding instance and reduce the length of the buffer queue at the same time. When one gop output by a certain coding instance is written into a file, searching a coding queue corresponding to the next gop, and if the next coding queue does not start to output a code stream, the video packaging thread waits.
Wherein the first frame of each coding instance must be a key frame and contain header information (sps/pps), ignoring keeping duplicate header information if the encoder type of each coding instance is the same and the encoding parameters are also the same; the timestamp information corresponding to each frame of code stream is also correspondingly stored in the cache queue.
Decoding, processing (rendering), encoding and IO are respectively carried out on different threads, and unidirectional data transmission (pipeline) is carried out among the threads through pipelines; since the decoding, processing, encoding may be handled by different modules of the device (e.g. hardware video acceleration module, GPU), this may avoid interaction between unrelated functions. In the case that the decoding or processing time is less than the encoding time, the number of frames to be encoded increases more than the input consumption of the encoder, and then a new encoder is started to encode the video in different time periods, for example, in a GOP. In case the decoding or processing takes more time than the encoding, according to the present solution there will be only one encoder and no new encoding instance will be created. I.e. in the case of video processing and video encoding, a new encoding instance can be started automatically if the video processing speed is higher than the video encoding speed; otherwise, if the video processing speed is lower than the video encoding speed, only one encoding instance is used (the encoder instance overhead is saved); the method has the main characteristic of self-adaption, and in a scene (mobile equipment) without determining the video processing performance and the coding performance, the occupancy rate of coding resources is improved to the greatest extent, so that the derivation speed is accelerated.
Claims (7)
1. A method for accelerating coding of multiple encoders at a mobile terminal is characterized by comprising the following steps:
(1) inputting the decoded video frame into a buffer queue of an encoding control module, wherein the buffer queue limits the maximum length and controls the rate of video decoding;
(2) the coding control module starts a first coding example and distributes the input video frame to an input buffer of the coding example;
(3) if the coding control module detects that the input buffer of the current coding instance is full, namely the coding speed of the coding instance is lower than the decoding processing speed, the coding control module determines to start a new coding instance and increase the length of a buffer queue according to the current memory state of the mobile terminal and the hardware capability of the equipment, and allocates a new video frame to the new coding instance; if the coding control module detects that the input buffer of the current coding instance is not full, directly entering the next step;
(4) and outputting the compressed video frames obtained after coding and the corresponding timestamps to a compressed data queue, performing video encapsulation in an independent thread, and connecting the output of each coding instance according to a time sequence to obtain final output.
2. The method as claimed in claim 1, wherein in step (1), the buffer queue length cannot be infinitely long due to limited memory resources that can be occupied by a single process on the mobile terminal, and the buffer queue length needs to be determined according to the current memory state, the code size, and the gop length of the mobile terminal.
3. The method as claimed in claim 1, wherein in step (3), the encoding control module further obtains the encoding instance currently in the idle state, and allocates a new video frame to the encoding instance in the idle state.
4. The method as claimed in claim 1 or 2, wherein in step (4), when the video frames in the input buffer are consumed by the coding instance, the encoding control module is in an idle state again, and the encoding control module is re-assignable next time.
5. The method as claimed in claim 4, wherein in step (4), the coding control module continuously queries whether each coding instance is in an idle state for a long time, and if it is found that any coding instance is in an idle state for a long time, the coding control module destroys the current coding instance and simultaneously reduces the length of the buffer queue.
6. The method of claim 1, wherein the first frame of each coding instance must be a key frame and contain header information, and if the type of encoder of each coding instance is the same and the encoding parameters are the same, the repeated header information is ignored; the timestamp information corresponding to each frame of code stream is also correspondingly stored in the cache queue.
7. The method as claimed in claim 2, wherein in step (4), after a gop output from a certain coding instance is written into the file, the coding queue corresponding to the next gop is searched, and if the next coding queue has not started to output the code stream, the video encapsulation thread waits.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010377900.3A CN111541941B (en) | 2020-05-07 | 2020-05-07 | Method for accelerating coding of multiple encoders at mobile terminal |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010377900.3A CN111541941B (en) | 2020-05-07 | 2020-05-07 | Method for accelerating coding of multiple encoders at mobile terminal |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111541941A true CN111541941A (en) | 2020-08-14 |
| CN111541941B CN111541941B (en) | 2021-10-29 |
Family
ID=71975339
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010377900.3A Active CN111541941B (en) | 2020-05-07 | 2020-05-07 | Method for accelerating coding of multiple encoders at mobile terminal |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111541941B (en) |
Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20060233516A1 (en) * | 2005-03-28 | 2006-10-19 | Pioneer Corporation | Encoded video conversion apparatus, conversion method and program product |
| CN1929605A (en) * | 2006-08-07 | 2007-03-14 | 爱普拉斯通信技术（北京）有限公司 | Device and method for flow control of video image data |
| EP2426940A1 (en) * | 2003-09-15 | 2012-03-07 | The DirecTV Group, Inc. | Method and system for adaptive transcoding and transrating in a video network |
| CN103716644A (en) * | 2013-12-05 | 2014-04-09 | 南京肯麦思智能技术有限公司 | H264 multi-granularity parallel handling method |
| CN104768026A (en) * | 2015-04-17 | 2015-07-08 | 中国工商银行股份有限公司 | Multi-channel audio and video transcoding device |
| CN105340215A (en) * | 2013-04-17 | 2016-02-17 | 佩克普股份公司 | Dynamic allocation of encoders |
| CN105847790A (en) * | 2015-01-16 | 2016-08-10 | 杭州海康威视数字技术股份有限公司 | Code stream transmission method and device |
| US20170094294A1 (en) * | 2015-09-28 | 2017-03-30 | Cybrook Inc. | Video encoding and decoding with back channel message management |
| WO2017127300A1 (en) * | 2016-01-20 | 2017-07-27 | Arris Enterprises Llc | Multistage panic rate control scheme for encoders |
| CN109429070A (en) * | 2017-08-23 | 2019-03-05 | 杭州海康威视系统技术有限公司 | A kind of mobile terminal video coding method, device and mobile terminal |
| CN110213598A (en) * | 2018-05-31 | 2019-09-06 | 腾讯科技（深圳）有限公司 | A kind of video code conversion system, method and Related product |
| CN110677715A (en) * | 2019-10-11 | 2020-01-10 | 北京达佳互联信息技术有限公司 | Decoding method, decoder, electronic device and storage medium |
-
2020
- 2020-05-07 CN CN202010377900.3A patent/CN111541941B/en active Active
Patent Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP2426940A1 (en) * | 2003-09-15 | 2012-03-07 | The DirecTV Group, Inc. | Method and system for adaptive transcoding and transrating in a video network |
| US20060233516A1 (en) * | 2005-03-28 | 2006-10-19 | Pioneer Corporation | Encoded video conversion apparatus, conversion method and program product |
| CN1929605A (en) * | 2006-08-07 | 2007-03-14 | 爱普拉斯通信技术（北京）有限公司 | Device and method for flow control of video image data |
| CN105340215A (en) * | 2013-04-17 | 2016-02-17 | 佩克普股份公司 | Dynamic allocation of encoders |
| CN103716644A (en) * | 2013-12-05 | 2014-04-09 | 南京肯麦思智能技术有限公司 | H264 multi-granularity parallel handling method |
| CN105847790A (en) * | 2015-01-16 | 2016-08-10 | 杭州海康威视数字技术股份有限公司 | Code stream transmission method and device |
| CN104768026A (en) * | 2015-04-17 | 2015-07-08 | 中国工商银行股份有限公司 | Multi-channel audio and video transcoding device |
| US20170094294A1 (en) * | 2015-09-28 | 2017-03-30 | Cybrook Inc. | Video encoding and decoding with back channel message management |
| WO2017127300A1 (en) * | 2016-01-20 | 2017-07-27 | Arris Enterprises Llc | Multistage panic rate control scheme for encoders |
| CN109429070A (en) * | 2017-08-23 | 2019-03-05 | 杭州海康威视系统技术有限公司 | A kind of mobile terminal video coding method, device and mobile terminal |
| CN110213598A (en) * | 2018-05-31 | 2019-09-06 | 腾讯科技（深圳）有限公司 | A kind of video code conversion system, method and Related product |
| CN110677715A (en) * | 2019-10-11 | 2020-01-10 | 北京达佳互联信息技术有限公司 | Decoding method, decoder, electronic device and storage medium |
Non-Patent Citations (2)
| Title |
|---|
| M. GARCIA-PINEDA ET AL.: "Cloud-based eHealth video encoding system for real time thermographic streaming: Performance evaluation", 《2017 IEEE INTERNATIONAL CONFERENCE ON COMMUNICATIONS (ICC)》 * |
| 张旭升: "分布式视频增强转码系统设计与优化", 《中国优秀博硕士学位论文全文数据库(硕士)信息科技辑(月刊)，2020年第01期》 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111541941B (en) | 2021-10-29 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US20110001643A1 (en) | System and method for providing high throughput entropy coding using syntax element partitioning | |
| CN110430441B (en) | Cloud mobile phone video acquisition method, system, device and storage medium | |
| US20110032995A1 (en) | Video encoding and decoding device | |
| US10652563B2 (en) | Parallel parsing in a video decoder | |
| US8345774B2 (en) | Hypothetical reference decoder | |
| JPH0865687A (en) | Expansion system of compression video data | |
| CN110830805A (en) | Multi-resolution output distributed file transcoding method and device | |
| US20190356911A1 (en) | Region-based processing of predicted pixels | |
| US9020284B2 (en) | Image encoding apparatus | |
| CN114222166B (en) | Multi-channel video code stream real-time processing and on-screen playing method and related system | |
| CN112087631A (en) | Synchronous parallel video coding and decoding and streaming media transmission system and method based on GPU | |
| CN111541941B (en) | Method for accelerating coding of multiple encoders at mobile terminal | |
| CN105338371A (en) | Multimedia transcoding scheduling method and apparatus | |
| CN112218082B (en) | Reconfigurable multi-video coding acceleration design-based method and system | |
| CN115297331A (en) | Video data processing method, device, equipment and system | |
| CN110868610B (en) | Streaming media transmission method, device, server and storage medium | |
| CN112543374A (en) | Transcoding control method and device and electronic equipment | |
| US11671976B2 (en) | Early notification for transmission of encoded video data | |
| CN108924471A (en) | A kind of conference system based on QSV fast video encoding and decoding | |
| CN110677664B (en) | Low-jitter code rate control method for 4K/8K ultra-high-definition video coding | |
| CN113055680B (en) | Distributed transcoding method | |
| CN115942000B (en) | H.264 format video stream transcoding method, device, equipment and medium | |
| US20220012004A1 (en) | Managing display data | |
| US20240146975A1 (en) | Dynamic queuing of entropy-coded data for transmission in a bitstream | |
| US20130287100A1 (en) | Mechanism for facilitating cost-efficient and low-latency encoding of video streams |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information | ||
| CB02 | Change of applicant information |
Address after: 22nd floor, block a, Huaxing Times Square, 478 Wensan Road, Xihu District, Hangzhou, Zhejiang 310000 Applicant after: Hangzhou Xiaoying Innovation Technology Co.,Ltd. Address before: 16 / F, HANGGANG Metallurgical Science and technology building, 294 Tianmushan Road, Xihu District, Hangzhou City, Zhejiang Province, 310012 Applicant before: HANGZHOU QUWEI SCIENCE & TECHNOLOGY Co.,Ltd. |
|
| GR01 | Patent grant | ||
| GR01 | Patent grant |