WO2017096947A1

WO2017096947A1 - Real-time control method and device for real-time transcoding

Info

Publication number: WO2017096947A1
Application number: PCT/CN2016/096002
Authority: WO
Inventors: 白茂生
Original assignee: 乐视控股（北京）有限公司; 乐视云计算有限公司
Priority date: 2015-12-07
Filing date: 2016-08-19
Publication date: 2017-06-15
Also published as: CN105898371A

Abstract

Provided are a real-time control method and device for real-time transcoding. In a transcoding process, feedback information about current encoding is acquired in real time; according to the feedback information, actual consumed time of a certain number of current encoding frames is calculated; according to pre-calculated expected consumed time and the actual consumed time, a time-consuming deviation of the certain number of encoding frames is calculated; and according to the time-consuming deviation, an encoding parameter is adjusted to carry out transcoding control. By means of the present invention, in the case where transcoding resources are limited, transcoding with smooth live experience can still be provided.

Description

Real-time control method and device for real-time transcoding

The present application claims priority to Chinese Patent Application No. 201510891571.3, filed on Dec. 7, 2015, and entitled "Real-time Control Method and Apparatus for Real-Time Transcoding", the entire contents of which are incorporated herein by reference. In the application.

Technical field

The present application relates to the field of video technologies, and in particular, to a real-time control method and apparatus for real-time transcoding.

Background technique

In recent years, the development of broadcasting and IP network video applications, the HD video era has officially arrived. For digital devices, the quality of video images is greatly improved, which also causes inconvenience to video content processing.

With the increasing popularity and application of cloud computing and virtualization technologies, real-time transcoding in the cloud has become a popular method for Internet video sites. Real-time transcoding is a kind of video transcoding technology that has been developed rapidly and gradually applied in recent years, and its function is consistent with offline transcoding.

The real-time transcoding consumes a huge amount of CPU and memory. How to provide a smooth live broadcast transcoding in the case of limited transcoding resources is a technical problem to be solved.

Summary of the invention

The embodiment of the present invention provides a real-time control method and device for real-time transcoding, which is used to solve the defect that the real-time transcoding system lacks resources and causes transcoding to be fluent, realizing real-time control of video transcoding and ensuring live broadcast. The fluency.

The embodiment of the present application provides a real-time control method for real-time transcoding, including:

Obtaining current encoded feedback information in real time during transcoding;

Calculating an actual time consumption of a current number of coded frames according to the feedback information;

Calculating a time-consuming deviation of the certain number of encoded frames according to the pre-calculated expected time consumption and the actual time consumption;

According to the time-consuming deviation, the coding parameters are adjusted for transcoding control.

The embodiment of the present application provides a real-time control device for real-time transcoding, including:

An information obtaining module, configured to obtain current feedback information in real time during transcoding;

a calculation module, configured to calculate, according to the feedback information, an actual time consumption of a current number of coded frames; and calculate a time-consuming deviation of the certain number of coded frames according to the pre-calculated expected time consumption and the actual time consumption;

The transcoding control module is configured to adjust the coding parameters according to the time-consuming deviation to perform transcoding control.

The embodiment of the present application provides an electronic device, including the real-time control device for real-time transcoding according to any of the foregoing embodiments.

The embodiment of the present application provides a non-transitory computer readable storage medium, wherein the non-transitory computer readable storage medium can store computer instructions, which can implement real-time transcoding provided by the embodiments of the present application. Some or all of the various implementations of the real-time control method.

An embodiment of the present application provides an electronic device, including: one or more processors; and a memory; wherein the memory stores instructions executable by the one or more processors, the instructions being set to A real-time control method for performing real-time transcoding of any of the above-mentioned applications of the present application.

An embodiment of the present application provides a computer program product, the computer program product comprising a computer program stored on a non-transitory computer readable storage medium, the computer program comprising program instructions, when the program instructions are executed by a computer, The computer is caused to perform the real-time control method for real-time transcoding of any of the above embodiments of the present application.

The real-time control method and device for real-time transcoding provided by the embodiment of the present application obtains feedback information of the encoding process in real time by monitoring the encoding process, and adjusts the complexity parameter of the encoding according to the feedback information, thereby realizing the performance of the transcoding system. The adjustment of the transcoding process does not affect the fluency of the live broadcast due to the load of the server; at the same time, the algorithm and framework of the technical solution proposed by the embodiment of the present application are relatively simple, and can be easily extended to other transcoders. Such as H.265.

DRAWINGS

In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, a brief description of the drawings used in the embodiments or the prior art description will be briefly described below. Obviously, the drawings in the following description It is a certain embodiment of the present application, and other drawings can be obtained according to the drawings without any creative work for those skilled in the art.

1 is a technical flowchart of Embodiment 1 of the present application;

2 is a diagram showing an example of time-consuming deviation level and coding parameter mapping in the embodiment of the present application;

3 is a schematic structural diagram of a device according to Embodiment 2 of the present application;

FIG. 4 is a schematic structural diagram of an electronic device according to an embodiment of the present application.

detailed description

The technical solutions in the embodiments of the present application are clearly and completely described in the following with reference to the accompanying drawings in the embodiments of the present application. Some embodiments are applied, not all of the embodiments. All other embodiments obtained by a person of ordinary skill in the art based on the embodiments of the present application without departing from the inventive scope are the scope of the present application.

Embodiment 1

1 is a technical flowchart of Embodiment 1 of the present application. Referring to FIG. 1, a real-time control method for real-time transcoding provided by an embodiment of the present application mainly includes the following steps:

Step 110: Acquire current encoded feedback information in real time during transcoding;

Video Transcoding refers to converting a video stream that has been compression-coded into another video stream to accommodate different network bandwidths, different terminal processing capabilities, and different user requirements. The transcoding is essentially a process of decoding and re-encoding. The core of the embodiment of the present application is to control the encoding complexity parameter of the encoding according to the encoded feedback information, thereby realizing real-time control of transcoding.

The feedback information in this embodiment includes the frame rate value of the transcoding, the encoding time of each frame in the current certain number of encoded frames, and the frame rate is the number of frames encoded per second.

The transcoder transcodes with the default encoding parameters, and monitors the performance of the encoder in real time during the transcoding process. In general, for better frame rate control, the encoder usually has a lookahead buffer (LAB, first buffer) to analyze the complexity of the input in real time.

If LAB=25 is set, the real first encoded frame will be output after 25 frames. Similarly, if LAB=30 is set, the real first encoded frame will be output after 30 frames.

Calculating the expected time-consuming calculation formula of the certain number of encoded frames according to the frame rate value is as follows:

Time_expected=frame number/frame rate value

The time_expected is the expected time consuming, and the number of frames is the total number of the certain number of encoded frames.

Assuming the current frame rate is 25 fps/s and the LAB = frame rate value = 25, the encoder delay is 1 s. If LAB=30, the conversion result of the time-consuming time of 30 frames calculated by the frame rate conversion encoder is (30 × 1 s) / 25 = 1.2 s.

Step 120: Calculate an actual time consumption of a current number of coded frames according to the feedback information.

Acquiring the encoding time of each frame in the certain number of encoded frames from the feedback information, and accumulating the actual time consumption of the certain number of encoded frames. The specific calculation formula is as follows:

[Correct according to Rule 26 08.09.2016]

The time_elasped is the actual time consuming, the time_frame is the coding time of each frame, N is the sum of the number of coded frames, and i is the sequence number of each frame, where i ∈ [1, N].

Step 130: Calculate a time-consuming deviation of the certain number of coded frames according to the pre-calculated expected time consumption and the actual time consumption;

The time-consuming deviation is calculated as follows:

[Correct according to Rule 26 08.09.2016]
_{Time_delte} =(time _elasped -time _expected )/time_expected

Among them, time_delta is the time-consuming deviation value, and time_expected is the expected time consuming.

Step 140: Adjust the coding parameters to perform transcoding control according to the time-consuming deviation.

In the embodiment of the present application, the time-consuming deviation level is preset, and the coding parameters to be adjusted are preset for each of the time-consuming deviation levels.

In this embodiment, an example of the time-consuming deviation level is as follows:

Enc_delay_thresh_0=10;

Enc_delay_thresh_10=30;

Enc_delay_thresh_30=50;

Enc_delay_thresh_50=70;

Enc_delay_thresh_70=100;

Enc_delay_thresh_100=200;

Where enc_delay_thresh_0 to enc_delay_thresh_100 indicate the actual encoding of the encoder The difference between the time consumption and the expected time consumption is different. The embodiment of the present application takes the percentage as the difference between the actual coding time consumption and the expected time consumption. For example, enc_delay_thresh_0 indicates that the actual coding time consumption and the expected time consumption deviation are 10%, in units of 1 s. Then the deviation is 100ms. Of course, the above data is used for example only, and is not limited to the embodiment of the present application.

In step 130, after the time-consuming deviation time_delta is calculated, the time-consuming deviation level set in advance is queried according to the time-consuming deviation, corresponding coding parameters are obtained, and corresponding parameters are adjusted for the coding. Of course, the contribution of each parameter adjustment to the overall system performance is adjusted in advance, and a specific analysis is performed in advance.

2 is a diagram showing an example of time-consuming deviation level and coding parameter mapping according to an embodiment of the present application. Referring to FIG. 2, the previous time-consuming deviation level interval is a subset of the next time-consuming deviation level interval. In Figure 2, thresh is a different time-consuming deviation level, parameter NULL: deviation within 10%, that can be adjusted; parameter Disable_i8x8: do not do i_8x8 intra prediction, only do i_4x4 and i_16x16; parameter Subme-1 : sub-pixel motion estimation complexity, minus 1 is based on the existing parameters minus 1; parameter Subme-2: sub-pixel motion estimation complexity, minus 2 is based on the existing parameters minus 2; parameter Subme-3: sub Pixel motion estimation complexity, minus 3 is reduced by 3 on the basis of existing parameters; parameter ME_RANGE: motion estimation search range; several parameters are combined to indicate that several parameters are simultaneously adjusted, thereby reducing coding complexity.

In this embodiment, by monitoring the encoding process, the feedback information of the encoding process is obtained in real time, and the complexity parameter of the encoding is adjusted according to the feedback information, thereby realizing the adjustment of the performance of the transcoding system, so that the transcoding process is not caused by the load of the server. , which affects the fluency of the live broadcast, thereby improving the user's viewing experience.

Finally, it should be understood that those skilled in the art can understand that all or part of the process of implementing the above embodiments can be completed by a computer program to instruct related hardware, and the program can be stored in a non-transitory computer. In a readable storage medium, the program, when executed, may include the flow of an embodiment of the methods as described above. The non-transitory computer readable storage medium may be a magnetic disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

Embodiment 2

3 is a schematic structural diagram of a device according to Embodiment 2 of the present application. Referring to FIG. 3, a real-time control device for real-time transcoding provided by an embodiment of the present application mainly includes the following modules: an information acquiring module 310, a computing module 320, and a transcoding control. Module 330.

The information obtaining module 310 is configured to obtain current encoded feedback information in real time during the transcoding process;

The calculating module 320 is configured to calculate an actual time consumption of a current number of coded frames according to the feedback information, and calculate a time-consuming deviation of the certain number of coded frames according to the pre-calculated expected time consumption and the actual time consumption;

The transcoding control module 330 is configured to adjust coding parameters according to the time-consuming deviation to perform transcoding control.

Specifically, the calculating module 320 is further configured to: obtain, from the feedback information, an encoding time of each frame in the certain number of encoded frames, and accumulate the actual time consumption of the certain number of encoded frames.

Specifically, the calculating module 320 is further configured to: before calculating a time-consuming deviation of the certain number of encoded frames, acquire the encoded frame rate value, and calculate, according to the frame rate value, the quantity of the encoded frame. The expectation is time consuming.

Specifically, the transcoding control module 330 is further configured to: preset a time-consuming deviation level, and preset an encoding parameter to be adjusted for each of the time-consuming deviation levels.

Specifically, the transcoding control module 330 is further configured to: query the pre-set time-consuming deviation level according to the time-consuming deviation, obtain a corresponding encoding parameter, and perform corresponding parameter adjustment on the encoding.

The device shown in FIG. 3 can perform the method of the corresponding embodiment of FIG. 1 and FIG. 2 . The implementation principle and technical effects can refer to the content of the embodiment corresponding to FIG. 1 and FIG. 2 , and details are not described herein again.

In another embodiment of the present application, an electronic device is provided, including the real-time control device for real-time transcoding described in any of the foregoing embodiments.

In another embodiment of the present application, a non-transitory computer readable storage medium is also provided, the non-transitory computer readable storage medium storing computer executable instructions executable by any of the above methods The real-time control method of real-time transcoding in the example.

4 is a schematic diagram showing the hardware structure of an electronic device for performing real-time transcoding real-time control according to an embodiment of the present application. As shown in FIG. 4, the device includes:

One or more processors 410 and memory 420, one processor 410 is exemplified in FIG.

The apparatus for performing the real-time control method of real-time transcoding may further include: an input device 430 and an output device 440.

The processor 410, the memory 420, the input device 430, and the output device 440 may be connected by a bus or other means, as exemplified by a bus connection in FIG.

The memory 420 is used as a non-transitory computer readable storage medium, and can be used for storing a non-volatile software program, a non-volatile computer executable program, and a module, as in the real-time control method of the real-time transcoding in the embodiment of the present application. Program instructions/modules (eg, information acquisition module 310, calculation module 320, and transcoding control module 330 shown in FIG. 3). The processor 410 executes various functional applications and data processing of the server by running non-volatile software programs, instructions and modules stored in the memory 420, that is, a real-time control method for real-time transcoding of the above method embodiments.

The memory 420 may include a storage program area and an storage data area, wherein the storage program area may store an operating system, an application required for at least one function; the storage data area may store data created according to the use of the real-time control device of the real-time transcoding Wait. Moreover, memory 420 can include high speed random access memory, and can also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other non-volatile solid state storage device. In some embodiments, memory 420 can optionally include memory remotely located relative to processor 410, which can be connected to a real-time control device for real-time transcoding via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

The input device 430 can receive the input digital or character information and generate a key signal input related to user settings and function control of the real-time control device of the real-time transcoding. Output device 440 can include a display device such as a display screen.

The one or more modules are stored in the memory 420, and when executed by the one or more processors 410, perform a real-time control method of real-time transcoding in any of the above method embodiments.

The above products can perform the methods provided by the embodiments of the present application, and have the corresponding functional modules and beneficial effects of the execution method. For technical details that are not described in detail in this embodiment, reference may be made to the method provided by the embodiments of the present application.

The electronic device of the embodiment of the present application exists in various forms, including but not limited to:

(1) Mobile communication devices: These devices are characterized by mobile communication functions and are mainly aimed at providing voice and data communication. Such terminals include: smart phones (such as iPhone), multimedia phones, functional phones, and low-end phones.

(2) Ultra-mobile personal computer equipment: This type of equipment belongs to the category of personal computers, with calculation and processing. Features, generally also have mobile Internet features. Such terminals include: PDAs, MIDs, and UMPC devices, such as the iPad.

(3) Portable entertainment devices: These devices can display and play multimedia content. Such devices include: audio, video players (such as iPod), handheld game consoles, e-books, and smart toys and portable car navigation devices.

(4) Server: A device that provides computing services. The server consists of a processor, a hard disk, a memory, a system bus, etc. The server is similar to a general-purpose computer architecture, but because of the need to provide highly reliable services, processing power and stability High reliability in terms of reliability, security, scalability, and manageability.

(5) Other electronic devices with data interaction functions.

The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separate, and the components displayed as units may or may not be physical units, ie may be located A place, or it can be distributed to multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the embodiment. Those of ordinary skill in the art can understand and implement without deliberate labor.

Through the description of the above embodiments, those skilled in the art can clearly understand that the various embodiments can be implemented by means of software plus a necessary general hardware platform, and of course, by hardware. Based on such understanding, the above-described technical solutions may be embodied in the form of software products in essence or in the form of software products, which may be stored in a computer readable storage medium such as ROM/RAM, magnetic Discs, discs, etc., include instructions for causing a computer device (which may be a personal computer, server, or network device, etc.) to perform the methods described in various embodiments or portions of the embodiments.

Finally, it should be noted that the above embodiments are only used to explain the technical solutions of the present application, and are not limited thereto; although the present application is described in detail with reference to the foregoing embodiments, those skilled in the art should understand that they can still The technical solutions described in the foregoing embodiments are modified, or the equivalents of the technical features are replaced by the equivalents. The modifications and substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

A real-time control method for real-time transcoding, which is characterized in that it is applied to an electronic device, and includes the following steps:

Obtaining current encoded feedback information in real time during transcoding;

Calculating an actual time consumption of a current number of coded frames according to the feedback information;

Calculating a time-consuming deviation of the certain number of encoded frames according to the pre-calculated expected time consumption and the actual time consumption;

According to the time-consuming deviation, the coding parameters are adjusted for transcoding control.
The method according to claim 1, wherein calculating the actual time consumption of the current number of coded frames according to the feedback information further comprises:

Acquiring the encoding time of each frame in the certain number of encoded frames from the feedback information, and accumulating the actual time consumption of the certain number of encoded frames.
The method according to claim 1, wherein before calculating the time-consuming deviation of the certain number of encoded frames, the method further comprises:

Obtaining the encoded frame rate value, and calculating the expected time consumption of the certain number of encoded frames according to the frame rate value.
The method according to claim 1, wherein adjusting the encoding parameters according to the time-consuming deviation further comprises:

The time-consuming deviation level is preset, and the coding parameters to be adjusted are preset for each of the time-consuming deviation levels.
The method according to claim 1 or 4, wherein adjusting the encoding parameters according to the time-consuming deviation further comprises:

And according to the time-consuming deviation, querying the preset time-consuming deviation level, obtaining corresponding coding parameters, and performing corresponding parameter adjustment on the coding.
A real-time control device for real-time transcoding, comprising the following modules:

An information obtaining module, configured to obtain current feedback information in real time during transcoding;

a calculation module, configured to calculate, according to the feedback information, a actual consumption of a current number of coded frames Calculating a time-consuming deviation of the certain number of coded frames according to the pre-calculated expected time consumption and the actual time consumption;

The transcoding control module is configured to adjust the coding parameters according to the time-consuming deviation to perform transcoding control.
The device according to claim 6, wherein the calculation module is further configured to:

Acquiring the encoding time of each frame in the certain number of encoded frames from the feedback information, and accumulating the actual time consumption of the certain number of encoded frames.
The device according to claim 6, wherein the calculation module is further configured to:

Before calculating the time-consuming deviation of the certain number of encoded frames, acquiring the encoded frame rate value, and calculating the expected time consumption of the certain number of encoded frames according to the frame rate value.
The apparatus according to claim 6, wherein said transcoding control module is further configured to:

The time-consuming deviation level is preset, and the coding parameters to be adjusted are preset for each of the time-consuming deviation levels.
The apparatus according to claim 6 or 9, wherein the transcoding control module is further configured to:

And according to the time-consuming deviation, querying the preset time-consuming deviation level, obtaining corresponding coding parameters, and performing corresponding parameter adjustment on the coding.
An electronic device characterized by comprising real-time control device for real-time transcoding according to any one of claims 6-10.
A non-transitory computer readable storage medium, wherein the non-transitory computer readable storage medium stores computer instructions for causing the computer to perform the method of any of claims 1-5 .
An electronic device, comprising:

One or more processors; and,

a memory communicatively coupled to the one or more processors; wherein

The memory stores instructions executable by the one or more processors, the instructions being The one or more processors are executed to enable the one or more processors to:

Obtaining current encoded feedback information in real time during transcoding;

Calculating an actual time consumption of a current number of coded frames according to the feedback information;

Calculating a time-consuming deviation of the certain number of encoded frames according to the pre-calculated expected time consumption and the actual time consumption;

According to the time-consuming deviation, the coding parameters are adjusted for transcoding control.
A computer program product comprising a computer program stored on a non-transitory computer readable storage medium, the computer program comprising program instructions that, when executed by a computer, cause the computer to execute The method of claims 1-5.