CN108366273A - Distributed video transcoding system and distributed video transcoding method - Google Patents

Distributed video transcoding system and distributed video transcoding method Download PDF

Info

Publication number
CN108366273A
CN108366273A CN201810220527.3A CN201810220527A CN108366273A CN 108366273 A CN108366273 A CN 108366273A CN 201810220527 A CN201810220527 A CN 201810220527A CN 108366273 A CN108366273 A CN 108366273A
Authority
CN
China
Prior art keywords
transcoding
node
task
video
surplus resources
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201810220527.3A
Other languages
Chinese (zh)
Inventor
钟微
刘科
方力
叶龙
张勤
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Communication University of China
Original Assignee
Communication University of China
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Communication University of China filed Critical Communication University of China
Priority to CN201810220527.3A priority Critical patent/CN108366273A/en
Publication of CN108366273A publication Critical patent/CN108366273A/en
Pending legal-status Critical Current

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/234—Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs
    • H04N21/2343—Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements
    • H04N21/234309—Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements by transcoding between formats or standards, e.g. from MPEG-2 to MPEG-4 or from Quicktime to Realvideo
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/40—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using video transcoding, i.e. partial or full decoding of a coded input stream followed by re-encoding of the decoded output stream
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs
    • H04N21/4402—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display
    • H04N21/440218—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display by transcoding between formats or standards, e.g. from MPEG-2 to MPEG-4

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)

Abstract

The present invention provides a kind of distributed video transcoding system and method, including：Acquire transcoding task；It is worth by video sequence of unit load unit time user clicks, by its descending sequence to transcoding task ranking；Determine each transcoding required by task load resource；Determine each transcoding node surplus resources；Choose the most transcoding node of surplus resources；Judge whether video sequence Maximum Value transcoding required by task load resource is less than the surplus resources of the most transcoding nodes of surplus resources：If being less than, the transcoding task is assigned to the transcoding node, time big transcoding task execution above-mentioned steps are chosen after distribution；If being not less than, video sequence Maximum Value transcoding task is skipped, other transcoding tasks is chosen and executes the judgment step successively, if there is no the transcoding tasks that required load resource is less than surplus resources, waiting has transcoding node to complete transcoding task, returns and determines surplus resources step.The above method and system keep costly video priority transcoding, transcoding rate high.

Description

Distributed video transcoding system and distributed video transcoding method
Technical field
The present invention relates to technical field of video processing, more specifically, be related to a kind of distributed video transcoding system and point Cloth video transcoding method.
Background technology
As the scenes such as the various high concurrent access of large-scale website, mass data processing are also more and more, in order to realize net The High Availabitity stood, easily flexible, expansible, safe etc. targets, distributed system are also just increasingly paid attention to.Turn in distributed video In code system, is asked in face of a large amount of transcodings, how effectively to distribute transcoding task, task scheduling algorithm is long-standing, Ke Yiying Used in different field.Task scheduling in distributed video transcoding system has static scheduling algorithm and dynamic dispatching algorithm.It is quiet The task scheduling strategy of state dispatching algorithm has designed in advance, and load equalizer executes the algorithm policy when realizing request distribution, Request of the scheduling distribution from client.Dynamic dispatching algorithm obtains the request connection quantity of each server node, load first The real-time load information such as state analyzes the real time load situation of group system, then according to the specific feelings of the load information of feedback New request is distributed to each service station in cluster and handled, reaches a generally dynamic load balancing by condition.
Traditional min-link algorithm LBLC (Locality-Based Least Connection based on locality Scheduling in the case that) its thought is that each server performance difference is little in group system, there will be identical purpose The request of IP is assigned to as possible on same server.If the request is to ask or handle for the first time the server of the request When full load, then dispatched according to Smallest connection number dispatching algorithm.Its dispatching algorithm is applied in video code conversion field, then It is by multiple same task as a subtask, and task distribution each time, the maximum node all from node surplus resources Start to distribute.
Traditional Smallest connection algorithm based on locality is located as the same task when there is multiple same tasks Reason, when transcoding task definition difference, task distribution each time can all divide since node surplus resources maximum node Match, and when task encounters obstruction, task original place is waited for be distributed down until the enough transcoding tasks of resource complete just continuation, in this way Very waste of resource and cause transcoding time lengthening, transcoding efficiency low.
Invention content
In view of the above problems, the object of the present invention is to provide a kind of effectively distribution transcoding task, transcoding rate is improved, is improved Resource utilization ratio distributed video transcoding system and distributed video transcoding method.
According to an aspect of the present invention, a kind of distributed video transcoding method is provided, including：Step 1, acquisition, which waits for, turns The transcoding task of code；Step 2, it is worth using the user clicks on unit load unit time as video sequence, according to video Sequence is worth descending sequence and is ranked up to transcoding task；Step 3, it determines required when each transcoding task transcoding Load resource；Step 4, the surplus resources of each transcoding node are determined；Step 5, the most transcoding node of surplus resources is chosen rmax；Step 6, it is most to judge whether the load resource of the transcoding required by task of video sequence Maximum Value is less than surplus resources Surplus resources on transcoding node：If the load resource is less than the surplus resources, step 7, by sequence Maximum Value Transcoding task is assigned to the most transcoding node r of surplus resourcesmax；After the maximum transcoding task distribution of video sequence value, in step In 8, secondary big transcoding task execution step 4 in selecting step 2；If the load resource is not less than the surplus resources, step Rapid 9, the transcoding task of the video sequence Maximum Value is skipped, chooses and is worth descending sequence sequence according to video sequence Other transcoding tasks execute step 6 successively, if the load resource of all transcoding tasks be not less than transcoding node rmaxIt is surplus Remaining resource, in step 10, waiting have transcoding node to complete transcoding task, return to step 4.
According to another aspect of the present invention, a kind of distributed video transcoding system is provided, including：Task acquisition module, The transcoding tasks to be transcoded such as acquisition；Sorting module, using the user clicks on unit load unit time as video sequence Value, is worth descending sequence according to video sequence and is ranked up to the transcoding task that task acquisition module acquires；Load Resource determination module determines load resource required when each transcoding task transcoding；Surplus resources determining module determines each The surplus resources of transcoding node；Calling module chooses the most transcoding node of the surplus resources of surplus resources determining module determination rmax, the transcoding task and load resource determining module of video sequence Maximum Value determine in sorting module video sequence valence It is worth the load resource of maximum transcoding required by task；First judgment module judges the transcoding task institute of video sequence Maximum Value Whether the load resource needed is less than the surplus resources on the most transcoding node of surplus resources, wherein if the load resource Less than the surplus resources, task allocating module is sent instructions to, task allocating module is by the transcoding task of sequence Maximum Value It is assigned to the most transcoding node r of surplus resourcesmax, after the maximum transcoding task distribution of video sequence value, send more new command to Sorting module and surplus resources determining module, obtain updated video sequence Maximum Value transcoding task and surplus resources most More transcoding node rmax；If the load resource is not less than the surplus resources, calling module is sent instructions to, calls mould Block calls other transcoding tasks and its correspondence that descending sequence sequence is worth according to video sequence of sorting module successively Load resource be sent to the first judgment module, when there are load resource be less than the transcoding node rmaxSurplus resources turn When code task, task allocating module is sent instructions to, the transcoding task is assigned to transcoding node r by task allocating modulemax, When there is no load resources to be less than the transcoding node rmaxSurplus resources transcoding task when, send instructions to surplus resources Determining module determines the surplus resources of each transcoding node in real time.
Distributed video transcoding method and system of the present invention introduce video code conversion task value, make costly regard Frequently preferential transcoding, and the mechanism that cannot distributing for task is blocked is encountered, it is improved to temporarily skip to continue inquiry mechanism backward, To improve whole transcoding rate, total transcoding deadline is reduced.
Description of the drawings
By reference to the explanation below in conjunction with attached drawing, with a fuller understanding of the present invention, other purposes of the invention And result will be more clearly understood and understood.In the accompanying drawings：
Fig. 1 is the flow chart of distributed video transcoding method of the present invention；
Fig. 2 is the composition block diagram of distributed video transcoding system of the present invention.
Specific implementation mode
In the following description, for purposes of illustration, it in order to provide the comprehensive understanding to one or more embodiments, explains Many details are stated.It may be evident, however, that these embodiments can also be realized without these specific details. Hereinafter, specific embodiments of the present invention will be described in detail with reference to the accompanying drawings.
Hereinafter, specific embodiments of the present invention will be described in detail with reference to the accompanying drawings.
Fig. 1 is the flow chart of distributed video transcoding method of the present invention, as shown in Figure 1, being distributed described in the present embodiment Formula video transcoding method includes：
Step 1, the transcoding task to be transcoded such as acquisition；
Step 2, it is worth using the user clicks on unit load unit time as video sequence, according to video sequence valence It is worth descending sequence to be ranked up transcoding task；
Step 3, load resource required when each transcoding task transcoding is determined；
Step 4, the surplus resources of each transcoding node are determined；
Step 5, the most transcoding node r of surplus resources is chosenmax；
Step 6, judge whether the load resource of the transcoding required by task of video sequence Maximum Value is less than surplus resources most Surplus resources on more transcoding nodes：
If step 7 the transcoding task of sequence Maximum Value is assigned to by the load resource less than the surplus resources The most transcoding node r of surplus resourcesmax；
After the maximum transcoding task distribution of video sequence value, in step 8, time big transcoding task is held in selecting step 2 Row step 4；
If the load resource is not less than the surplus resources, step 9, turning for the video sequence Maximum Value is skipped Code task, other transcoding tasks that selection is worth descending sequence sequence according to video sequence execute step 6 successively, if The load resource of all transcoding tasks is not less than transcoding node rmaxSurplus resources, in step 10, waiting have transcoding node Complete transcoding task, return to step 4.
Above-mentioned distributed video transcoding method introduces video code conversion task value, makes costly video priority transcoding, And the mechanism that the task that encountering in method cannot distribute is blocked is figured based on locality min-link for traditional, is improved Continue inquiry mechanism backward temporarily to skip, to improve whole transcoding rate, reduces total transcoding deadline.
Preferably, in step 10, when waiting has transcoding node to complete transcoding task, step 1 collects new transcoding and appoints Business, then stop waiting for, and sequence, which is added, in new transcoding task executes step 2.
Furthermore it is preferred that the step 1 further includes：
Inquiry etc. transcoding task to be transcoded quantity, when inquiry etc. transcoding task to be transcoded quantity be 0 when, respectively Transcoding node enters dormant state, when not being 0, executes step 2.
Above-mentioned steps 3 may be used in the static scheduling algorithm, dynamic dispatching algorithm or LBLC dispatching algorithms of the prior art Load resource required when each transcoding task transcoding is determined, it is, however, preferable that each transcoding task transcoding of the determination The method of the load resource of Shi Suoxu includes：
It chooses a REF video as reference, the resource that REF video transcoding is consumed is set as 1；
Product of the video of transcoding task relative to the ratio of the lateral resolution of REF video, longitudinal resolution and frame per second, For the load resource of transcoding required by task,
Wherein, i indexes for transcoding task, miFor i-th of transcoding task transcoding when required load resource, wi、hiAnd fiPoint Not Wei i-th of transcoding task lateral resolution, longitudinal resolution and frame per second, wref、hrefAnd frefThe respectively transverse direction of REF video point Resolution, longitudinal resolution and frame per second.
In a preferred embodiment of the invention, resolution ratio is chosen to be 640*480, regard on the basis of the video that frame per second is 25 Frequency as reference, sets load resource that the REF video transcoding of 640*480*25 is consumed as 1, and resolution ratio is higher, the inside it is macro Block number is more, and the transcoding resource to be consumed is also more.
Above-mentioned steps 4 may be used in the static scheduling algorithm, dynamic dispatching algorithm or LBLC dispatching algorithms of the prior art The surplus resources of each transcoding node are determined, it is, however, preferable that the method for determining the surplus resources of each transcoding node includes：
The cycle tests with multiple videos is set, setting refers to transcoding node；
Record CPU the and RAM idleness that the multiple video is occupied in each transcoding node transcoding；
CPU the and RAM idleness occupied in the node transcoding with reference to transcoding by each video of cycle tests is under Formula (1) and (2) obtain the CPU static state weight coefficient and RAM static state weight coefficients with reference to transcoding node,
Wherein, k is the video index of cycle tests, and h is the video total number of cycle tests, WcpuTo refer to transcoding node CPU static state weight coefficients, WRAMFor the RAM static state weight coefficients with reference to transcoding node, Δ CPUk_refFor the kth of cycle tests The CPU idleness that a video occupies when with reference to transcoding node transcoding, is Δ RAMk_refExist for k-th of video of cycle tests The RAM idleness occupied when with reference to transcoding node transcoding；
It is accounted for by cycle tests when referring to transcoding node and not as other transcoding node transcodings of reference transcoding node (3) obtain other described transcoding nodes relative to the environmental rating with reference to transcoding node to CPU and RAM idleness according to the following formula The factor,
Wherein, j indexes for transcoding node, Δ CPUk_jThe CPU occupied in transcoding node j transcodings for k-th of video is idle Rate, Δ RAMk_jFor the RAM idleness that k-th of video is occupied in transcoding node j transcodings, εjFor the environmental rating of transcoding node j The factor；
By reference to the CPU static state weight coefficient and RAM static state weight coefficient of transcoding node, each transcoding node relative to ginseng (4) are built the current CPU idleness and RAM idleness of the environmental rating factor and each transcoding node of examining transcoding node according to the following formula The surplus resources model of each node,
rj=εj*[(1-CPUj_used)*Wcpu+(1-RAMj_used)*WRAM] (4)
Wherein, rjFor the surplus resources of j-th of transcoding node, CPUj_usedFor the occupied CPU moneys of j-th of transcoding node Source percentage, RAMj_usedFor the occupied RAM resources percentage of j-th of transcoding node.
Fig. 2 is the composition block diagram of distributed video transcoding system of the present invention, as shown in Fig. 2, dividing described in the present embodiment Cloth video code conversion system includes：
Task acquisition module 10, the transcoding task to be transcoded such as acquisition；
Sorting module 20 is worth using the user clicks on unit load unit time as video sequence, according to video Sequence is worth descending sequence and is ranked up to the transcoding task that task acquisition module acquires；
Load resource determining module 30 determines load resource required when each transcoding task transcoding；
Surplus resources determining module 40 determines the surplus resources of each transcoding node；
Calling module 50 chooses the most transcoding node r of the surplus resources of surplus resources determining module determinationmax, sequence mould The video sequence Maximum Value that the transcoding task and load resource determining module of video sequence Maximum Value determine in block turns The load resource of code required by task；
It is surplus to judge whether the load resource of the transcoding required by task of video sequence Maximum Value is less than for first judgment module 60 Surplus resources on the most transcoding node of remaining resource；
Transcoding task is assigned to corresponding transcoding by task allocating module 70 according to the judging result of the first judgment module 60 Node, specifically：
If the load resource is less than the surplus resources, task allocating module, task allocating module are sent instructions to The transcoding task of sequence Maximum Value is assigned to the most transcoding node r of surplus resourcesmax, the maximum transcoding of video sequence value After task distribution, more new command discharge sequence module and surplus resources determining module are sent, obtains updated video sequence value Maximum transcoding task and the most transcoding node r of surplus resourcesmax；
If the load resource is not less than the surplus resources, calling module is sent instructions to, calling module is adjusted successively With other transcoding tasks and its corresponding load money for being worth descending sequence sequence according to video sequence of sorting module Source is sent to the first judgment module, when there are load resources to be less than the transcoding node rmaxSurplus resources transcoding task when, Task allocating module is sent instructions to, the transcoding task is assigned to transcoding node r by task allocating modulemax, when there is no negative It carries resource and is less than the transcoding node rmaxSurplus resources transcoding task when, send instructions to surplus resources determining module, it is real When determine the surplus resources of each transcoding node.
Preferably, the distributed video transcoding system further includes enquiry module 80, the acquisition of query task acquisition module Etc. the quantity of transcoding task to be transcoded, when inquiry etc. the quantity of transcoding task to be transcoded be 0 when, send dormancy instruction to Each transcoding node.
Furthermore it is preferred that the distributed video transcoding system further includes the second judgment module 90, when there is no loads to provide Source is less than the transcoding node rmaxSurplus resources transcoding task when, judge in the surplus resources determining module stand-by period, adopt Whether collection module collects new transcoding task, wherein
The stand-by period is to wait for having transcoding node to complete transcoding task, and there are load resources to be less than the transcoding node rmaxSurplus resources transcoding task time；
When acquisition module collects new transcoding task, sorting module is sent instructions to, new transcoding task is added Sequence.
Above-mentioned distributed video transcoding system is proposed the LBLC dispatching algorithms being worth based on transcoding task, makes hot video Preferential transcoding, make the distribution of transcoding task will not stop because of obstruction waiting do not reallocate, can continually look for backward task until Until resource allocation is complete.
In one embodiment of the invention, above-mentioned load resource determining module 30 includes：
First setup unit 31, a REF video are set as 1 as reference, by the resource that REF video transcoding is consumed；
Load resource computing unit 32 calculates lateral resolution of the video relative to REF video, the longitudinal direction of transcoding task The product with the ratio of frame per second is differentiated, is the load resource of transcoding required by task.
In one embodiment of the invention, above-mentioned surplus resources determining module 40 includes：
Second setup unit 41, sets the cycle tests with multiple videos, and setting refers to transcoding node；
Recording unit 42 records CPU the and RAM idleness that the multiple video is occupied in each transcoding node transcoding；
With reference to static weight coefficient determination unit 43, by each video of cycle tests in the reference transcoding node transcoding CPU the and RAM idleness of occupancy according to the following formula (1) and (2) obtain with reference to transcoding node CPU static state weight coefficient and RAM it is quiet State weight coefficient,
Wherein, k is the video index of cycle tests, and h is the video total number of cycle tests, WcpuTo refer to transcoding node CPU static state weight coefficients, WRAMFor the RAM static state weight coefficients with reference to transcoding node, Δ CPUk_refFor the kth of cycle tests The CPU idleness that a video occupies when with reference to transcoding node transcoding, is Δ RAMk_refExist for k-th of video of cycle tests The RAM idleness occupied when with reference to transcoding node transcoding；
Environmental rating factor specifying unit 44, by cycle tests with reference to transcoding node and not as with reference to transcoding node Other transcoding node transcodings when CPU the and RAM idleness that occupies according to the following formula (3) obtain other described transcoding nodes relative to With reference to the environmental rating factor of transcoding node,
Wherein, j indexes for transcoding node, Δ CPUk_jThe CPU occupied in transcoding node j transcodings for k-th of video is idle Rate, Δ RAMk_jFor the RAM idleness that k-th of video is occupied in transcoding node j transcodings, εjFor the environmental rating of transcoding node j The factor；
Surplus resources model construction unit 45, by reference to the CPU static state weight coefficient and RAM static state weights of transcoding node Coefficient, each transcoding node relative to reference to transcoding node the environmental rating factor and each transcoding node current CPU idleness and (4) build the surplus resources model of each node to RAM idleness according to the following formula,
rj=εj*[(1-CPUj_used)*Wcpu+(1-RAMj_used)*WRAM] (4)
Wherein, rjFor the surplus resources of j-th of transcoding node, CPUj_usedFor the occupied CPU moneys of j-th of transcoding node Source percentage, RAMj_usedFor the occupied RAM resources percentage of j-th of transcoding node.
In one particular embodiment of the present invention, using the polling dispatching algorithm (Round-Robin of the prior art Scheduling, RR are dispatched successively according to the low sequence of server, and every computer node is rotated one time successively) calculation of+knapsack The distributed video transcoding of method and the present invention have carried out transcoding, REF video 640*480*25 to the video of SD video format The resource that transcoding is consumed is 1, obtains the transcoding deadline and transcoding rate is as shown in the table,
Table 1
As can be seen from the above table, it is traditional based on locality min-link algorithm in the task of distribution, when required by task is negative Carry be less than node resource when then stop distributing, and this paper algorithms continue backward inquiry can distribute task, reach reasonable distribution turn The purpose of code task.Also refer to video value concept simultaneously so that costly video priority transcoding.Same task, this Literary algorithm transcoding efficiency promotes 3.26%, and the transcoding deadline reduces 2.70%.
In conclusion being described by way of example with reference to according to the distributed video transcoding proposed by the present invention System and distributed video transcoding method.It will be understood by those skilled in the art, however, that being for what aforementioned present invention was proposed System and method, can also make various improvement on the basis of not departing from the content of present invention.Therefore, protection scope of the present invention is answered It is determined when by the content of appended claims.

Claims (10)

1. a kind of distributed video transcoding method, which is characterized in that including：
Step 1, the transcoding task to be transcoded such as acquisition；
Step 2, using the user clicks on unit load unit time as video sequence be worth, according to video sequence value by Small sequence is arrived greatly to be ranked up transcoding task；
Step 3, load resource required when each transcoding task transcoding is determined；
Step 4, the surplus resources of each transcoding node are determined；
Step 5, the most transcoding node r of surplus resources is chosenmax；
Step 6, it is most to judge whether the load resource of the transcoding required by task of video sequence Maximum Value is less than surplus resources Surplus resources on transcoding node：
If step 7 the transcoding task of sequence Maximum Value is assigned to residue by the load resource less than the surplus resources The most transcoding node r of resourcemax；
After the maximum transcoding task distribution of video sequence value, in step 8, time big transcoding task execution walks in selecting step 2 Rapid 4；
If the load resource is not less than the surplus resources, step 9, the transcoding for skipping the video sequence Maximum Value is appointed Business, other transcoding tasks that selection is worth descending sequence sequence according to video sequence execute step 6 successively, if all The load resource of transcoding task is not less than transcoding node rmaxSurplus resources, in step 10, waiting have transcoding node completion Transcoding task, return to step 4.
2. distributed video transcoding method according to claim 1, which is characterized in that each transcoding task of the determination The method of required load resource includes when transcoding：
It chooses a REF video as reference, the resource that REF video transcoding is consumed is set as 1；
Product of the video of transcoding task relative to the ratio of the lateral resolution of REF video, longitudinal resolution and frame per second, to turn The load resource of code required by task.
3. distributed video transcoding method according to claim 1, which is characterized in that each transcoding node of determination The method of surplus resources includes：
The cycle tests with multiple videos is set, setting refers to transcoding node；
Record CPU the and RAM idleness that the multiple video is occupied in each transcoding node transcoding；
CPU the and RAM idleness occupied in the node transcoding with reference to transcoding by each video of cycle tests is according to the following formula (1) (2) the CPU static state weight coefficient and RAM static state weight coefficients with reference to transcoding node are obtained,
Wherein, k is the video index of cycle tests, and h is the video total number of cycle tests, WcpuFor the CPU with reference to transcoding node Static weight coefficient, WRAMFor the RAM static state weight coefficients with reference to transcoding node, Δ CPUk_refFor k-th of video of cycle tests The CPU idleness occupied when with reference to transcoding node transcoding, is Δ RAMk_refTurn in reference for k-th of video of cycle tests The RAM idleness occupied when code node transcoding；
It is occupied by cycle tests when referring to transcoding node and not as other transcoding node transcodings of reference transcoding node CPU and RAM idleness according to the following formula (3) obtain other described transcoding nodes relative to the environmental rating with reference to transcoding node because Son,
Wherein, j indexes for transcoding node, Δ CPUk_jFor the CPU idleness that k-th of video is occupied in transcoding node j transcodings, ΔRAMk_jFor the RAM idleness that k-th of video is occupied in transcoding node j transcodings, εjFor transcoding node j environmental rating because Son；
Turned relative to reference by reference to the CPU static state weight coefficient and RAM static state weight coefficient of transcoding node, each transcoding node (4) build each section according to the following formula for the environmental rating factor of code node and the current CPU idleness and RAM idleness of each transcoding node The surplus resources model of point,
rj=εj*[(1-CPUj_used)*Wcpu+(1-RAMj_used)*WRAM] (4)
Wherein, rjFor the surplus resources of j-th of transcoding node, CPUj_usedFor the occupied cpu resource percentage of j-th of transcoding node Than RAMj_usedFor the occupied RAM resources percentage of j-th of transcoding node.
4. distributed video transcoding method according to claim 1, which is characterized in that the step 1 further includes：
Inquiry etc. transcoding task to be transcoded quantity, when inquiry etc. transcoding task to be transcoded quantity be 0 when, each transcoding Node enters dormant state, when not being 0, executes step 2.
5. distributed video transcoding method according to claim 1, which is characterized in that in step 10, waiting has transcoding When node completes transcoding task, step 1 collects new transcoding task, then stops waiting for, and sequence is added in new transcoding task Execute step 2.
6. a kind of distributed video transcoding system, which is characterized in that including：
Task acquisition module, the transcoding task to be transcoded such as acquisition；
Sorting module is worth using the user clicks on unit load unit time as video sequence, according to video sequence valence It is worth descending sequence to be ranked up the transcoding task that task acquisition module acquires；
Load resource determining module determines load resource required when each transcoding task transcoding；
Surplus resources determining module determines the surplus resources of each transcoding node；
Calling module chooses the most transcoding node r of the surplus resources of surplus resources determining module determinationmax, regard in sorting module The transcoding task for the video sequence Maximum Value that the transcoding task and load resource determining module of frequency sequence Maximum Value determine Required load resource；
First judgment module, judges whether the load resource of the transcoding required by task of video sequence Maximum Value is less than surplus resources Surplus resources on most transcoding nodes, wherein
If the load resource is less than the surplus resources, task allocating module is sent instructions to, task allocating module is by sequence The transcoding task of row Maximum Value is assigned to the most transcoding node r of surplus resourcesmax, the maximum transcoding task of video sequence value After distribution, more new command discharge sequence module and surplus resources determining module are sent, updated video sequence Maximum Value is obtained Transcoding task and the most transcoding node r of surplus resourcesmax；
If the load resource is not less than the surplus resources, calling module is sent instructions to, the calling module row of calling successively Other transcoding tasks that descending sequence sequence is worth according to video sequence of sequence module and its corresponding load resource are sent out It is sent to the first judgment module, when there are load resources to be less than the transcoding node rmaxSurplus resources transcoding task when, send It instructs to task allocating module, the transcoding task is assigned to transcoding node r by task allocating modulemax, when there is no loads to provide Source is less than the transcoding node rmaxSurplus resources transcoding task when, send instructions to surplus resources determining module, in real time really The surplus resources of fixed each transcoding node.
7. distributed video transcoding system according to claim 6, which is characterized in that the load resource determining module packet It includes：
First setup unit, a REF video are set as 1 as reference, by the resource that REF video transcoding is consumed；
Load resource computing unit, calculate transcoding task video relative to the lateral resolution of REF video, longitudinal resolution and The product of the ratio of frame per second is the load resource of transcoding required by task.
8. distributed video transcoding system according to claim 6, which is characterized in that the surplus resources determining module packet It includes：
Second setup unit, sets the cycle tests with multiple videos, and setting refers to transcoding node；
Recording unit records CPU the and RAM idleness that the multiple video is occupied in each transcoding node transcoding；
With reference to static weight coefficient determination unit, occupied in the node transcoding with reference to transcoding by each video of cycle tests (1) and (2) obtains the CPU static state weight coefficient and RAM static state weights with reference to transcoding node to CPU and RAM idleness according to the following formula Coefficient,
Wherein, k is the video index of cycle tests, and h is the video total number of cycle tests, WcpuFor the CPU with reference to transcoding node Static weight coefficient, WRAMFor the RAM static state weight coefficients with reference to transcoding node, Δ CPUk_refFor k-th of video of cycle tests The CPU idleness occupied when with reference to transcoding node transcoding, is Δ RAMk_refTurn in reference for k-th of video of cycle tests The RAM idleness occupied when code node transcoding；
Environmental rating factor specifying unit, by cycle tests in reference transcoding node and not as other of reference transcoding node (3) obtain other described transcoding nodes relative to reference turn to CPU the and RAM idleness occupied when transcoding node transcoding according to the following formula The environmental rating factor of code node,
Wherein, j indexes for transcoding node, Δ CPUk_jFor the CPU idleness that k-th of video is occupied in transcoding node j transcodings, ΔRAMk_jFor the RAM idleness that k-th of video is occupied in transcoding node j transcodings, εjFor transcoding node j environmental rating because Son；
Surplus resources model construction unit, by reference to the CPU static state weight coefficient and RAM static state weight coefficient of transcoding node, Each transcoding node is empty relative to the current CPU idleness and RAM of the environmental rating factor and each transcoding node with reference to transcoding node (4) build the surplus resources model of each node to not busy rate according to the following formula,
rj=εj*[(1-CPUj_used)*Wcpu+(1-RAMj_used)*WRAM] (4)
Wherein, rjFor the surplus resources of j-th of transcoding node, CPUj_usedFor the occupied cpu resource percentage of j-th of transcoding node Than RAMj_usedFor the occupied RAM resources percentage of j-th of transcoding node.
9. distributed video transcoding system according to claim 6, which is characterized in that further include：
Enquiry module, query task acquisition module acquisition etc. transcoding task to be transcoded quantity, when inquiry etc. it is to be transcoded Transcoding task quantity be 0 when, send dormancy instruction to each transcoding node.
10. distributed video transcoding system according to claim 6, which is characterized in that further include：
Second judgment module, when there is no load resources to be less than the transcoding node rmaxSurplus resources transcoding task when, sentence In the disconnected surplus resources determining module stand-by period, whether acquisition module collects new transcoding task, wherein
The stand-by period is to wait for having transcoding node to complete transcoding task, and there are load resources to be less than the transcoding node rmax Surplus resources transcoding task time；
When acquisition module collects new transcoding task, sorting module is sent instructions to, sequence is added in new transcoding task.
CN201810220527.3A 2018-03-16 2018-03-16 Distributed video transcoding system and distributed video transcoding method Pending CN108366273A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201810220527.3A CN108366273A (en) 2018-03-16 2018-03-16 Distributed video transcoding system and distributed video transcoding method

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201810220527.3A CN108366273A (en) 2018-03-16 2018-03-16 Distributed video transcoding system and distributed video transcoding method

Publications (1)

Publication Number Publication Date
CN108366273A true CN108366273A (en) 2018-08-03

Family

ID=63000835

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201810220527.3A Pending CN108366273A (en) 2018-03-16 2018-03-16 Distributed video transcoding system and distributed video transcoding method

Country Status (1)

Country Link
CN (1) CN108366273A (en)

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110113632A (en) * 2019-04-22 2019-08-09 武汉星汉互娱科技有限公司 A kind of Streaming Media low delay transcoding method for scheduling task and device
CN110879749A (en) * 2018-09-06 2020-03-13 阿里巴巴集团控股有限公司 Scheduling method and scheduling device for real-time transcoding task
CN111294647A (en) * 2018-12-10 2020-06-16 中国移动通信集团有限公司 Video processing method, device and equipment and storage medium
CN114024973A (en) * 2021-11-01 2022-02-08 上海绚显科技有限公司 Resource scheduling method, device, server and system for live cloud transcoding
CN114598927A (en) * 2022-03-03 2022-06-07 京东科技信息技术有限公司 Method and system for scheduling transcoding resources and scheduling device
CN115174948A (en) * 2022-06-30 2022-10-11 上海哔哩哔哩科技有限公司 Scheduling method and device based on transcoding resources

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20120051440A1 (en) * 2010-08-24 2012-03-01 Lsi Corporation Video transcoder with flexible quality and complexity management
US20140321835A1 (en) * 2013-04-25 2014-10-30 Albert Straub Space efficiency and management of content
CN105872595A (en) * 2016-03-31 2016-08-17 乐视控股（北京）有限公司 Transcoding task distribution method and device
CN105992020A (en) * 2015-07-24 2016-10-05 乐视云计算有限公司 Video conversion resource distribution method and system

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20120051440A1 (en) * 2010-08-24 2012-03-01 Lsi Corporation Video transcoder with flexible quality and complexity management
US20140321835A1 (en) * 2013-04-25 2014-10-30 Albert Straub Space efficiency and management of content
CN105992020A (en) * 2015-07-24 2016-10-05 乐视云计算有限公司 Video conversion resource distribution method and system
CN105872595A (en) * 2016-03-31 2016-08-17 乐视控股（北京）有限公司 Transcoding task distribution method and device

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
刘格非: "大规模视频转码调度算法的研究与设计", 《中国优秀硕士学位论文全文数据库 信息科技辑》 *
谭智一: "分布式视频转码服务调度算法研究", 《中国优秀硕士学位论文全文数据库 信息科技辑》 *

Cited By (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110879749A (en) * 2018-09-06 2020-03-13 阿里巴巴集团控股有限公司 Scheduling method and scheduling device for real-time transcoding task
CN110879749B (en) * 2018-09-06 2023-04-07 阿里巴巴集团控股有限公司 Scheduling method and scheduling device for real-time transcoding task
CN111294647A (en) * 2018-12-10 2020-06-16 中国移动通信集团有限公司 Video processing method, device and equipment and storage medium
CN111294647B (en) * 2018-12-10 2022-04-08 中国移动通信集团有限公司 Video processing method, device and equipment and storage medium
CN110113632A (en) * 2019-04-22 2019-08-09 武汉星汉互娱科技有限公司 A kind of Streaming Media low delay transcoding method for scheduling task and device
CN114024973A (en) * 2021-11-01 2022-02-08 上海绚显科技有限公司 Resource scheduling method, device, server and system for live cloud transcoding
CN114024973B (en) * 2021-11-01 2023-11-24 上海绚显科技有限公司 Resource scheduling method, device, server and system for direct broadcast cloud transcoding
CN114598927A (en) * 2022-03-03 2022-06-07 京东科技信息技术有限公司 Method and system for scheduling transcoding resources and scheduling device
CN115174948A (en) * 2022-06-30 2022-10-11 上海哔哩哔哩科技有限公司 Scheduling method and device based on transcoding resources
CN115174948B (en) * 2022-06-30 2023-12-15 上海哔哩哔哩科技有限公司 Scheduling method and device based on transcoding resources

Similar Documents

Publication Publication Date Title
CN108366273A (en) Distributed video transcoding system and distributed video transcoding method
CN103530189B (en) It is a kind of towards the automatic telescopic of stream data and the method and device of migration
CN104657221B (en) The more queue flood peak staggered regulation models and method of task based access control classification in a kind of cloud computing
CN106020933B (en) Cloud computing dynamic resource scheduling system and method based on ultralight amount virtual machine
Liu et al. Resource preprocessing and optimal task scheduling in cloud computing environments
CN110231976B (en) Load prediction-based edge computing platform container deployment method and system
CN108519911A (en) The dispatching method and device of resource in a kind of cluster management system based on container
CN106534318B (en) A kind of OpenStack cloud platform resource dynamic scheduling system and method based on flow compatibility
CN104050042B (en) The resource allocation methods and device of ETL operations
CN109582448B (en) Criticality and timeliness oriented edge calculation task scheduling method
CN104902001B (en) Web request load-balancing method based on operating system virtualization
CN104298550A (en) Hadoop-oriented dynamic scheduling method
CN102279771A (en) Method and system for adaptively allocating resources as required in virtualization environment
CN1706165A (en) Method and apparatus for network field communication control
CN109783225B (en) Tenant priority management method and system of multi-tenant big data platform
CN105242956A (en) Virtual function service chain deployment system and deployment method therefor
CN102262567A (en) Virtual machine scheduling decision system, platform and method
CN102611723A (en) Method for building high-performance computing application service based on virtualization technology
CN103701886A (en) Hierarchic scheduling method for service and resources in cloud computation environment
CN110308967A (en) A kind of workflow cost based on mixed cloud-delay optimization method for allocating tasks
CN104881322A (en) Method and device for dispatching cluster resource based on packing model
CN103297499A (en) Scheduling method and system based on cloud platform
CN103491024A (en) Job scheduling method and device for streaming data
CN103257896B (en) A kind of Max-D job scheduling method under cloud environment
CN106790332A (en) A kind of resource regulating method, system and host node

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
RJ01 Rejection of invention patent application after publication
RJ01 Rejection of invention patent application after publication

Application publication date: 20180803