CN104378665B - A kind of distributed trans-coding system and method based on DTV - Google Patents
A kind of distributed trans-coding system and method based on DTV Download PDFInfo
- Publication number
- CN104378665B CN104378665B CN201410683153.0A CN201410683153A CN104378665B CN 104378665 B CN104378665 B CN 104378665B CN 201410683153 A CN201410683153 A CN 201410683153A CN 104378665 B CN104378665 B CN 104378665B
- Authority
- CN
- China
- Prior art keywords
- transcoding
- content
- section
- server
- matchmaker
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/25—Management operations performed by the server for facilitating the content distribution or administrating data related to end-users or client devices, e.g. end-user or client device authentication, learning user preferences for recommending movies
- H04N21/266—Channel or content management, e.g. generation and management of keys and entitlement messages in a conditional access system, merging a VOD unicast channel into a multicast channel
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/231—Content storage operation, e.g. caching movies for short term storage, replicating data over plural servers, prioritizing data for deletion
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/234—Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs
- H04N21/2343—Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements
- H04N21/234309—Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements by transcoding between formats or standards, e.g. from MPEG-2 to MPEG-4 or from Quicktime to Realvideo
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
- H04N21/83—Generation or processing of protective or descriptive data associated with content; Content structuring
- H04N21/845—Structuring of content, e.g. decomposing content into time segments
- H04N21/8456—Structuring of content, e.g. decomposing content into time segments by decomposing the content in the time domain, e.g. in time segments
Abstract
The invention discloses a kind of distributed trans-coding system and method based on DTV, the system includes：Content source storage server, the source contents file of content is provided for storing matchmaker in a manner of section；Transcoding dispatch server, the slice size and number of sections stored for configuring source contents file, and issuing scheduler task to transcoding server cluster makes the section of source contents file read in transcoding server cluster；Slice graph display after control transcoding provides the complete file of content to form matchmaker, and is written into content and integrates in storage server；Transcoding server cluster includes one or more transcoding nodes, and one or more transcoding nodes are used to carry out transcoding to section according to scheduler task；Content integrates storage server, and the complete file of content is provided for storing matchmaker.The beneficial effects of the present invention are the idling-resource of VOD system has obtained effective utilization, and lness ratio of plant is greatly reduced, and improves transcoding efficiency, also saves system Construction cost.
Description
Technical field
The present invention relates to digital TV field, more particularly to a kind of distributed trans-coding system and side based on DTV
Method.
Background technology
At present, because business needs, Digital Cable Interactive TV System is by construction content aggregation platform.Using PB
(Petabyte) cloud storage of level, the internet video and individual video of magnanimity have been polymerize it, to realize that the video of multi-source is having
Depositing for line interactive digital television network is broadcast, it is necessary to carry out transcoding processing, existing transcoding server computing resource to huge volumes of content
Have too many difficulties to cope with, and merely dilatation must increase cost by very big, and according to TV user watched custom, there is part-time system network
Substantial amounts of backstage slack resources in network be present not to be utilized effectively.Original transcoding technology can not meet demand, together
Computing resource idle Shi great Liang also provides condition for new technology in this programme.
Existing transcoding technology
Referring to Fig. 1, Fig. 1 is the trans-coding system of the Digital Cable Interactive TV of prior art, and traditional transcoding technology is a kind of
The mode of procedural style, using special transcoding server, trunking mode is disposed, the step using transcoding as content integrated flow,
Without the dispatching method customized.When content complete editor examination ＆ verification etc. step deposit content source server after, transcoding server with
Carry out read in transcoding and by after transcoding content be stored in content integrating server, finally by Content Management System scheduling be published to
Stored in CDN (i.e. content distributing network).
Resource service condition counts in existing network
Referring to Fig. 2, Fig. 2 is the daily user conversation number curve figure of VOD system, abscissa represent one day in time, indulge
The user conversation number of online request while coordinate representation a certain moment, by simple data analysis, one day a certain
Individual period user conversation number well below hump session, in user's program request number is in lower this period, equipment CPU,
I/O resource occupancy is all relatively low, flow pulling equipment, and a series of background devices such as user conversation management equipment are by overall in idle shape
State.This is just for present solution provides necessary condition.
Traditional transcoding technology will have problems with when in face of the present situation of the aggregate video of current magnanimity：
1) because transcoding task is various, it is necessary to which substantial amounts of transcoding server participates in transcoding work, existing server cluster
Can not meet demand, simple dilatation can be added significantly to system Construction budget.
2) dispatching method is fixed inflexible, can not adapt to the network environment that flow constantly changes.
3) the substantial amounts of storage resource of the storage consumption of the video of magnanimity, budget is added using traditional storage server,
Can not adaption demand.
4) according to TV user watched custom, have in part-time grid and substantial amounts of backstage slack resources be present and do not have
It is utilized effectively.And this part resource can be utilized effectively come transcoding.
The content of the invention
The technical problem to be solved in the present invention is, for above-mentioned server cluster in the prior art can not meet demand,
Dispatching method can not adapt to the network environment that flow constantly changes, and storage server adds budget, the substantial amounts of idle money in backstage
The problem of source is not utilized effectively, there is provided a kind of distributed trans-coding system and method based on DTV.
The technical solution adopted for the present invention to solve the technical problems is：A kind of distribution based on DTV is constructed to turn
Code system, it is characterised in that integrate storage server including content source storage server, content, pass through cloud storage service device portion
The transcoding server cluster and transcoding dispatch server of administration；Wherein,
The content source storage server, the source contents file of content is provided for storing matchmaker in a manner of section；
The transcoding dispatch server, the slice size and number of sections stored for configuring the source contents file,
And issuing scheduler task to the transcoding server cluster makes the section of the source contents file read in the transcoding server collection
Group；Slice graph display after control transcoding provides the complete file of content to form matchmaker, and is written into the integrated storage clothes of the content
It is engaged in device；
The transcoding server cluster includes one or more transcoding node, and one or more described transcoding node is used for according to institute
State scheduler task and transcoding is carried out to the section；
The content integrates storage server, and the complete file of content is provided for storing the matchmaker.
In distributed trans-coding system of the present invention, the distributed trans-coding system also includes content distribution storage clothes
Business device and Content Management System；Wherein,
The content distribution storage server, the complete file for the matchmaker to be provided to content are distributed to content delivery network
Network；
The Content Management System, the metadata of the source contents file of content is provided for matchmaker described in management and control, obtains the source
The section of content file simultaneously travels through one or more described transcoding node, and any one extracted in one or more described transcoding node is empty
Not busy node, so as to assigned tasks to the transcoding dispatch server.
In distributed trans-coding system of the present invention, the transcoding dispatch server includes scheduling node and section
With reconstitution component；Wherein,
The scheduling node, for saving acquired section distribution to the free time extracted according to being assigned for task
The section is carried out transcoding by point, the idle node；The complete file write-in content of matchmaker money content is integrated and deposited
Store up in server；
The section and reconstitution component, for being cut into slices and the source contents file to the slice graph display after transcoding
The complete file of content is provided for matchmaker.
In distributed trans-coding system of the present invention, the section includes section sub-component, restructuring with reconstitution component
Sub-component and cloud storage slice scheduling server；Wherein,
The section sub-component, the source contents file for providing content to the matchmaker carries out being cut into section, and will be cut
The section cut is back to the cloud storage slice scheduling server；
The restructuring sub-component, for being the complete file of matchmaker's money content to the slice graph display after transcoding, and will be described complete
Whole file is back to the cloud storage slice scheduling server；
The cloud storage slice scheduling server, for the section returned and complete file to be sent to the scheduling
Node.
In distributed trans-coding system of the present invention, the transcoding dispatch server also includes priority and calculates mould
Block；
The priority computation module, the matchmaker is obtained for providing the metadata of source contents file of content according to the matchmaker
Provide n parameter of content：x1, x2... xn；Respectively to n weights corresponding to the n parameter setting：w1, w2... wn；And point
It is other to n discrete function corresponding to the n parameter setting：f1(x1), f2(x2) ... fn(xn)；Personal upload is set to indicate
xp, individual's upload weights y；Transcoding priority P is set：
In distributed trans-coding system of the present invention, the n parameter comprise at least user clicks, recommendation degree,
Video maximum program request amount, resolution ratio and code check.
In distributed trans-coding system of the present invention, the Content Management System is additionally operable to preferential according to the transcoding
Level P sets transcoding queue, according to task described in the transcoding queue allocation.
In distributed trans-coding system of the present invention, the Content Management System is additionally operable to establish a Resources list,
Described the Resources list is used for the resource information and scheduling thresholds for recording one or more transcoding node in the transcoding server cluster；
It is additionally operable to extract the idle node according to described the Resources list.
In distributed trans-coding system of the present invention, the transcoding dispatch server also reports mould including resource information
Block；
The resource information reporting module, for being obtained according to default time interval one in the transcoding server cluster
To the resource service condition of multiple transcoding nodes.
On the other hand, there is provided a kind of distributed trans-coding method based on DTV, including：
The source contents file that matchmaker provides content is stored in a manner of section, it is big to configure the section that the source contents file is stored
Small and number of sections；
Issue scheduler task；
Read in the section of the source contents file；
Transcoding is carried out to the section according to the scheduler task；
The slice graph display after transcoding is controlled to form the complete file that matchmaker provides content；
Write and store the complete file that the matchmaker provides content.
A kind of distributed trans-coding system and method based on DTV disclosed above has the advantages that：Cluster
Soft transcoding capabilities enhancing, the transcoding demand of aggregation platform massive video is met, transcoding is not turned into the bottleneck of content distribution,
Without the participation of transcoding hardware, space and resource are saved to a certain extent, save cost.The idling-resource of VOD system
Effective utilization is obtained, lness ratio of plant is greatly reduced.The horizontal scalability of this framework is very high, it is necessary to during dilatation, substantially
Without raising single device hardware configuration, the equipment in any network can participate in operation at any time.Distributed trans-coding single device
Transcoding section is small, and scheduling is flexible, and Single Point of Faliure does not interfere with overall performance substantially.Fault-tolerance is very high.Conventional stream is broken away from
Formula contents processing, contents processing mode is more flexible and changeable, enhances the treatment effeciency of whole VOD system.
Brief description of the drawings
Fig. 1 is the structural representation of the DTV trans-coding system of prior art；
Fig. 2 is the coordinate diagram of digital cable customers session amount；
Fig. 3 is the schematic diagram of server disposition provided by the invention；
Fig. 4 is a kind of structural representation for distributed trans-coding system based on DTV that one embodiment of the invention provides
Figure；
Fig. 5 is a kind of structural representation for distributed trans-coding system based on DTV that another embodiment of the present invention provides
Figure；
Fig. 6 is the structured flowchart of the transcoding dispatch server provided by the invention；
Fig. 7 is a kind of flow chart of the distributed trans-coding method based on DTV provided in an embodiment of the present invention；
Fig. 8 is the flow chart of assignment transcoding task provided by the invention；
Fig. 9 is the flow chart of transcoding node scheduling provided by the invention.
Embodiment
In order to make the purpose , technical scheme and advantage of the present invention be clearer, it is right below in conjunction with drawings and Examples
The present invention is further elaborated.It should be appreciated that specific embodiment described herein is only to explain the present invention, not
For limiting the present invention.
The present invention provides a kind of distributed trans-coding system and method based on DTV, it is intended that will be network
In the remaining computing resource that changes over time effectively use, system deployment such as Fig. 3, Fig. 3 are server provided by the invention
The schematic diagram of deployment, the equipment that transcoding is participated in figure belong to existing cable television VOD system, wherein distribution buffer memory device is
CDN (content distributing network) caching, VSS (Video Stream Server) are video stream pushing server, ISS (Intemet
Stream Server) it is IP-based video stream pushing server, MAP is unified session management server, and USM is unified resource
Management server.And other configurable network equipments, these equipment are got up by network association, plus being just responsible for depositing originally
Storage and the cloud storage service device of transcoding, unify management and control by transcoding scheduling node active and standby each other, form the system deployment of the present invention.
Referring to Fig. 4, Fig. 4 is a kind of distributed trans-coding system 100 based on DTV that one embodiment of the invention provides
Structural representation, the distributed trans-coding system 100 includes content source storage server 1, content integrates storage server 4, logical
Cross the transcoding server cluster 3 and transcoding dispatch server 2 of cloud storage service device deployment；Wherein,
The content source storage server 1, the source contents file of content is provided for storing matchmaker in a manner of section；Content source
Storage server 1 virtualizes gained space by cloud storage, stores the source contents file not through transcoding, is stored in a manner of section,
Section work will be completed by cloud storage service device, and slice size and number of sections are configured by transcoding scheduling node.
The transcoding dispatch server 2, the slice size and number of sections stored for configuring the source contents file,
And issuing scheduler task to the transcoding server cluster 3 makes the section of the source contents file read in the transcoding server collection
Group 3；Slice graph display after control transcoding provides the complete file of content to form matchmaker, and is written into the integrated storage clothes of the content
It is engaged in device 4；There to be two-server active and standby deployment transcoding dispatcher software (i.e. transcoding dispatch server each other referring to Fig. 3, in network
2) role of scheduling, is served as.Function mainly has：
A) maintenance schedule resource pool, can the artificial pair of resource distribution that can be scheduled for intervene；
B) maintenance schedule system configuration table, manually the parameter configuration of transcoding dispatch server 2 can be intervened；
C) during current network conditions transit code inadequate resource, transcoding task is ranked, and need to safeguard transcoding queue.
D) transcoding priority generates, and sets, management.
E) file is cut into slices, the management and control of restructuring.
F) reporting policy management, information is reported.
The transcoding server cluster 3 includes one or more transcoding node, and one or more described transcoding node is used for foundation
The scheduler task carries out transcoding to the section；One or more transcoding node (i.e. transcoding node 1, transcoding node 2 in Fig. 1,
Transcoding node 3 ...) it is the server that transcoding software is deployed in network, the rule proposed by this patent, dispatched and saved by transcoding
Point assigned tasks read in section from content source storage and carry out transcoding.And by the integrated storage of section write-in content after transcoding.
The content integrates storage server 4, and the complete file of content is provided for storing the matchmaker.The integrated storage clothes of content
Device 4 of being engaged in virtualizes gained space by cloud storage, and before VOD dissemination systems are not entered into, content will be integrated in that content is integrated to deposit
Chu Zhong, restocking issue is waited, matchmaker's money content now needs to be merged into complete file, and it is complete by cloud storage service device to merge work
Into.
Referring to Fig. 5, Fig. 5 is a kind of distributed trans-coding system based on DTV that another embodiment of the present invention provides
Structural representation, the embodiment are different from upper embodiment part and are that the distributed trans-coding system also includes content distribution
Storage server 5 and Content Management System 6；Wherein,
The content distribution storage server 5, the complete file for the matchmaker to be provided to content are distributed to content delivery network
Network；Content distribution storage server 5 virtualizes gained space by cloud storage, and content is finally according to the A3 interface quilts in NGOD specifications
It is injected into content distribution storage, the CDN subsystems belonged in VOD system.
The Content Management System 6, the metadata of the source contents file of content is provided for matchmaker described in management and control, obtains the source
The section of content file simultaneously travels through one or more described transcoding node, and any one extracted in one or more described transcoding node is empty
Not busy node, so as to assigned tasks to the transcoding dispatch server 2.Content Management System 6：It is the module in existing VOD system,
It is responsible for the functions such as the metadata of management and control content, the storage of Scheduling content, examination ＆ verification, transcoding, issue.The Content Management System 6 is also
For establishing a Resources list, described the Resources list is used to record one or more transcoding node in the transcoding server cluster 3
Resource information and scheduling thresholds；It is additionally operable to extract the idle node according to described the Resources list.
Each transcoding task that Content Management System 6 (CMS) issues is to transcoding dispatch server 2, transcoding dispatch server 2
By cloud storage slice scheduling engine (i.e. cloud storage slice scheduling server 223), section component is called to split task,
It is subsequently sent to carry out transcoding on every transcoding node server, after the completion of transcoding, transcoding node can put forward transcoding result upwards
Transcoding dispatch server 2 is sent to, dispatch server recalls reconstitution component and each small task is re-assemblied, finally
To a complete program.
In a word, in Fig. 5, matchmaker provides content storage 401 into content source storage server 1, in content source storage server 1
Matchmaker provides content section reading 402 to each transcoding node, transcoding dispatch server 2 and controls 403 content source storage servers 1 to carry out
Deficient piece restructuring after section write-in and transcoding, write-in 404 to content integrates storage server 4 after the restructuring of transcoding node, in addition,
The idle transcoding node that transcoding dispatch server 2 is dispatched in 405 transcoding server clusters 3 carries out transcoding, and its scheduler task passes through
Content Management System 6 assigns 406, and the also matchmaker of management and control 407 of Content Management System 6 provides the storage of content, and complete file is finally from content
The integrated injection 408 of storage server 4 is into content distribution storage server 5.
Corresponding, other in Fig. 3 are configurable to be had transcoding between server and cloud storage service device (i.e. cloud storage 1-n) and cuts
There is also transcoding between the relation of piece read-write, cloud storage and content distributing network (distributing caching, VSS, ISS, MAP, USM) to cut
Piece is read and write, and other configurable server, cloud storage service device and content distributing networks are respectively to the (bag of transcoding dispatch server 2
Include active and standby two) resource management and control is carried out, transcoding dispatch server 2 responds resource management and control difference distribution scheduling task.
In summary, transcoding scheduling node needs configuration data as follows
1st, file header of cutting into slices identifies
Matchmaker provides section file and includes mark file header, is cut into slices for identifying, and data format is：
| Parameter name | Description |
| FileID | The complete file ID of section |
| SplitNumber | File number of sections |
| Offset | The deviation post of current slice |
| SerialNumber | The sequence number of current slice |
| IsIncomplete | Whether system minimum slice size is less than |
MinFileSize during IsIncomplete parameters configure according to Content Management System 6 is calculated and obtained.In distribution
In transcoding, file size if less than limiting value in the case of carry out section and can increase unnecessary read-write operation, and flow
To be excessively cumbersome, and reduce efficiency on the contrary, distributed trans-coding should not be carried out.This section can be related in both cases：
1, file size is less than setting value MinFileSize；2, file is cut into slices last a piece of.
2nd, transcoding resource pool
Scheduling node 21 safeguards resource pool MemberGroup { }, and it records the resource information that each Internet resources reports
And scheduling thresholds, main configuration parameter are as follows：
Each node is owned by threshold value and the busy setting of oneself, but actually each server ordinary circumstance difference in theory
Less, unified set can be taken.If paying close attention to some nodes, also can individually be configured.
3rd, system configuration table
The table name is SystemConfig, and main configuration parameters are as follows：
| Parameter name | Type | Description |
| SplitNumber | Unsigned | Configurable number of sections |
| MinFileSize | Unsigned | The minimal size of file section, in units of MB |
| BusyTime | Datetime | Single stand-by period when resource is all occupied |
| MaxAttempts | Unsigned | It is maximum when resource is all occupied to wait number |
| ReportTime | Datetime | Equipment reports information cycle |
| PriorityTable{} | Object | Priority calculation basis table |
4th, transcoding queue
Transcoding queue TranscodingQueue { }：Due to transcoding work existence time difference, when computing resource in network
It is in the case of deficiency, the transcoding queue in being stored by the maintenance content source of scheduling node 21, major parameter is as follows：
| Parameter name | Type | Description |
| TaskID | String | Task ID |
| FileID | String | File ID |
| SplitNumber | Unsigned | File number of sections |
| SourceFilePath{} | String | The store path of each section |
| TargetFilePath | String | File store path after the completion of section |
| Priority | Unsigned | Task priority |
| SourceFormat | String | Encoding video content form |
| TransCodingFormat | String | Video content transcoded format |
Transcoding priority can be by human configuration, or uses acquiescence, will be by priority computation module automatic assignment.
Referring to Fig. 6, Fig. 6 is the structured flowchart of the transcoding dispatch server 2 provided by the invention, the transcoding scheduling clothes
Business device 2 includes scheduling node 21 and section and reconstitution component 22；Wherein,
The scheduling node 21, for saving acquired section distribution to the free time extracted according to being assigned for task
The section is carried out transcoding by point, the idle node；The complete file write-in content of matchmaker money content is integrated and deposited
Store up in server 4；
The section and reconstitution component 22, for being cut into slices to the source contents file and to the section weight after transcoding
Group provides the complete file of content for matchmaker.It is described section with reconstitution component 22 include section sub-component 221, restructuring sub-component 222 with
And cloud storage slice scheduling server 223；
The section sub-component 221, the source contents file for providing content to the matchmaker be cut into section, and by institute
The section of cutting is back to the cloud storage slice scheduling server 223；
The restructuring sub-component 222, for being the complete file of matchmaker's money content to the slice graph display after transcoding, and by described in
Complete file is back to the cloud storage slice scheduling server 223；
The cloud storage slice scheduling server 223, for the section returned and complete file to be sent to described
Scheduling node 21.
In figure 6, work is performed between each structure in the following order：
601st, it is (main to receive CMS (also into transcoding scheduling node, its service interface externally provided) for transcoding dispatch server 2
To be CMS front end systems, referred to as " front end system ") issue transcoding task.
602nd, SplitNumber, MinFileSize parameter generation section in SystemConfig allocation lists are appointed
Business, is sent to slice scheduling engine (i.e. cloud storage slice scheduling server 223).
603rd, section task is issued to cloud storage some child node (i.e. transcoding node) and goes to perform by slice scheduling engine.
604th, child node calls section component to carry out a Virtual cropping (will not produce file fragmentation), and handle to file
The result of cutting returns to slice scheduling engine.
605th, the result of section is returned to transcoding scheduling node by slice scheduling engine, and the latter is analyzed result, note
Record, if accreditation section result, will confirm that message is sent to slice scheduling engine.
606th, file cutting is produced into actual burst by section sub-component 221.
607th, by transcoding scheduling node scheduling progress distributed trans-coding, (the subfile burst that transcoding is completed, which can store, specifies
Cloud storage catalogue in).
608th, transcoding progress is fed back to front end system by transcoding scheduling node in real time.
609th, after transcoding task is completed, transcoding scheduling node generation restructuring task is issued to slice scheduling engine.
610th, the subtask of merging is issued to some cloud storage child node by slice scheduling engine.
611st, child node calls reconstitution component, merges task and completes, the object information of completion is returned into slice scheduling and drawn
Hold up that (merging the destination file completed can store in specified cloud storage catalogue, and can be all clear by file fragmentation before
Except).
612nd, the task status of merging is returned to transcoding scheduling node by slice scheduling engine.
613rd, the situation end-state of transcoding is reported to front end system by transcoding scheduling node.
The transcoding dispatch server 2 also includes priority computation module；The priority computation module, for according to institute
The metadata for stating the source contents file that matchmaker provides content obtains the n parameter that the matchmaker provides content：x1, x2... xn,；Respectively to institute
State n weights corresponding to n parameter setting：w1, w2... wn；And respectively to n corresponding to the n parameter setting discrete letters
Number：f1(x1), f2(x2) ... fn(xn)；Personal upload is set to indicate xp, individual's upload weights y；Transcoding priority P is set：
The n parameter comprises at least user clicks, recommendation degree, video maximum program request amount, resolution ratio and code check.This
Outside, the Content Management System 6 is additionally operable to set transcoding queue according to the transcoding priority P, according to the transcoding queue point
Send the task.
Such as：Transcoding priority level initializing depends on the metadata information of content source, the user's program request amount referred here to, closes
Join video temperature, resolution ratio, code check etc., priority can be by following condition settings.
1st, keeper's setting value is preferential；
2nd, current source video network program request amount or recommendation degree are higher, and priority will be higher.This is directly obtained from network
Metadata information；
3rd, there is associated video, such as TV play in current source video, and program request amount of the associated video in VSS is higher, excellent
First level is higher；
4th, the resolution ratio of current source video and code check are higher, and priority is higher；
5th, the video uploaded for individual, priority will be set above all-network video；
If user clicks x1, recommendation degree x2, associated video maximum program request amount x3, resolution ratio x4, code check x5, individual, which uploads, to mark
Will xp{ value 0,1 }, priority P simply can be expressed as by formula：
fn(xn) it is a discrete function, value is found a function by a configurable table, such as：In the function of program request amount, if
Click volume constant C, there is following computational chart：
The y values in weights P and C and formula in above-mentioned table are all constant, can be configured on demand by keeper.Due to every
Kind parameter values difference is larger, needs exist for being set respectively for every kind of parameter, and one shares n discrete function
In addition, the transcoding dispatch server 2 also includes resource information reporting module；The resource information reporting module,
Resource for obtaining one or more transcoding node in the transcoding server cluster 3 according to default time interval uses feelings
Condition.I.e. scheduling node 21 safeguards resource distribution table MemberGroup by the resource information of each transcoding node of periodic receipt
{}.Strategy is：
1st, scheduling node 21 provides facility registration interface, and his URL will be in the transcoding node that every needs connect locally
Configuration, first communication will complete resource registering process by interface, and registration is completed corresponding transcoding node and is logged
In MemberGroup { }.Transcoding node need to distinguish cloud storage node and non-cloud memory node.
2nd, scheduling node 21 provides equipment and deletes interface, and application information or scheduling node 21 are sent actively by transcoding device
OPT-OUT NOTICE is sent, scheduling node 21 removes the nodal information, reports mechanism to terminate.The equipment is no longer transcoding node
2nd, scheduling node 21 must be configured to each threshold value of corresponding transcoding node and free time.If artificially do not enter
Row is set, then will use system default value.
3rd, transcoding nodes configuration parameters, such as ReportTime are handed down to by scheduling node 21.
4th, under normal circumstances：ReportTime of the transcoding node in SystemConfig carries out statistics own resource
Simultaneously heartbeat reports service condition.
5th, non-cloud transcoding node, the time beyond IdleTime_Start~IdleTime_End will not participate in transcoding and appoints
Business, will also be suspended without resource reporting, transcoding program, issue wake-up information by scheduling node 21 and information of stopping is called out
Wake up and stop.
6th, transcoding scheduling node manages the upper down status of each transcoding node, equipment active communication or starts to free time
To reach the standard grade, do not fed back in the stipulated time after periodic report time-out or transcoding mission dispatching, system will judge transcoding node off-line, and
No longer it is scheduled, untill equipment sends connection message again.
Referring to Fig. 7, Fig. 7 is a kind of flow of the distributed trans-coding method based on DTV provided in an embodiment of the present invention
Figure, the distributed trans-coding method comprise the following steps：
S1, the source contents file that matchmaker provides content is stored in a manner of section, configure that the source contents file stored cuts
Piece size and number of sections；
S2, issue scheduler task；
S3, the section for reading in the source contents file；
S4, according to the scheduler task to it is described section carry out transcoding；
S5, control transcoding after slice graph display with form matchmaker provide content complete file；
S6, write-in simultaneously store the complete file that the matchmaker provides content.
Wherein, for the issue scheduler task allocation strategy in step S2 referring to Fig. 8, Fig. 8 is assignment transcoding provided by the invention
The flow chart of task, the flow are as follows：
1st, in the case of no any task, scheduling node 21 is waited for.
2nd, when there is new task to assign in transcoding queue, the task that scheduling node 21 is taken out highest priority is carried out
Scheduling.
3rd, scheduling node 21 obtains file section from task.If section all have passed through processing, then can is clear
Except present completed task in queue, go to extract next task；If there is remaining section, next section is taken out.
4th, after section is got, scheduling node 21 goes to obtain transcoding node in resource pool, when all nodes are all traversed
And when meeting condition currently without node, scheduling node 21 can wait a time BusyTime to go to travel through again, the parameter
It can be configured in allocation list.If there is no idle node still, then wait a BusyTime.So repeatedly, repeatedly
MaxAttempts selects abandoning strategy after this here.Directly carry out wait state and start next task, reason is at this
There may be the higher task of priority to arrive in the section stand-by period.It is original to continue to arrange in queue without completing for task
Team, until being selected again.
5th, in previous step in any link, the transcoding node got first determines whether it is cloud storage node, if
Directly judge whether itself CPU and RAM resource meets idle demand for cloud node, if satisfied, transcoding task is dispatched into the node.
Because cloud node is without other business demands, transcoding work, the distribution therefore nothing takes time can be put into all strength.Rule of judgment is
CPULoad ＜ CPULoadThresholdValue^RAMUsed ＜ RAMUsedThresholdValue
If the 6th, the non-cloud storage cloud node of the node, then it be possible to for caching distribution, VSS, ISS, MAP, USM or other
Business device, in user's program request quantity higher period, it can not participate in transcoding task, therefore the current time need to be judged
Whether at one's leisure between scope.The time range is IdleTime_Start~IdleTime_End in configuration file.
For the detailed process of step S3-S6 shown in Fig. 7 referring to Fig. 9, Fig. 9 is the stream of transcoding node scheduling provided by the invention
Cheng Tu, after assigning transcoding task, transcoding node obtains task configuration information and slice information, obtain SourceFilePath and
TargetFilePath parameters, input and output can be carried out, referring to the step S1- in Fig. 9 to be read and write with cloud storage direct interaction
S4.But mainly there is the difference of three kinds of situations
1st, after transcoding success, transcoding node sends correct return to scheduling node 21, informs transcoding success, then will be transcoded into
The integrated storage of file write-in content after work(.And scheduling node 21 needs to cut into slices into rower to the file after the success of task transit code
Know, if task is interrupted if as the reason for inadequate resource, then need to know which section is when restarting task next time
Through completion.Referring to the step S5-S9 in Fig. 9.
If the 2, transcoding because some reasons do not succeed, transcoding node send mistake return, scheduling node 21 by this
Scheduling is cancelled, and cuts into slices also to will be returned in queue and be rescheduled.Referring to the step S10-S12 in Fig. 9.
If the 3rd, transcoding service is interrupted, or network interruption, then transcoding scheduling node can not still receive instead for a period of time
During feedback, it is believed that certain node off-line, at this moment scheduling node 21 reschedules the section, and the node device that this can not be fed back is set
To be offline.Referring to the step S13-S16 in Fig. 9.
By provided by the present invention, plug-flow phase can effectively be utilized according to user's program request rule of cable television VOD system
Close the strategy that equipment idle computing resources carry out transcoding.Meanwhile proposed by the framework in technical scheme and resource management and control strategy
Height it is open, the cable television VOD system architecture of Highly Scalable.In addition, it can also be entered by the scheduling strategy in technical scheme
Row transcoding task is cut into slices, and is lined up and is distributed to the realization of each node of cable television VOD system.
Embodiments of the invention are described above in conjunction with accompanying drawing, but the invention is not limited in above-mentioned specific
Embodiment, above-mentioned embodiment is only schematical, rather than restricted, one of ordinary skill in the art
Under the enlightenment of the present invention, in the case of present inventive concept and scope of the claimed protection is not departed from, it can also make a lot
Form, these are belonged within the protection of the present invention.
Claims (2)
1. a kind of distributed trans-coding system based on DTV, it is characterised in that including content source storage server, content set
The transcoding server cluster and transcoding dispatch server disposed into storage server, by cloud storage service device；Wherein,
The content source storage server, the source contents file of content is provided for storing matchmaker in a manner of section；
The transcoding dispatch server, the slice size and number of sections stored for configuring the source contents file, concurrently
Cloth scheduler task to the transcoding server cluster makes the section of the source contents file read in the transcoding server cluster；Control
Slice graph display after transcoding processed provides the complete file of content to form matchmaker, and is written into the content and integrates storage server
In；
The transcoding server cluster includes one or more transcoding nodes, and one or more described transcoding nodes are used for according to the tune
Degree task carries out transcoding to the section；
The content integrates storage server, and the complete file of content is provided for storing the matchmaker；
The distributed trans-coding system also includes content distribution storage server and Content Management System；Wherein,
The content distribution storage server, the complete file for the matchmaker to be provided to content are distributed to content distributing network；
The Content Management System, the metadata of the source contents file of content is provided for matchmaker described in management and control, obtains the source contents
The section of file simultaneously travels through one or more described transcoding nodes, extracts any one free time section in one or more described transcoding nodes
Point, so as to assigned tasks to the transcoding dispatch server；
The transcoding dispatch server includes scheduling node and section and reconstitution component；Wherein,
The scheduling node, for the idle node for extremely being extracted acquired section distribution according to being assigned for task, institute
State idle node and the section is subjected to transcoding；The complete file that the matchmaker is provided to content writes the integrated storage service of the content
In device；
The section and reconstitution component, for being cut into slices to the source contents file and being matchmaker to the slice graph display after transcoding
Provide the complete file of content；
The section includes section sub-component, restructuring sub-component and cloud storage slice scheduling server with reconstitution component；Wherein,
The section sub-component, the source contents file for providing content to the matchmaker be cut into section, and will be cut
Section is back to the cloud storage slice scheduling server；
The restructuring sub-component, for being the complete file of matchmaker's money content to the slice graph display after transcoding, and will the complete text
Part is back to the cloud storage slice scheduling server；
The cloud storage slice scheduling server, saved for the section returned and complete file to be sent to the scheduling
Point；
The transcoding dispatch server also includes priority computation module；
The priority computation module, obtained for providing the metadata of source contents file of content according to the matchmaker in matchmaker's money
N parameter of appearance：x1,x2,...xn；Respectively to n weights corresponding to the n parameter setting：w1,w2,...wn；It is and right respectively
N discrete function corresponding to the n parameter setting：f1(x1),f2(x2),...fn(xn)；Personal upload is set to indicate xp, it is individual
People uploads weights y；Transcoding priority P is set：
<mrow>
<mi>P</mi>
<mo>=</mo>
<mfrac>
<mrow>
<msubsup>
<mi>&Sigma;</mi>
<mn>1</mn>
<mi>n</mi>
</msubsup>
<msub>
<mi>w</mi>
<mi>n</mi>
</msub>
<msub>
<mi>f</mi>
<mi>n</mi>
</msub>
<mrow>
<mo>(</mo>
<msub>
<mi>x</mi>
<mi>n</mi>
</msub>
<mo>)</mo>
</mrow>
</mrow>
<mrow>
<msubsup>
<mi>&Sigma;</mi>
<mn>1</mn>
<mi>n</mi>
</msubsup>
<msub>
<mi>w</mi>
<mi>n</mi>
</msub>
</mrow>
</mfrac>
<mo>+</mo>
<msub>
<mi>x</mi>
<mi>p</mi>
</msub>
<mi>y</mi>
<mo>;</mo>
</mrow>
The n parameter comprises at least user clicks, recommendation degree, video maximum program request amount, resolution ratio and code check；
The Content Management System is additionally operable to set transcoding queue according to the transcoding priority P, according to the transcoding queue point
Send the task；
The Content Management System is additionally operable to establish a Resources list, and described the Resources list is used to record the transcoding server collection
The resource information and scheduling thresholds of one or more transcoding node in group；It is additionally operable to according to described the Resources list extraction idle section
Point；
The transcoding dispatch server also includes resource information reporting module；
The resource information reporting module, it is one to multiple in the transcoding server cluster for being obtained according to default time interval
The resource service condition of individual transcoding node.
A kind of 2. distributed trans-coding method based on DTV, it is characterised in that provide as claimed in claim 1 distributed
Trans-coding system, including：
The source contents file that matchmaker provides content is stored in a manner of section, configure slice size that the source contents file stored and
Number of sections；
Issue scheduler task；
Read in the section of the source contents file；
Transcoding is carried out to the section according to the scheduler task；
The slice graph display after transcoding is controlled to form the complete file that matchmaker provides content；
Write and store the complete file that the matchmaker provides content.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410683153.0A CN104378665B (en) | 2014-11-24 | 2014-11-24 | A kind of distributed trans-coding system and method based on DTV |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410683153.0A CN104378665B (en) | 2014-11-24 | 2014-11-24 | A kind of distributed trans-coding system and method based on DTV |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN104378665A CN104378665A (en) | 2015-02-25 |
| CN104378665B true CN104378665B (en) | 2018-02-27 |
Family
ID=52557268
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201410683153.0A Active CN104378665B (en) | 2014-11-24 | 2014-11-24 | A kind of distributed trans-coding system and method based on DTV |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN104378665B (en) |
Families Citing this family (24)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104837029B (en) * | 2015-05-08 | 2018-07-03 | 厦门大学 | A kind of storage of resource hard disk of HD video VOD system and access method |
| CN105354254A (en) * | 2015-10-21 | 2016-02-24 | 杭州施强网络科技有限公司 | Method for converting document file format by using node server |
| CN106686406B (en) * | 2015-11-05 | 2019-05-17 | 中国电信股份有限公司 | For realizing the pretreated method and apparatus of video real-time transcoding |
| CN105610823B (en) * | 2015-12-28 | 2018-09-11 | 武汉鸿瑞达信息技术有限公司 | A kind of the Streaming Media processing method and its processing system framework of task based access control vector |
| CN105763886A (en) * | 2016-03-01 | 2016-07-13 | 深圳市茁壮网络股份有限公司 | Distributed transcoding method and apparatus |
| CN108255871B (en) * | 2016-12-29 | 2022-01-28 | 华为技术有限公司 | Data query method and data query node |
| CN106791928A (en) * | 2016-12-29 | 2017-05-31 | 上海幻电信息科技有限公司 | The high performance video trans-coding system and method for a kind of self adaptation |
| CN106791932B (en) * | 2017-01-06 | 2019-08-30 | 广州亦云信息技术股份有限公司 | Distributed trans-coding system, method and device thereof |
| CN106878735A (en) * | 2017-01-18 | 2017-06-20 | 环球智达科技（北京）有限公司 | A kind of screen protection system of video compression coding |
| CN107743246A (en) * | 2017-01-24 | 2018-02-27 | 贵州白山云科技有限公司 | Task processing method, system and data handling system |
| CN107707592A (en) * | 2017-01-24 | 2018-02-16 | 贵州白山云科技有限公司 | Task processing method, node and content distributing network |
| CN107071449B (en) * | 2017-05-18 | 2019-12-03 | 北京数码视讯软件技术发展有限公司 | A kind of trans-coding system and method |
| CN107147921B (en) * | 2017-05-23 | 2020-11-03 | 北京网梯科技发展有限公司 | Video playing acceleration method and device based on slice and intelligent CDN scheduling |
| CN107302570B (en) * | 2017-06-09 | 2020-05-26 | 东华大学 | Equipment monitoring cloud component design method based on priority queue and Canvas technology |
| CN107295425B (en) * | 2017-06-12 | 2020-02-21 | 北京达沃时代科技股份有限公司 | Method for rapidly splicing transcoding fragmented files |
| CN107231565A (en) * | 2017-07-25 | 2017-10-03 | 网宿科技股份有限公司 | A kind of live task management dispatching method and its system |
| CN109413443B (en) * | 2017-08-16 | 2021-03-16 | 中国移动通信有限公司研究院 | Method and device for realizing time shifting function |
| CN110769278B (en) * | 2019-10-29 | 2022-02-08 | 鹏城实验室 | Distributed video transcoding method and system |
| CN110996172B (en) * | 2019-12-17 | 2022-01-11 | 杭州当虹科技股份有限公司 | Method for quickly synthesizing 4K MXF file |
| CN111163335B (en) * | 2019-12-31 | 2022-04-01 | 深圳市瑞讯云技术有限公司 | Video processing method and device for distributed server |
| CN112035405B (en) * | 2020-08-29 | 2023-10-13 | 平安科技（深圳）有限公司 | Document transcoding method and device, scheduling server and storage medium |
| CN113965775A (en) * | 2021-10-22 | 2022-01-21 | 武汉博晟安全技术股份有限公司 | Multimedia video safe playing method based on slice and intelligent CDN acceleration |
| CN114501067A (en) * | 2022-01-10 | 2022-05-13 | 成都易达数安科技有限公司 | Multi-GPU concurrent automatic scheduling method for media asset transcoding tasks |
| CN114745601B (en) * | 2022-04-01 | 2024-04-19 | 暨南大学 | Distributed audio and video transcoding system and method thereof |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101184209A (en) * | 2007-12-12 | 2008-05-21 | 中山大学 | VOD client terminal proxy caching server in digital family |
| CN101459848A (en) * | 2008-12-26 | 2009-06-17 | 西安交通大学 | Distributed multi-format digital video transcoding construction design method |
| US7650331B1 (en) * | 2004-06-18 | 2010-01-19 | Google Inc. | System and method for efficient large-scale data processing |
| CN102859961A (en) * | 2009-12-22 | 2013-01-02 | Gvbb控股股份有限公司 | Distributed video transcode system with adaptive file processing |
| CN103297485A (en) * | 2012-03-05 | 2013-09-11 | 日电（中国）有限公司 | Distributed cache automatic management system and distributed cache automatic management method |
| CN103838779A (en) * | 2012-11-27 | 2014-06-04 | 深圳市腾讯计算机系统有限公司 | Idle computing resource multiplexing type cloud transcoding method and system and distributed file device |
| CN104158909A (en) * | 2014-08-29 | 2014-11-19 | 北京阅联信息技术有限公司 | Distributed media processing method and system thereof |
-
2014
- 2014-11-24 CN CN201410683153.0A patent/CN104378665B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7650331B1 (en) * | 2004-06-18 | 2010-01-19 | Google Inc. | System and method for efficient large-scale data processing |
| CN101184209A (en) * | 2007-12-12 | 2008-05-21 | 中山大学 | VOD client terminal proxy caching server in digital family |
| CN101459848A (en) * | 2008-12-26 | 2009-06-17 | 西安交通大学 | Distributed multi-format digital video transcoding construction design method |
| CN102859961A (en) * | 2009-12-22 | 2013-01-02 | Gvbb控股股份有限公司 | Distributed video transcode system with adaptive file processing |
| CN103297485A (en) * | 2012-03-05 | 2013-09-11 | 日电（中国）有限公司 | Distributed cache automatic management system and distributed cache automatic management method |
| CN103838779A (en) * | 2012-11-27 | 2014-06-04 | 深圳市腾讯计算机系统有限公司 | Idle computing resource multiplexing type cloud transcoding method and system and distributed file device |
| CN104158909A (en) * | 2014-08-29 | 2014-11-19 | 北京阅联信息技术有限公司 | Distributed media processing method and system thereof |
Also Published As
| Publication number | Publication date |
|---|---|
| CN104378665A (en) | 2015-02-25 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN104378665B (en) | A kind of distributed trans-coding system and method based on DTV | |
| WO2018130162A1 (en) | Method and device for managing and controlling cloud computing terminal and operation of cloud server | |
| Wei et al. | QoS-aware resource allocation for video transcoding in clouds | |
| CN103164283B (en) | Virtualization resource dynamic dispatching management method and system in a kind of virtual desktop system | |
| WO2017113868A1 (en) | Method and system for self-adaptive bandwidth control for cdn platform | |
| CN107465708A (en) | A kind of CDN bandwidth scheduling systems and method | |
| CN109788315A (en) | Video transcoding method, apparatus and system | |
| CN102270363B (en) | Intelligent queuing system | |
| CN105007287A (en) | public cloud resource dynamic configuration method and system | |
| CN101938416A (en) | Cloud computing resource scheduling method based on dynamic reconfiguration virtual resources | |
| Zhang et al. | Toward transcoding as a service in a multimedia cloud: Energy-efficient job-dispatching algorithm | |
| CN104038392A (en) | Method for evaluating service quality of cloud computing resources | |
| CN103152393A (en) | Charging method and charging system for cloud computing | |
| CN111083519A (en) | VR content distribution system and method based on cloud and edge computing | |
| Rahman et al. | Edge computing assisted joint quality adaptation for mobile video streaming | |
| CN112804361B (en) | Edge alliance game method for content cooperation cache | |
| CN109656685A (en) | Container resource regulating method and system, server and computer readable storage medium | |
| CN103325168A (en) | Queuing and service system based on Bluetooth | |
| CN101702761A (en) | Multimedia video interactive method and system | |
| CN103281355A (en) | Fuzzy decision-based server push method | |
| CN101626305A (en) | Trusted dynamic level scheduling method for improving security of network environment | |
| CN104994150A (en) | Request allocation method oriented to cloud video service | |
| CN104994149A (en) | Cloud video service oriented resource provision method | |
| CN102611924B (en) | Flow control method and system of video cloud platform | |
| CN104753964B (en) | Networking products methods of exhibiting, apparatus and system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |