CN101707543A - Enterprise media bus system supporting multi-task type and enterprise media bus method supporting multi-task type - Google Patents
Enterprise media bus system supporting multi-task type and enterprise media bus method supporting multi-task type Download PDFInfo
- Publication number
- CN101707543A CN101707543A CN200910310703A CN200910310703A CN101707543A CN 101707543 A CN101707543 A CN 101707543A CN 200910310703 A CN200910310703 A CN 200910310703A CN 200910310703 A CN200910310703 A CN 200910310703A CN 101707543 A CN101707543 A CN 101707543A
- Authority
- CN
- China
- Prior art keywords
- task
- scheduling unit
- task scheduling
- path
- executing units
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention relates to an enterprise media bus system supporting multi-task type and an enterprise media bus method supporting multi-task type, namely an electronic computer network system and a data processing method. The system comprises a client, a buffer memory device, a database, a task scheduling unit, a plurality of task executing units, an application and interface server, a configuring tool and a BS system monitoring server, wherein the database, the task scheduling unit, the plurality of task executing units, the application and interface server, the configuring tool and the BS system monitoring server are connected with one another and are connected with the client through Ethernet networks. Because of the adoption of the task scheduling unit and the plurality of task executing units, the system supports various types of tasks, and supports on-line transmission, transcoding, technology review, computation and verification of MD5, and hybrid tasks of the subtasks. By splitting a task and detecting the service condition of a hardware system, the enterprise media bus system makes full use of the read and write bandwidth of a disk and CPU resource according to configurable task combination,, thereby greatly improving overall system performance.
Description
Technical field
The present invention relates to a kind of enterprise's media bus system and method for supporting the multitask type, being a kind of electronical computer network system and data processing method, is that a kind of enterprise that is used for TV station or other making and use digital television program is used for inner network system and the method that transmits the Television Digital media file.
Background technology
For an enterprise that has a large amount of digital media files to transmit mutually between inner all departments, the network system that is specifically designed to the digital document transmission of big capacity, high bandwidth is very necessary.Digital media file is not a simple carrying problem in the transmission of enterprises, is the complex process that needs provide various functions.With the TV station is example, and a TV station has a plurality of audio frequency related service systems of looking, each operation system inside, and major part all can have demands such as transcoding, the skill of looking audio file are examined, transmission.Look audio file when exchange between the operation system, more from not the above-mentioned functions demand, inner or other making of a therefore common TV station and use the enterprise of digital television program all can set up enterprise's media bus.
The media bus EMB of enterprise (Enterprise Media Bus) is that the bus architecture of audio file transmissions function is looked in a kind of main realization.Domestic and international existing EMB system or class EMB system are not very complete on systemic-function at present.Present existing EMB system has only supports transformation task, the support transcoding that has and two kinds of tasks of transmission.Existing EMB system does not support the mixed type task that independent skill is examined, independent MD5 calculates checking and multiple subtask (as: transcoding+skill is examined+MD5, transmission+skill is careful+MD5 etc.).It is the abbreviation of technological audit that skill is examined.It is objective evaluating at the picture signal technical quality of program that traditional skill is examined, and mainly refers to the test of indexs such as picture carrier, R-Y, B-Y amplitude and colour pase; Along with going deep into of TV station's digital improvement, the business such as adopting, compile, broadcast, manage, deposit of TV station has entered digital Age, program and material also exist with the form of digital file mostly, examine except traditional skill is examined content at the skill of digital file, also the compressed digital of file being looked audio content verifies, comprise: the checking of the checking of file structure aspect, compression and encoding context, file are looked the checking of audio content, thereby seem more complicated.Existing EMB system does not support to comprise in the task situation that transcoding, transmission, skill such as examine at a plurality of independent subtasks yet, needs to submit to separately for a plurality of dissimilar subtasks like this, uses very inconvenient.If the file destination that the source file that the machine that has in the EMB system can only access task, the machine that has can only access tasks, but do not have the machine source and target file of access task simultaneously, for this situation, existing at present EMB system can not handle.A physical file may have multiple different access mode: Ethernet file-sharing mode, optical fiber access mode, FTP access mode, the priority difference of different access mode.When a kind of access mode occurs when wrong, the user wishes that the EMB system can select other access modes automatically.Existing EMB system does not support the path load equilibrium under this complex network environment.Along with the high resolution video and audio application is more and more, the file volume is increasing, and file format becomes increasingly complex various, the EMB task is carried out efficient have higher requirement, and task executions efficient need improve in the existing EMB system.The disk read-write bandwidth of high-performance storage networking can be very high, when carrying out transformation task in the EMB system, if the transmission speed of task is not limited, may take very high readwrite bandwidth.Total bandwidth is limited, and the operable bandwidth of other operation systems will significantly reduce like this, thereby influences the normal use of whole service system.Existing EMB system need realize transformation task speed limit function according to the demand of using bandwidth, to guarantee the EMB system bandwidth is reasonably taken.
Summary of the invention
For solving prior art problems, the objective of the invention is to propose a kind of enterprise's media bus system and method for supporting the multitask type.Described system and method is realized supporting the multitask type, and is made the present invention make full use of resource by intelligentized design by the fractionation to task, carries out various tasks fast and efficiently.
The object of the present invention is achieved like this: a kind of media bus system of enterprise that supports the multitask type, the media bus system of enterprise of described support multitask type comprises: task is submitted client to, buffer storage, database, the task scheduling unit, a plurality of task executing units, use and interface server, configuration tool, BS system monitoring server, described database, the task scheduling unit, a plurality of task executing units, use and interface server, configuration tool, BS system monitoring server via Ethernet interconnects and submits to client to be connected with task, and described task executing units is connected with the memory device of each system that need carry out the digital media file exchange.
Use a kind of enterprise's media transmission method of supporting the multitask type of said system, the step of described method is as follows:
Add the step of data transfer task: be used for task and submit to client to propose data transfer task, and the task scheduling unit is added data transfer task by application and interface server;
Data transfer task is analyzed the step that splits with one-level: be used for the task scheduling unit transfer of data task definition and demand are analyzed, task definition comprises: independent transcoding, transmission, skill are examined the MD5 checking, or mix multiple subtask: transcoding adds that skill is examined, transcoding adds skill and examines and add MD5, transcoding and add MD5, transmission and add that transcoding, transmission add MD5, transmission adds transcoding and adds MD5; To the task that mix multiple subtask, the one-level that system carries out task splits, and is split as the subtask of a plurality of single task role types automatically;
The step that task scheduling unit task scheduling begins: after being used for the success initialization of task scheduling unit, the beginning duty cycle;
Whether detect the step whether idle task executing units is arranged: whether be used for the detection of task scheduling unit has the task of idle task executing units and each task executing units full, if "No" then enter resting state, " step that task scheduling unit task scheduling begins " got back in dormancy after a period of time, if "Yes" then carry out next step;
The step whether waiting task is arranged: be used for the task scheduling unit and detect whether waiting task is arranged, if "No" then enter resting state, " step that task scheduling unit task scheduling begins " got back in dormancy after a period of time, if "Yes" then carry out next step;
The balanced step that detects of path load: each path that is used for may using task the task scheduling unit is detected, and the parameter of detection comprises: path priority level, speed-limiting messages;
Judge whether the step in operable path: be used for the task scheduling unit and use the requirement in path and the operating position of existing route to compare to task, judged whether operable path, if "No" then enter resting state, " step that task scheduling unit task scheduling begins " got back in dormancy after a period of time, if "Yes" then carry out next step;
Task is carried out the step that secondary splits: be used for the task scheduling unit task or the subtask that the one-level task splits the single task role type that obtains split, comprise: FTP splits, middle route splits, distributed fractionation, so that according to the actual performance element and the path of the state assignment task of path and task executing units;
The step of coupling path and task executing units: be used for the access ability of task scheduling unit, select to be fit to carry out through splitting the various piece path and the task executing units of task according to task executing units and path;
The step of Processing tasks: be used for task executing units task is handled, in the task implementation, according to the progress report time interval and the percentage-proportion intervals of configuration, to task scheduling unit Report Tasks progress;
The step of updating task state: be used for the information that the task scheduling unit receives that the Task Progress report updating task of task executing units is carried out;
Task is carried out the step of end and result's readjustment: finish in the task executing units task and carry out the back to task scheduling unit Report Tasks execution result, the task scheduling unit is by the task result readjustment service of application and interface server calling task submission side, readjustment task action result;
Judge whether the step that manually stops: be used to judge whether manually stop the task scheduling circulation, if "No" then get back to " step that task scheduling unit task scheduling begins ", if "Yes" then stop task scheduling;
Query task result's step: be used for task submission side and call application and interface server, query task implementation progress and execution result, this step and " judge whether manually stop step " parallel execution, and can carry out in each process in " step of Processing tasks " and carry out.
The beneficial effect that the present invention produces is: owing to be provided with buffer storage, database, task scheduling unit, a plurality of task executing units, the present invention supports polytype task, be supported in that transmission, transcoding, skill are examined, calculating and the checking of MD5, and the hybrid task of these tasks (transcoding+skill is examined+MD5, and transmission+skill is careful+MD5 etc.).The present invention splits and the path dynamic load equilibrium technology by dynamic task, and the task under the complex network environment is handled good support.The present invention makes up according to configurable task, supports the dissimilar a plurality of tasks of the concurrent execution of unit, has made full use of disk read-write bandwidth and cpu resource, and overall system performance is improved a lot.The present invention with the distributed executive mode of single transcoding task multimachine device, makes full use of the clustering functionality of system by the fractionation task, has improved the efficient of transcoding task greatly.The present invention supports transformation task speed limit function by the configuration of detection dish battle array and memory block speed limit, make the present invention to the bandwidth of operation system in controlled range, avoided high bandwidth to take the harmful effect that operation system is produced.
Description of drawings
The invention will be further described below in conjunction with drawings and Examples.
Fig. 1 is the embodiment of the invention one described a kind of enterprise's media bus system principle diagram of supporting the multitask type;
Fig. 2 is the embodiment of the invention five described a kind of flow charts of supporting enterprise's media transmission method of multitask type;
Fig. 3 carries out sub-process figure in the step that secondary splits to task in the described a kind of enterprise's media transmission method of supporting the multitask type of the embodiment of the invention six.
Embodiment
Embodiment one:
Present embodiment is a kind of media bus system of enterprise that supports the multitask type, as shown in Figure 1.The media bus system of enterprise of the described support multitask of present embodiment type comprises: task is submitted client to, buffer storage, database, the task scheduling unit, a plurality of task executing units, use and interface server, configuration tool, BS system monitoring server, described database, the task scheduling unit, a plurality of task executing units, use and interface server, configuration tool, BS system monitoring server via Ethernet interconnects and submits to client to be connected with task, and described task executing units is connected with the memory device of each system that need carry out the digital media file exchange.
Enterprise's media bus (Enterprise Media Bus, the English EMB that is called for short) system is a kind of based on bus architecture, to concentrate looking the system that audio frequency is handled. the characteristics of bus architecture are to be responsible for the unified of other each operation system data handled by a centring system, rather than handle by the direct exchange of each operation system. under the bus architecture, each operation system only exchanges with centring system, pass between each system mutually, newly-increased or deletion operation system does not influence other system, therefore bus architecture has reduced the coupling between the operation system, simplified the deployment and the maintenance of system. present embodiment is a bus architecture, each operation system is coupled together, unification is handled looking audio frequency media, finish the mutual and transmission of media file between each operation system, main processing mode comprises the transmission of looking audio file, transcoding between the different file formats, and skill is examined automatically, .MD5 such as MD5 verification are a kind of disclosed data summarization algorithms, by this algorithm, if the file of each different content can obtain a different MD5 value. file has any modification, the MD5 value all can change. therefore, whether whether the MD5 verification is can authenticating documents complete, be modified.
The effect of the described system of present embodiment each several part:
EMB database: mission bit stream, task executing units and the task scheduling unit information of storage whole system, and system parameters configuration.
Task scheduling unit (Manager): be responsible for task is carried out dynamic resolution and scheduling.The task scheduling unit can be arranged on and form the task scheduling server in the independent server, or shares a server with task executing units.An independent task scheduling server both can be set in an EMB system, the backup server of task scheduling server also can be set outside the task dispatch server, two or more task scheduling servers form heat and are equipped with, and realize the high availability of system.The task scheduling unit further is embedded in the task executing units, can simplified system dispose, and improves Fault Tolerance.Present embodiment is an intelligent system, can determine the process of transmission according to the ruuning situation of system, utilizes resource to greatest extent, efficiently executes the task with the fastest speed.The task scheduling unit plays critical effect in the present embodiment, the path of the continuous detected transmission media file in task scheduling unit, the running status of storage system, by analysis and fractionation to task, find best task execution path, it is efficient that realization is executed the task.
Task executing units (Actor): task executing units is connected with the memory device of each system that need carry out the digital media file exchange, be responsible for carrying out the EMB task, read the source media file of task, as required to goal systems write media file, carry out that automatic skill is examined, transcoding or MD5 verification, and Report Tasks progress and result.
Use and the external interface server: dispose the WebService interface module of the described system of present embodiment, and the monitoring of BS is used.The WebService interface module is deployed in the service invocation request that the task of external system submits to client to submit to except acceptance division, and present embodiment is also realized the loopback of task result by this module.
Configuration tool: information such as responsible operational factor to system, store path, task executing units, task scheduling unit are configured.
The BS system monitoring: present embodiment provides the BS monitoring based on Web, and the user can check the running status of EMB system by browser.BS is the abbreviation of Browser Server structure, is based on a kind of system design deployment and the operational mode of browser-server architecture.Supervisory control system is the BS framework, and APD is deployed on the application server, and the BS monitor client only need just can be visited by browser, no longer needs to install other softwares.This software architecture has been simplified the installment work of BS monitor client, is convenient to software upgrading, and is also user-friendly, and supports by Internet remote access.
Buffer storage:, do not have buffer storage can't realize that then middle route splits and FTP splits for route fractionation and FTP fractionation task in the middle of carrying out provide buffer memory.
Embodiment two:
Present embodiment is the improvement of embodiment one, is that embodiment one is about task scheduling unit and task executing units refinement.Present embodiment described task scheduling unit and a plurality of task executing units are separately positioned in separately the server, form task scheduling server and a plurality of task and carry out server.
Task scheduling unit and task executing units all are arranged in separately the server in the present embodiment, that is to say and arrange a task executing units in the server, perhaps task scheduling unit. the communication that task is carried out between server and the task scheduling server can be adopted other modes such as MQ, Socket. and the advantage of present embodiment setting is that hardware requirement is low, but disposes complicated.
Embodiment three:
Present embodiment is the improvement of embodiment two, is the refinement of embodiment two about the task scheduling server.The described task scheduling server of present embodiment has standby task scheduling server, is connected by heartbeat between described task scheduling server and the standby task scheduling server.
Because the key effect of task scheduling unit, and the task scheduling unit is arranged on separately in the station server, in a single day this server goes wrong, for example crash, paralysis will appear in whole business data transmission system, and therefore for improving fault-tolerant ability, present embodiment is provided with task scheduling server standby host, a task scheduling unit promptly is set in another station server again, forms task scheduling server host and task scheduling server standby host.The hardware of task scheduling server standby host and task scheduling server host, software systems dispose just the same, have only increased switching device shifter.
Task scheduling server standby host and task scheduling server host be new data more synchronously, just can not occur the interruption that any task is carried out like this when switching.Task scheduling server host and task scheduling server standby host are a kind of addresses, promptly be called the task scheduling server host at just at work task scheduling server, the task scheduling server that is in standby condition is called task scheduling server standby host, after switching, task scheduling server standby host originally becomes the task scheduling server host, and task scheduling server host originally becomes task scheduling server standby host.
Detect by heartbeat between task scheduling server host and the task scheduling server standby host, check whether the task scheduling server host survives.If the task scheduling server host occurs unexpected, task scheduling server standby host will oneself be updated to the task scheduling server host by switching.
For increasing reliability, present embodiment can also be provided with many task scheduling server standby hosts, on each task scheduling server standby host a task scheduling unit is arranged respectively, these task scheduling server standby hosts realize having only a task scheduling server standby host can be updated to the task scheduling server host by the database locking mechanism.
Embodiment four:
Present embodiment is the improvement of embodiment one, is the refinement of embodiment one about the task scheduling unit.The described task scheduling of present embodiment unit is embedded in the task executing units.
The task scheduling unit of present embodiment is embedded in the task executing units, existing task executing units in the server so just occurs, and the situation of task scheduling unit is also arranged.This mode no longer needs to dispose independently task scheduling server, has reduced the system deployment complexity.Owing to can embed the task schedulings unit, that is to say to have occurred that the task scheduling unit is arranged in a plurality of servers that standby host each other between the task scheduling unit has strengthened fault-tolerant ability in each server in a plurality of task executing units.
After in a plurality of task executing units, embedding the task scheduling unit, a plurality of task schedulings unit occurs, one of them can be set be the main task scheduling unit.After the server at place is made mistakes, other any one have a task scheduling unit server can work on as standby task scheduling unit, strengthened Fault Tolerance.
Embodiment five:
Present embodiment is to use a kind of enterprise's media transmission method of supporting the multitask type of the described system of the foregoing description, and flow process as shown in Figure 2.
Present embodiment is by the mode of tree-shaped task structure, task fractionation, task plug-in unit, support transcoding, transmission, independent skill to examine, independent MD5 calculates and the hybrid task of the task of types such as checking and these subtasks (transcoding+skill is examined+MD5, transmission+skill is careful+MD5 etc.).And can also support more task type by the mode of increase task plug-in unit.
Dynamically carry out because task splits, adopt the task tree structure, can realize the task dynamic resolution easily, tree is also very directly perceived on displaying interface, is convenient to the user and understands.Describe the task tree structure below in detail: each task all has a root node, essential informations such as the title of description task, submission person, submission time, specifying informations such as the source file of task, file destination are described in child node, have only the afterbody node (leaf node) in the task tree to be only finally can directly carrying out of task.For example, the user has submitted a hybrid task that comprises transmission and two task types of transcoding simultaneously to, can generate a task root node and transmission, two task child nodes of transcoding automatically in the system.System is when handling transformation task, may find that this transformation task need carry out the centre route and split, so for the transformation task child node, can be split as the subtask of two next stage automatically: source path is to the subtask of buffer storage, and buffer storage is to the subtask of destination path.When system handles the transcoding task, may find that need carry out FTP to this task splits, so for transcoding task child node, can be split as the subtask of two next stage automatically: from the copy subtask of source FTP to middle buffer memory, middle buffer memory is transcoded onto the destination path subtask.(if destination path also is FTP, can be split as three subtasks).In the present embodiment, the structure of final task tree is as follows:
The task root node
The transmission subtask
Source path is to the subtask of buffer storage
Buffer storage is to the subtask of destination path
The transcoding subtask
Source FTP is to the copy subtask of middle buffer memory
Middle buffer memory is transcoded onto the destination path subtask
Present embodiment can adopt tree-shaped task structure, in the task implementation, according to system configuration, task is split, and dynamically generates new subtask node, then new subtask is handled.Present embodiment has disposed the multiple path access mode of each physical storage area, comprises Ethernet file-sharing mode, optical fiber access mode, FTP access mode etc., and every kind of access mode can be provided with priority, maximum number of connections.Which task executing units can also be set visit which path, so just can know the corresponding relation between each path and each task executing units.
If the file destination that the source file that the server that has in the system can only access task, the server that has can only access tasks, but there is not the station server source and target file of access task simultaneously.For having disposed buffer storage in this situation present embodiment, present embodiment is split as the source to this task to buffer storage automatically, carry out the subtask that buffer storage is carried out to two orders of target, thereby an original task node is split out two sub-task nodes, and route split in the middle of this task dynamic resolution was called.
A physical file may have multiple different access mode: Ethernet file-sharing mode, optical fiber access mode, FTP access mode, the priority difference of different access mode.When a kind of access mode occurs when wrong, the user wishes that the EMB system can select other access modes automatically.Existing EMB system does not support the path load equilibrium under this complex network environment.At this situation, present embodiment is provided with the task scheduling unit.The task scheduling unit is when carrying out task scheduling, according to the corresponding relation between path priority, the current linking number in path, path maximum number of connections, path and the task executing units, calculate optimum path and the task executing units of mating is most executed the task.Increase the current path linking number before executing the task, the current path linking number is reduced in the back of executing the task, and has realized the dynamic load leveling in path like this.
Especially, for the transcoding task, after the path load equilibrium, when all available paths all comprise the FTP path, do not support the form of the direct transcoding of FTP for some, the task scheduling unit can carry out FTP task dynamic resolution.FTP task dynamic resolution is divided into following three kinds of situations:
If just the source is FTP, can be split as task from the source and copies buffer storage+buffer storage to two subtasks of target transcoding.
If just target is FTP, can be split as task and be transcoded onto buffer storage+buffer storage from the source and copies two subtasks of target to.
If source and target all is FTP, can be split as task and copies buffer storage+buffer storage from the source to and be transcoded onto buffer storage+buffer storage and copy three subtasks of target to.
Present embodiment can be supported the dissimilar a plurality of tasks of the concurrent execution of unit according to configurable task combination, has made full use of disk read-write bandwidth and cpu resource, and overall system performance is improved a lot.The task of general transport-type can take a large amount of disk read-write bandwidth, but only takies very little cpu resource.Transcoding type task is just opposite, can take a large amount of cpu resources, but few for bandwidth occupancy.Present embodiment can make up task type according to configuration, allows transport-type task and transcoding type task carry out simultaneously on a machine, can farthest utilize disk read-write bandwidth and cpu resource like this, improves the whole efficiency of system.
Present embodiment can make full use of the clustering functionality of a plurality of task executing units by the distributed execution technique of single transcoding task multimachine device because a plurality of task executing units are arranged, and has improved the efficient of transcoding task greatly.
Because the transcoding task needs the long period usually, for the very high transcoding task of some priority, the user wishes to finish the work as early as possible, and present embodiment then uses a plurality of task executing units this task of executed in parallel simultaneously, and the transcoding task is finished rapidly.
Present embodiment splits the realization distributed trans-coding by the distribution of task.Task is carried out distributed fractionation, each transcoding task is carried out simultaneously by many task executing units servers, carry out the automatic merging of file destination in the implementation, this mode has made full use of the task executing units clustering functionality, has improved transcoding task executions efficient greatly.
Present embodiment is supported transformation task speed limit function by dish battle array and the configuration of memory block speed limit, make present embodiment to the bandwidth of operation system in controlled range, avoided high bandwidth to take harmful effect to the operation system generation.
Present embodiment is in operation and detects the spendable bandwidth in path, in the hope of formulating best task implementation strategy according to existing bandwidth resources.On the memory block, increased the notion of dish battle array.Dish battle array and memory block are the relations of one-to-many.Can on the dish battle array, total speed limit bandwidth be set, acquiescence single task maximum bandwidth, three parameters of acquiescence single task minimum bandwidth.In the memory block below the dish battle array,, single task maximum bandwidth and single task minimum bandwidth can be set also in each path owing to a memory block may have the multiple path of different access mode.The task scheduling unit of present embodiment is when executing the task scheduling, except carrying out the path dynamic load leveling according to path priority and linking number, also according to the maximum speed limit parameter of coiling battle array and path, and when shroud battle array utilized bandwidth, calculate the maximum available bandwidth of this task.If this task maximum available bandwidth is less than the task minimum bandwidth that is provided with, this task wouldn't be carried out, wait other inter-related tasks complete after, when satisfying bandwidth speed limit condition, carry out this task again.The utilized bandwidth of increase dish battle array before carrying out, the utilized bandwidth of minimizing dish battle array after carrying out.By this speed limit mechanism, make present embodiment to the bandwidth occupancy of operation system in controlled range, avoided high bandwidth to take the harmful effect that operation system is produced.
The step of the described method of present embodiment is described below:
Add the step of data transfer task: be used for task and submit to client to propose data transfer task, and the task scheduling unit is added data transfer task by application and interface server.Present embodiment adds task interface and can be set to support tree-shaped task interface, uses tree-shaped task interface can realize complicated task execution sequence.
Data transfer task is analyzed the step that splits with one-level: be used for the task scheduling unit transfer of data task definition and demand are analyzed, task definition comprises: independent transcoding, transmission, skill is examined the MD5 checking, or mix multiple subtask: transcoding adds skill to be examined, transcoding adds the careful MD5 that adds of skill, transcoding adds MD5, transmission adds transcoding, transmission adds MD5, transmission adds the basic task type that transcoding adds the support of MD5. present embodiment and comprises transcoding, transmission, independent skill is examined, independent MD5 calculates checking etc. also support the task of multiple subtask type hybrid, examine+MD5 as transcoding+skill, transmission+skill examines+MD5 etc. because the complexity of task, simple processing can't be carried out the task of such complexity, therefore at first must analyze task, if task has comprised a plurality of task types simultaneously, the one-level of then carrying out task splits, and task is split as the subtask of a plurality of single task role types.
The step that task scheduling unit task scheduling begins: after being used for the success initialization of task scheduling unit, the beginning duty cycle.
Whether detect the step whether idle task executing units is arranged: whether be used for the detection of task scheduling unit has the task of idle task executing units and each task executing units full, if "No" then enter resting state, " step that task scheduling unit task scheduling begins " got back in dormancy after a period of time, if "Yes" then carry out next step; Being not only that whole task executing units is idle in this step just thinks the free time to think that also this task executing units is idle when the task of task executing units is full inadequately.
The step whether waiting task is arranged: be used for the task scheduling unit and detect whether waiting task is arranged, if "No" then enter resting state, " step that task scheduling unit task scheduling begins " got back in dormancy after a period of time, if "Yes" then carry out next step.
The balanced step that detects of path load: each path that is used for may using task the task scheduling unit is detected, and the parameter of detection comprises: path priority level, speed-limiting messages.
Judge whether the step in operable path: be used for the task scheduling unit and use the requirement in path and the operating position of existing route to compare to task, judged whether operable path, if "No" then enter resting state, " step that task scheduling unit task scheduling begins " got back in dormancy after a period of time, if "Yes" then carry out next step.
Task is carried out the step that secondary splits: be used for the task scheduling unit secondary fractionation is carried out in task or subtask that the one-level task splits the single task role type that obtains, comprise: FTP splits, middle route splits, distributed fractionation, so that according to the actual performance element and the path of the state assignment task of path and task executing units.
The step of coupling path and task executing units: be used for the access ability of task scheduling unit, select to be fit to carry out through splitting the various piece path and the task executing units of task according to task executing units and path.
The step of Processing tasks: be used for task executing units task is handled, in the task implementation, according to the progress report time interval and the percentage-proportion intervals of configuration, to task scheduling unit Report Tasks progress.
The step of updating task state: be used for the information that the task scheduling unit receives that the Task Progress report updating task of task executing units is carried out, so that the task scheduling unit is other Task Distribution resources.
Task is carried out the step of end and result's readjustment: finish in the task executing units task and carry out the back to task scheduling unit Report Tasks execution result, the task scheduling unit is by the task result readjustment service of application and interface server calling task submission side, readjustment task action result.
Judge whether the step that manually stops: be used to judge whether manually stop the task scheduling circulation, if "No" then get back to " step that task scheduling unit task scheduling begins ", if "Yes" then stop task scheduling.
Query task result's step: be used for task submission side and call application and interface server, query task implementation progress and execution result, this step and " judge whether manually stop step " parallel execution, and can carry out in each process in " step of Processing tasks " and carry out.
Embodiment six:
Present embodiment is the improvement of embodiment five, be embodiment five about task is carried out the refinement of secondary splitting step, flow chart is as shown in Figure 3.
Present embodiment is described, and that task is carried out the substep that step that secondary splits comprises is as follows:
Need to judge whether the substep of FTP fractionation: be used for judging whether the transcoding task must comprise the FTP path, and determine whether needs carry out FTP and split according to looking audio format configuration, if "Yes" then enter " FTP split substep ", if "No" then enter " substep that route splits in the middle of whether needing ".
The substep that FTP splits: be used to utilize buffer storage, whether according to source or target is the FTP path, and a FTP transcoding task is split as 2 or 3 subtasks: see middle buffer memory transcoding to copying of middle buffer memory from source FTP, behind the transcoding to the copy of target FTP, updating task tree structure simultaneously.
Whether the substep whether in the middle of needing route splits: being used to check has the task scheduling unit of access originator and destination path simultaneously, if not then need to carry out centre route fractionation.Route splits and then to enter " substep that middle route splits " if desired, if route splits then enters " whether needing distributed fractionation substep " in the middle of not needing.
The substep that middle route splits: utilize buffer storage, a task is split as two subtasks: source path subtask in the buffer storage, buffer storage is to the subtask of destination path.
The substep that whether needs distributed fractionation: for the transcoding task, option during according to the task of submission to, task priority, the task executing units quantity factor of current free time, whether decision carries out distributed fractionation, if "Yes" then enter " substep of distributed fractionation ", if "No" then enter " step of coupling path and task executing units ".
The substep of distributed fractionation:, a transcoding task torn open to a plurality of transcodings subtask add an automatic merging task according to the quantity of present idle task performance element; There is a fragment sequence number each transcoding subtask, be responsible for generating a fragment of file destination,, merge task automatically by the concurrent execution of a plurality of task executing units, can be according to the fragment numeric order, the fragment that transcoding is finished is in time merged into the final objective file.
It should be noted that at last, below only unrestricted in order to technical scheme of the present invention to be described, although the present invention is had been described in detail with reference to the preferred arrangement scheme, those of ordinary skill in the art is to be understood that, can be to technical scheme of the present invention (such as the bus form that system uses, the configuration of each service, connected mode, the front and back sequential scheduling of each step) makes amendment or be equal to replacement, and do not break away from the spirit and scope of technical solution of the present invention.
Claims (6)
1. media bus system of enterprise that supports the multitask type, described bus system comprises: task is submitted client to, buffer storage, database, the task scheduling unit, a plurality of task executing units, use and interface server, configuration tool, BS system monitoring server, described database, the task scheduling unit, a plurality of task executing units, use and interface server, configuration tool, BS system monitoring server via Ethernet interconnects and submits to client to be connected with task, and described task executing units is connected with the memory device of each system that need carry out the digital media file exchange.
2. according to the described bus system of claim 1, it is characterized in that described task scheduling unit and a plurality of task executing units are separately positioned in separately the server, form task scheduling server and a plurality of task and carry out server.
3. bus system according to claim 2 is characterized in that, described task scheduling server has standby task scheduling server, is connected by heartbeat between described task scheduling server and the standby task scheduling server.
4. according to the described bus system of claim 1, it is characterized in that described task scheduling unit is embedded in the task executing units.
5. use a kind of enterprise's media transmission method of supporting the multitask type of the described system of claim 1, it is characterized in that the step of described method is as follows:
Add the step of data transfer task: be used for task and submit to client to propose data transfer task, and the task scheduling unit is added data transfer task by application and interface server;
Data transfer task is analyzed the step that splits with one-level: be used for the task scheduling unit transfer of data task definition and demand are analyzed, task definition comprises: independent transcoding, transmission, skill are examined, the MD5 checking, or mix multiple subtask: transcoding adds that skill is examined, transcoding adds skill and examines and add MD5, transcoding and add MD5, transmission and add that transcoding, transmission add MD5, transmission adds transcoding and adds MD5; To the task that mix multiple subtask, the one-level that system carries out task splits, and is split as the subtask of a plurality of single task role types automatically;
The step that task scheduling unit task scheduling begins: after being used for the success initialization of task scheduling unit, the beginning duty cycle;
Whether detect the step whether idle task executing units is arranged: whether be used for the detection of task scheduling unit has the task of idle task executing units and each task executing units full, if "No" then enter resting state, " step that task scheduling unit task scheduling begins " got back in dormancy after a period of time, if "Yes" then carry out next step;
The step whether waiting task is arranged: be used for the task scheduling unit and detect whether waiting task is arranged, if "No" then enter resting state, " step that task scheduling unit task scheduling begins " got back in dormancy after a period of time, if "Yes" then carry out next step;
The balanced step that detects of path load: each path that is used for may using task the task scheduling unit is detected, and the parameter of detection comprises: path priority level, speed-limiting messages;
Judge whether the step in operable path: be used for the task scheduling unit and use the requirement in path and the operating position of existing route to compare to task, judged whether operable path, if "No" then enter resting state, " step that task scheduling unit task scheduling begins " got back in dormancy after a period of time, if "Yes" then carry out next step;
Task is carried out the step that secondary splits: be used for the task scheduling unit secondary fractionation is carried out in task or subtask that the one-level task splits the single task role type that obtains, comprise: FTP splits, middle route splits, distributed fractionation, so that according to the actual performance element and the path of the state assignment task of path and task executing units;
The step of coupling path and task executing units: be used for the access ability of task scheduling unit, select to be fit to carry out through splitting the various piece path and the task executing units of task according to task executing units and path;
The step of Processing tasks: be used for task executing units task is handled, in the task implementation, according to the progress report time interval and the percentage-proportion intervals of configuration, to task scheduling unit Report Tasks progress;
The step of updating task state: be used for the information that the task scheduling unit receives that the Task Progress report updating task of task executing units is carried out;
Task is carried out the step of end and result's readjustment: finish in the task executing units task and carry out the back to task scheduling unit Report Tasks execution result, the task scheduling unit is by the task result readjustment service of application and interface server calling task submission side, readjustment task action result;
Judge whether the step that manually stops: be used to judge whether manually stop the task scheduling circulation, if "No" then get back to " step that task scheduling unit task scheduling begins ", if "Yes" then stop task scheduling;
Query task result's step: be used for task submission side and call application and interface server, query task implementation progress and execution result, this step and " judge whether manually stop step " parallel execution, and can carry out in each process in " step of Processing tasks " and carry out.
6. method according to claim 5 is characterized in that, the substep that the described step that task is split comprises is as follows:
Need to judge whether the substep of FTP fractionation: be used for judging whether the transcoding task has the FTP path, and determine whether needs carry out FTP and split according to looking audio format configuration, if "Yes" then enter " FTP split substep ", if "No" then enter " substep that route splits in the middle of whether needing ";
The substep that FTP splits: be used to utilize buffer storage, whether according to source or target is the FTP path, and a FTP transcoding task is split as 2 or 3 subtasks: from the copy of source FTP to middle buffer memory, and middle buffer memory transcoding, behind the transcoding to the copy of target FTP, updating task tree structure simultaneously;
The substep whether in the middle of needing route splits: be used to check the task scheduling unit whether energy while access originator and destination path are arranged, if do not have the task scheduling unit of simultaneously access originator and destination path then enter " substep that middle route splits ", if route splits then enters " whether needing distributed fractionation substep " in the middle of not needing;
The substep that middle route splits: utilize buffer storage, a task is split as two subtasks: source path subtask in the buffer storage, buffer storage is to the subtask of destination path;
The substep that whether needs distributed fractionation: for the transcoding task, option during according to the task of submission to, task priority, the task executing units quantity factor of current free time, whether decision carries out distributed fractionation, if "Yes" then enter " substep of distributed fractionation ", if "No" then enter " step of coupling path and task executing units ";
The substep of distributed fractionation:, a transcoding task torn open to a plurality of transcodings subtask add an automatic merging task according to the quantity of present idle task performance element; There is a fragment sequence number each transcoding subtask, be responsible for generating a fragment of file destination,, merge task automatically by the concurrent execution of a plurality of task executing units, can be according to the fragment numeric order, the fragment that transcoding is finished is in time merged into the final objective file.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2009103107033A CN101707543B (en) | 2009-11-30 | 2009-11-30 | Enterprise media bus system supporting multi-task type and enterprise media bus method supporting multi-task type |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2009103107033A CN101707543B (en) | 2009-11-30 | 2009-11-30 | Enterprise media bus system supporting multi-task type and enterprise media bus method supporting multi-task type |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN101707543A true CN101707543A (en) | 2010-05-12 |
| CN101707543B CN101707543B (en) | 2012-03-28 |
Family
ID=42377738
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN2009103107033A Expired - Fee Related CN101707543B (en) | 2009-11-30 | 2009-11-30 | Enterprise media bus system supporting multi-task type and enterprise media bus method supporting multi-task type |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN101707543B (en) |
Cited By (20)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102006499A (en) * | 2010-12-10 | 2011-04-06 | 北京中科大洋科技发展股份有限公司 | Method for detecting video and audio quality of digital television program file |
| CN102137276A (en) * | 2010-01-27 | 2011-07-27 | 新奥特（北京）视频技术有限公司 | Check system with interactive function |
| CN102438030A (en) * | 2010-09-29 | 2012-05-02 | 新奥特（北京）视频技术有限公司 | Packing system and method suitable for broadcast television cloud architecture platform |
| CN102546270A (en) * | 2010-12-13 | 2012-07-04 | 深圳市财付通科技有限公司 | Network system control method and device utilizing same |
| CN102655518A (en) * | 2011-03-03 | 2012-09-05 | 哈尔滨工业大学 | Service processing system and method |
| CN102736862A (en) * | 2011-04-07 | 2012-10-17 | 索尼公司 | Reproducing device and reproducing method |
| CN102752497A (en) * | 2011-12-27 | 2012-10-24 | 新奥特（北京）视频技术有限公司 | Comprehensive programming managing platform based on programming |
| CN103167222A (en) * | 2013-03-29 | 2013-06-19 | 北京中科大洋科技发展股份有限公司 | Nonlinear cloud editing system |
| CN103581225A (en) * | 2012-07-25 | 2014-02-12 | 中国银联股份有限公司 | Distributed system node processing task method |
| CN103853607B (en) * | 2012-12-03 | 2017-06-23 | 中国银联股份有限公司 | A kind of mutual Preparation Method of task scheduling |
| CN107844368A (en) * | 2017-10-26 | 2018-03-27 | 苏州市伏泰信息科技股份有限公司 | Environmental sanitation platform task scheduling system |
| CN107885589A (en) * | 2017-11-22 | 2018-04-06 | 链家网（北京）科技有限公司 | A kind of job scheduling method and device |
| CN107908487A (en) * | 2017-11-08 | 2018-04-13 | 中国平安人寿保险股份有限公司 | Task control management method, device, equipment and computer-readable recording medium |
| CN108255820A (en) * | 2016-12-28 | 2018-07-06 | 阿里巴巴集团控股有限公司 | The method, apparatus and electronic equipment of data loading in distributed system |
| CN108289226A (en) * | 2018-01-19 | 2018-07-17 | 数码辰星科技发展（北京）有限公司 | Projection method, server and the system of digital movie video data |
| CN108647093A (en) * | 2018-05-09 | 2018-10-12 | 厦门南讯软件科技有限公司 | A kind of distributed task scheduling processing system and its application method |
| CN108900639A (en) * | 2018-08-10 | 2018-11-27 | 陈雪 | Concentrating type cloud computing system |
| CN111338917A (en) * | 2018-12-19 | 2020-06-26 | 贵州白山云科技股份有限公司 | Dynamic control method and device for determining server service capability |
| CN111935467A (en) * | 2020-08-31 | 2020-11-13 | 南昌富佑多科技有限公司 | Outer projection arrangement of virtual reality education and teaching |
| CN114168601A (en) * | 2022-02-10 | 2022-03-11 | 浙江大学 | Mobile-end-oriented data stream processing method and equipment |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN100552709C (en) * | 2004-10-15 | 2009-10-21 | 中央电视台 | The asset management system of broadcasters and regulate and control method thereof |
-
2009
- 2009-11-30 CN CN2009103107033A patent/CN101707543B/en not_active Expired - Fee Related
Cited By (30)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102137276A (en) * | 2010-01-27 | 2011-07-27 | 新奥特（北京）视频技术有限公司 | Check system with interactive function |
| CN102137276B (en) * | 2010-01-27 | 2013-08-14 | 新奥特（北京）视频技术有限公司 | Check system with interactive function |
| CN102438030A (en) * | 2010-09-29 | 2012-05-02 | 新奥特（北京）视频技术有限公司 | Packing system and method suitable for broadcast television cloud architecture platform |
| CN102006499B (en) * | 2010-12-10 | 2012-10-24 | 北京中科大洋科技发展股份有限公司 | Method for detecting video and audio quality of digital television program file |
| CN102006499A (en) * | 2010-12-10 | 2011-04-06 | 北京中科大洋科技发展股份有限公司 | Method for detecting video and audio quality of digital television program file |
| CN102546270A (en) * | 2010-12-13 | 2012-07-04 | 深圳市财付通科技有限公司 | Network system control method and device utilizing same |
| CN102655518A (en) * | 2011-03-03 | 2012-09-05 | 哈尔滨工业大学 | Service processing system and method |
| CN102655518B (en) * | 2011-03-03 | 2016-03-02 | 哈尔滨工业大学 | Transaction processing system and method |
| CN102736862A (en) * | 2011-04-07 | 2012-10-17 | 索尼公司 | Reproducing device and reproducing method |
| CN102752497A (en) * | 2011-12-27 | 2012-10-24 | 新奥特（北京）视频技术有限公司 | Comprehensive programming managing platform based on programming |
| CN103581225A (en) * | 2012-07-25 | 2014-02-12 | 中国银联股份有限公司 | Distributed system node processing task method |
| CN103853607B (en) * | 2012-12-03 | 2017-06-23 | 中国银联股份有限公司 | A kind of mutual Preparation Method of task scheduling |
| CN103167222A (en) * | 2013-03-29 | 2013-06-19 | 北京中科大洋科技发展股份有限公司 | Nonlinear cloud editing system |
| CN103167222B (en) * | 2013-03-29 | 2016-06-08 | 北京中科大洋科技发展股份有限公司 | A kind of nonlinear cloud editing system |
| CN108255820A (en) * | 2016-12-28 | 2018-07-06 | 阿里巴巴集团控股有限公司 | The method, apparatus and electronic equipment of data loading in distributed system |
| CN108255820B (en) * | 2016-12-28 | 2022-03-04 | 阿里巴巴集团控股有限公司 | Method and device for data storage in distributed system and electronic equipment |
| CN107844368A (en) * | 2017-10-26 | 2018-03-27 | 苏州市伏泰信息科技股份有限公司 | Environmental sanitation platform task scheduling system |
| CN107908487B (en) * | 2017-11-08 | 2020-11-24 | 中国平安人寿保险股份有限公司 | Task control management method, device and equipment and computer readable storage medium |
| CN107908487A (en) * | 2017-11-08 | 2018-04-13 | 中国平安人寿保险股份有限公司 | Task control management method, device, equipment and computer-readable recording medium |
| CN107885589A (en) * | 2017-11-22 | 2018-04-06 | 链家网（北京）科技有限公司 | A kind of job scheduling method and device |
| CN107885589B (en) * | 2017-11-22 | 2021-02-12 | 贝壳找房(北京)科技有限公司 | Job scheduling method and device |
| CN108289226A (en) * | 2018-01-19 | 2018-07-17 | 数码辰星科技发展（北京）有限公司 | Projection method, server and the system of digital movie video data |
| CN108647093A (en) * | 2018-05-09 | 2018-10-12 | 厦门南讯软件科技有限公司 | A kind of distributed task scheduling processing system and its application method |
| CN108900639A (en) * | 2018-08-10 | 2018-11-27 | 陈雪 | Concentrating type cloud computing system |
| CN108900639B (en) * | 2018-08-10 | 2021-04-06 | 乾元云硕科技(深圳)有限公司 | Cluster type cloud computing system |
| CN111338917A (en) * | 2018-12-19 | 2020-06-26 | 贵州白山云科技股份有限公司 | Dynamic control method and device for determining server service capability |
| CN111338917B (en) * | 2018-12-19 | 2023-03-28 | 贵州白山云科技股份有限公司 | Dynamic control method and device for determining server service capability |
| CN111935467A (en) * | 2020-08-31 | 2020-11-13 | 南昌富佑多科技有限公司 | Outer projection arrangement of virtual reality education and teaching |
| CN114168601A (en) * | 2022-02-10 | 2022-03-11 | 浙江大学 | Mobile-end-oriented data stream processing method and equipment |
| CN114168601B (en) * | 2022-02-10 | 2022-07-22 | 浙江大学 | Mobile-end-oriented data stream processing method and equipment |
Also Published As
| Publication number | Publication date |
|---|---|
| CN101707543B (en) | 2012-03-28 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN101707543B (en) | Enterprise media bus system supporting multi-task type and enterprise media bus method supporting multi-task type | |
| US7502850B2 (en) | Verifying resource functionality before use by a grid job submitted to a grid environment | |
| US10491560B2 (en) | Message delivery in messaging networks | |
| TWI224899B (en) | Dynamic binding and fail-over of comparable web service instances in a services grid | |
| CN112000448A (en) | Micro-service architecture-based application management method | |
| US7584292B2 (en) | Hierarchical system configuration method and integrated scheduling method to provide multimedia streaming service on two-level double cluster system | |
| US20030126260A1 (en) | Distributed resource manager | |
| CN109815294B (en) | Distributed parallel data storage method and system without main nodes | |
| CN112882813A (en) | Task scheduling method, device and system and electronic equipment | |
| CN112181648A (en) | Configuration-driven lightweight hybrid infrastructure platform and data processing method | |
| CN111046008A (en) | System and method for synchronously distributing files in real time | |
| CN115115344A (en) | Multidisciplinary simulation optimization business service management method and central station system | |
| US20060271663A1 (en) | A Fault-tolerant Distributed Data Processing System | |
| CN114900449A (en) | Resource information management method, system and device | |
| CN110704196B (en) | Resource data transfer method, device and block chain system | |
| CN117118982A (en) | Message transmission method, device, medium and equipment based on cloud primary multi-cluster | |
| US7739420B2 (en) | Communication error information output method, communication error information output device and recording medium therefor | |
| US10148518B2 (en) | Method and apparatus for managing computer system | |
| CN111309552A (en) | Service log acquisition system and method | |
| US11775401B1 (en) | Intelligent coordination of log analysis and repair processes in a multi-cloud system | |
| US20230344520A1 (en) | Intelligent Monitoring and Repair of Network Services Using Log Feeds Provided Over Li-Fi Networks | |
| CN117041893B (en) | Method and system for sending international short message by using cloud computing technology | |
| CN116320055B (en) | Network protocol conversion method and system | |
| WO2023032103A1 (en) | Job control system and job control method | |
| US20200252336A1 (en) | Switching fabric configuration and management system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20120328 Termination date: 20211130 |
|
| CF01 | Termination of patent right due to non-payment of annual fee |