The content of the invention
In order to overcome the deficiencies in the prior art, it is an object of the invention to provide distributed trans-coding system, method and its dress
Put, it can realize the distributed trans-coding that transcoding machine itself is voluntarily dispatched.
The purpose of the present invention is realized using following technical scheme:
Distributed trans-coding system, including transcoding module, database, shared-file system;
Shared-file system is used to receive the source file that sends of request end of outside and stores;
Database is used to receive the task data bag that sends of request end of outside and stores;
Transcoding module is used to obtain task data bag from database, and according to task data bag from shared-file system
It is middle to obtain corresponding source file, and transcoding is carried out to source file;Transcoding file after transcoding is stored into shared-file system.
Distributed trans-coding method, is applied to distributed trans-coding system, comprises the following steps:
S1, shared-file system receive source file and store;Database receives task data bag and stores;
S2, the daemon thread of transcoding module obtain task data bag from database, and in the work pool of transcoding module
Create a transcoding process;Transcoding source file path of the work pool in task data bag obtains from shared file clipping system
Corresponding source file;
S3, work pool carry out transcoding to source file, and the successful transcoding file of transcoding is stored into shared-file system.
Preferably, work pool includes heartbeat unit and transcoding units;
S2 is specially:Daemon thread obtains task data bag from database, and one transcoding of establishment enters in work pool
Journey;The timing of heartbeat unit is sent in the task data bag in heartbeat data to database, so that during heartbeat in task data bag
Between be updated to receive time of the heartbeat data;Transcoding units are obtained according to transcoding source file path from shared file clipping system
Take corresponding source file.
Preferably, the daemon thread in S2 obtains the task data bag for treating transcoding from database;And from database
It is middle obtain operation in and heart time apart from current time exceed the scheduled time task data bag.
Preferably, daemon thread is additionally operable to detect the capacity in work pool, and when the capacity of work pool has surplus, keep
Shield thread obtains task data bag from database again, and another transcoding process is created in work pool.
Preferably, S2 also includes:The source file that work pool will get is stored into CACHE DIRECTORY.
Preferably, S3 also includes:Daemon thread sends transcoding pass signal to correspondence after the success of transcoding units transcoding
Task data bag in, and update the status information of the task data bag to complete.
Preferably, daemon thread is additionally operable to after the failure of transcoding units transcoding, transcoding failure signal is sent to corresponding
In task data bag, it is failure to update the status information of the task data bag.
Preferably, in S3, transcoding demand of the work pool in task data bag carries out transcoding to source file.
Distributed trans-coding device, is applied to distributed trans-coding system, including:
First memory module, is applied to shared-file system, for receiving source file and storing;Second memory module, should
For database, for receiving task data bag and storing;
Creation module, is applied to the daemon thread of transcoding module, for obtaining task data bag from database, and is turning
A transcoding process is created in the work pool of code module;
Acquisition module, is applied to work pool, for the transcoding source file path in task data bag from shared file
Corresponding source file is obtained in clipping system;
Operation module, is applied to work pool, and transcoding is carried out to source file, and the successful transcoding file of transcoding is stored to shared
In file system.
Compared to existing technology, the beneficial effects of the present invention are:By the removal of central task mixing system, by each transcoding machine
Device is realized voluntarily dispatching in itself, so as to reduce the risk for breaking down, saves the input of machine cost, reduces later maintenance
Difficulty.
Specific embodiment
Below, with reference to accompanying drawing and specific embodiment, the present invention is described further:
Distributed trans-coding system, as shown in figure 1, including transcoding machine, database, shared-file system and transmission assembly;
Transmission assembly is used to receive needs to carry out the source file of transcoding and task data bag, and source file is stored to shared
In source file module in file system, and task data bag is stored into the task pool of database.Each task data
Include task ID, transcoding source file path, status information (being initially " treating transcoding "), heart time, transcoding file destination road
The information such as footpath, transcoding demand, transcoding working machine.
It is provided with transcoding module in transcoding machine, the task data bag that transcoding module is obtained from the task pool in database,
And send and update heart time in heartbeat data to task pool in corresponding task data bag, and turning according to task data bag
Code source file path obtains corresponding source file from the source file module in shared-file system.It is provided with each transcoding machine
Multiple transcoding modules.
Transcoding module carries out transcoding according to task data bag and source file, while source file is stored into delaying to transcoding machine
In depositing catalogue.Transcoding module stores the successful transcoding file of transcoding in the transcoding file module into shared-file system, and
In task pool in transmission transcoding pass signal to database in corresponding task data bag, the state of the task data bag is updated
Information is completion.
In some cases, the form of such as source file do not support, or source file such as damages at the reason, and transcoding module cannot be by
The source file transcoding, then transcoding will fail, now task identification can be failure by transcoding module, and not make any text
Part copy.
Distributed trans-coding method, is applied to distributed trans-coding system, as shown in Fig. 2 comprising the following steps:
Request end outside step 100, transmission assembly reception carries out the source file and number of tasks of transcoding the need for sending
According to bag, and source file is stored into shared-file system, and task data bag is stored into database;
Specifically, by the source file module in the shared-file system of source file storage;And deposit task data bag
In storage to the task pool of database;When each task data bag includes task ID, transcoding source file path, status information, heartbeat
Between, transcoding file destination path, transcoding demand and transcoding working machine;Status information includes:Treat transcoding, operation in, complete and lose
Lose.
Step 101, transcoding module obtain task data bag from database, and create a transcoding process, transcoding module
In timing transmission heartbeat data to task pool in corresponding task data bag, and according to transcoding source file path from shared file
Corresponding source file is obtained in clipping system.
Transcoding module includes daemon thread and work pool;Work pool includes heartbeat unit and transcoding units.Specifically, transcoding
The daemon thread of module obtains task data bag from task pool, and a transcoding process is created in work pool, work pool
In heartbeat unit timing transmission heartbeat data to task pool in corresponding task data bag, and transcoding units are according to transcoding source document
Part path obtains corresponding source file from shared file clipping system.
Transcoding source file path is corresponding with source file module in shared file clipping system, and transcoding module can be according to transcoding
Source file path obtains corresponding source file from the source file module of shared file clipping system.
Transcoding module sends heartbeat data and is specially:Transcoding module timing sends a heartbeat data to task data bag,
Heart time is updated to receive the time of heartbeat data.For example, transcoding module sends a heartbeat data to task every 30s
Packet, transcoding module is 06:00:20 send a heartbeat data to task data bag for the first time, and heart time is updated to 06:
00:20;Transcoding module is 06:00:To task data bag, heart time is updated to 06 to 50 second transmission heartbeat data packets:00:
50。
Further, transcoding module obtains task data bag from task pool;The status information of the task data bag is to treat
Transcoding, or status information is in running and heart time exceedes the scheduled time apart from current time.For example, the scheduled time is 3 points
Clock, current time is 06:05:00, and the heart time of a certain task data bag is 06:01:20;Now heart time distance is worked as
The preceding time was more than three minutes, therefore transcoding module obtains the task data bag from task pool.
Further, daemon thread is additionally operable to the capacity in detection work pool, if the capacity in work pool has surplus, keeps
Shield thread can again obtain task data bag from database, and another transcoding process is created in work pool.For example, each work
Make the capacity in pond for 10 transcoding processes can be created, and now there are 7 transcoding processes in work pool, then daemon thread can be again
It is secondary that task data bag is obtained from database, transcoding process is created again.
Step 102, transcoding module carry out transcoding to source file, and the successful transcoding file of transcoding is stored to shared file system
In system.
Specifically, transcoding demand of the transcoding units in task data bag carries out transcoding to source file;For example, task is needed
Source file is transcoded into the file of the rmvb forms that form is high definition, or the file that form is super clear mp4 forms for Seeking Truth.
Further, transcoding units are also stored into the CACHE DIRECTORY of the transcoding machine where transcoding module source file;
The successful transcoding file of transcoding is stored into the transcoding file module of shared-file system.
Further, daemon thread sends correspondence in transcoding pass signal to task pool after the success of transcoding units transcoding
Task data bag in, update the status information of the task data bag to complete.
Further, transcoding module is additionally operable to be sent after transcoding failure corresponding in transcoding failure signal to task pool
In business packet, it is failure to update the status information of the task data bag.
Detailed explanation will be carried out to modules below:
Shared-file system (Sharing File System), the system provides interface, transmission assembly and all transcoding moulds
Block is dependent on the interface and carrys out accessing file.In the present embodiment, shared-file system uses the XNODE that Yi Yun companies provide
Shared file storage system.
Database (Data Base), for store tasks packet, the system provides interface, transmission assembly and all turns
Code module is dependent on the interface of the system to preserve the task data bag related to inquiry.In the present embodiment, database is used
Be increase income MySQL service.
Transmission assembly (Transcode Request Server), this component is the component for receiving transcoding request, the group
The major responsibility of part is that the source file that will need transcoding is written in the middle of share directory, while creating a task in database
Packet, and status information is set to " treating transcoding ", because action is simple, in the middle of actual application process, this part
Logic even without there is a single process to process.In the present embodiment, the component has been integrated into treatment client request
WebApp Server in the middle of.
Transcoding module (Transcode Worker), is component that transcoding reality of work is performed, and transcoding module is designed as
Stateless component, can arbitrarily stretch, and this is depending on the task queue quantity and the idle running situation of component in current database.
Task pool (Task Pool), is present in a worksheet in database, and task pool includes multiple tasks data
Bag, have recorded the task data bag for needing transcoding, and each task data includes task ID, transcoding source file path, state letter
The information such as breath (being initially " treating transcoding "), task heartbeat, transcoding file destination path, transcoding working machine.Wherein, task heartbeat packet
It is required that current transcoding working machine timing updates (usually 30s), if the time gap of task heartbeat packet final updating is current
Time, the then task data bag exception, the task can be obtained by other transcoding modules more than 3 minutes, and the transcoding since new.
Source file module (Origin File), in shared-file system, the source file mesh of transcoding is needed for depositing
Record.
Task heartbeat packet (Transcoded File), in shared-file system, the target for depositing transcoding completion
File directory.
Daemon thread (Daemon Thread), the task data bag in thread meeting timing scan database in task pool,
And the task is put into the middle of the work pool of the machine.
Work pool (Working Pool), for depositing the transcoding task for working, the work pool of each transcoding module
Size can determine according to the idle CPU core number of current transcoding machine.One double-core CPU machine is used for if transcoding completely,
So work pool is exactly 2, represents that current working node can simultaneously carry out 2 transcoding tasks.
Task Thread:The thread of transcoding work is carried out, comprising 2 submodules, one is transcoding module, the industry of core
Business module, for being another video format by a kind of video format transcoding.Another module is Heartbeat (heartbeat) mould
Block, as long as the progress of work is normally run, heartbeat thread will constantly update the data the heart time of this task in storehouse.
Distributed trans-coding device, as shown in figure 3, distributed trans-coding system is applied to, including:
First memory module, is applied to shared-file system, for receiving source file and storing;Second memory module, should
For database, for receiving task data bag and storing;
Creation module, is applied to the daemon thread of transcoding module, for obtaining task data bag from database, and is turning
A transcoding process is created in the work pool of code module;
Acquisition module, is applied to work pool, for the transcoding source file path in task data bag from shared file
Corresponding source file is obtained in clipping system;
Operation module, is applied to work pool, and transcoding is carried out to source file, and the successful transcoding file of transcoding is stored to shared
In file system.
It will be apparent to those skilled in the art that technical scheme that can be as described above and design, make other various
It is corresponding to change and deformation, and all these change and deformation should all belong to the protection domain of the claims in the present invention
Within.