Summary of the invention
The object of the invention is to address the above problem, a kind of media file edit methods is provided, can be a media file with video data, voice data and auxiliary data synchronously.
Another object of the present invention is to provide a kind of media file playing method, carry out the synchronous output of video data, voice data and auxiliary data for the media that are integrated synchronously.
Another purpose of the present invention is to provide a kind of media file editing system, can be a media file synchronously with video data, voice data and auxiliary data.
A further object of the present invention is to provide a kind of media file-playing system, can be a media file synchronously with video data, voice data and auxiliary data.
Technical scheme of the present invention is: the present invention has disclosed a kind of media file edit methods, comprising:
Difference inputting video data, voice data and auxiliary data;
Respectively described video data is carried out Video coding, described voice data is carried out audio coding, described auxiliary data is carried out the auxiliary data coding; And
Temporal information based on described video data, described voice data and described auxiliary data, according to predefined file layout standard, described video data, described voice data and described auxiliary data behind the coding are synthesized a media file, and wherein said auxiliary data is used for retrieval and the editor to described video data and described voice data.
According to an embodiment of media file edit methods of the present invention, described auxiliary data comprises the search information of described media file, comprises the information such as video presentation, edit session, broadcasting time.
The present invention has also disclosed a kind of media file playing method, comprising:
Read a media file, comprise video data, voice data and the auxiliary data of time-based information synchronization in the described media file, wherein said auxiliary data is used for retrieval and the editor to described video data and described voice data;
According to predefined file layout standard, from described media file, decomposite described video data, described voice data and described auxiliary data;
Respectively described video data, described voice data and described auxiliary data are decoded;
Decoded described video data, described voice data and described auxiliary data are exported synchronously according to timestamp.
According to an embodiment of media file playing method of the present invention, described auxiliary data comprises the search information of described media file, comprises the information such as video presentation, edit session, broadcasting time.
The present invention has disclosed again a kind of media file editing system, comprising:
The video data load module, the input of receiving video data;
The voice data load module, the input of audio reception data;
The auxiliary data load module receives the input of auxiliary data;
Video encoder connects described video data load module, and the described video data of inputting is carried out Video coding;
Audio coder connects described voice data load module, and the described voice data of inputting is carried out audio coding;
The auxiliary data scrambler connects described auxiliary data load module, and the described auxiliary data of input is carried out the auxiliary data coding;
Synthesis module, connect described video encoder, described audio coder, described auxiliary data scrambler, according to predefined file layout standard, described video data, described voice data and described auxiliary data after the time-based stamp will be encoded synthesize a media file, and wherein said auxiliary data is used for retrieval and the editor to described video data and described voice data.
According to an embodiment of media file editing system of the present invention, described auxiliary data comprises the search information of described media file, comprises the information such as video presentation, edit session, broadcasting time.
The present invention has disclosed again a kind of media file-playing system, comprising:
The media file load module reads a media file, and described media file comprises synchronous video data, voice data and the auxiliary data of time-based stamp, and wherein said auxiliary data is used for retrieval and the editor to described video data and described voice data;
The document analysis module connects described media file load module, according to predefined file layout standard, decomposites described video data, described voice data and described auxiliary data from described media file;
Video Decoder connects described document analysis module, and the described video data that decomposites is decoded;
Audio decoder connects described document analysis module, and the described voice data that decomposites is decoded;
Auxiliary data decoder connects described document analysis module, and the described auxiliary data that decomposites is decoded;
Isochronous controller connects described Video Decoder, described audio decoder and described auxiliary data decoder, the synchronous operation that decoded described video data, described voice data and described auxiliary data are exported;
The video output module connects described isochronous controller, the video information after the output synchronously;
The audio frequency output module connects described isochronous controller, the audio-frequency information after the output synchronously;
The supplementary output module connects described isochronous controller, the auxiliary data after the output synchronously;
According to an embodiment of media file-playing system of the present invention, described auxiliary data comprises the search information of described media file, comprises the information such as video presentation, edit session, broadcasting time.
The present invention contrasts prior art following beneficial effect: the solution of the present invention is that the descriptive information for Audio and Video is synthesized a media file as auxiliary data and video data, voice data.Other various supplementarys have also been comprised in the auxiliary data, these supplementarys have comprised the requisite search information of video website, this mode is the search information media file that writes direct, rather than the information of will searching for is as filename, this Method of Data Organization is that other parts of system are (such as video website backstage analysis software, be used for analyzing search information from media file, the search information of generation specific descriptions media file) provide redundancy, that is to say if analyzing good search information has damaged, can reanalyse media file and obtain again correct search file.And adopt this method to preserve search information because be not subjected to the restriction of filename length, so can preserve more search information.
How the present invention also produces also auxiliary data is described, the user is in the editing media file, need the input descriptive information to be used for describing the feature of this media file, the information that these information and editing system produce automatically (as is uploaded the time, broadcasting time etc.) as search information, and other supplementarys (such as captions, auxiliary audio frequency, auxiliary video) compress after the mixing, and voice data, video data synthesize rear auxiliary data flow as media file.The user uploads to video website to this media file after editor finishes, these files are for backstage analysis software and the Play System of video website.
The present invention innovates the behavior of Play System, just plays this document in the time of traditional played, can not make amendment to media file.The Play System that is fit to video website can upgrade the information such as broadcasting time that media file is deposited after playing end, that is to say that the application's Play System can be made amendment to file, upgrades at any time the search information in the media file.
The present invention is by adding search information in media file, these search information and traditional auxiliary datas such as captions form auxiliary data flow together, the auxiliary data that the application proposes can make video website search for better and utilize media file, also help the user to use better these video website.
Embodiment
The invention will be further described below in conjunction with drawings and Examples.
The embodiment of media file edit methods
Fig. 1 shows the flow process of the embodiment of media file edit methods of the present invention, sees also Fig. 1, and the step of the media file edit methods of present embodiment is as follows.
Step S10: difference inputting video data, voice data and auxiliary data.
Can read in video data from video capture device, disk or network flow, form can be YUV or RGB.
Can read in voice data from audio collecting device, disk or network flow, form can be PCM.
Can read in auxiliary data from disk or network flow, auxiliary data comprises captions, picture, label, audio frequency, video etc.Auxiliary data is used for retrieval and the editor to video data and voice data (that is, media file).When auxiliary data is used for the search of media file, such as comprising the information such as video presentation, edit session, broadcasting time.
Step S12: respectively video data is carried out Video coding, voice data is carried out audio coding, auxiliary data is carried out the auxiliary data coding.
Coding to video data can adopt present existing form, such as H.264 waiting.Coding to voice data can adopt present existing form, such as AAC etc.Coding to auxiliary data can adopt self-defining scrambler to encode.
Step S14: based on the temporal information of video data, voice data and auxiliary data, according to predefined file layout standard, video data, voice data and auxiliary data behind the coding are synthesized a media file.
Voice data, video data and auxiliary data that each road encodes are synthesized according to self-defining file layout, need to consider in the time of synthetic that each road temporal information carries out synchronously.
The form of the media file after synthetic is shown in Fig. 9 B, comprise file header, audio stream, video flowing and auxiliary data flow, and the form of auxiliary data comprises supplementary head, search information, edit file, caption information, label information, auxiliary audio frequency information and auxiliary video information shown in Fig. 9 A.
At last, the file output after synthesizing can be saved as file to disk, also can pass through procotol (such as RTSP) real-time release to network.
The embodiment of media file playing method
Fig. 2 shows the flow process of the embodiment of media file playing method of the present invention.See also Fig. 2, the step of the media file edit methods of present embodiment is as follows.
Step S20: read a media file.
From disk or network, read in the data of media file.The video data, voice data and the auxiliary data that comprise the time-based information synchronization in the media file.Auxiliary data is used for retrieval and the editor to video data and voice data (that is, media file).When auxiliary data is used for the search of media file, such as comprising the information such as video presentation, edit session, broadcasting time.
The form of synthetic media file is shown in Fig. 9 B, comprise file header, audio stream, video flowing and auxiliary data flow, and the form of auxiliary data comprises supplementary head, search information, edit file, caption information, label information, auxiliary audio frequency information and auxiliary video information shown in Fig. 9 A.
Step S22: according to predefined file layout standard, from media file, decomposite video data, voice data and auxiliary data.
Step S24: respectively video data, voice data and auxiliary data are decoded.
Video data is decoded the unpacked data of output YUV or rgb format.Voice data is decoded the unpacked data of output PCM form.Auxiliary data is decoded the unpacked data of output user-defined format.
Step S26: decoded video data, voice data and auxiliary data are exported synchronously according to timestamp.
If the time that certain data arrives is larger than the output time of its setting, need these data are abandoned and do not export.
The synchronously operation of output has: for video output, the unpacked data of YUV or rgb format is shown on the concrete display device; For audio frequency output, the unpacked data of PCM form is outputed on the concrete audio output apparatus; Export for supplementary, incompressible auxiliary data is exported according to its form, if auxiliary data is captions, picture, label, video, the mode of reference video output is exported, if auxiliary data is audio frequency, then the mode of reference audio output is exported.If auxiliary data is search information, the mode that can be used as the output of customer interaction information reference video is exported.
The embodiment of media file editing system
Fig. 3 shows the principle of the embodiment of media file editing system of the present invention, and Fig. 4 shows the operational scheme of the system of this embodiment.Please be simultaneously referring to Fig. 3 and Fig. 4, the media file editing system of present embodiment comprises: video data load module 100, voice data load module 101, auxiliary data load module 102, video encoder 103, audio coder 104, auxiliary data scrambler 105, synthesis module 106.
Annexation between these modules is: video data load module 100 connects video encoder 103, voice data load module 101 connects audio coder 104, auxiliary data load module 102 connects auxiliary data scrambler 105, and video encoder 103, audio coder 104 and auxiliary data scrambler 105 connect synthesis module 106.
The input of video data load module 100 receiving video datas can be read in video data from video capture device, disk or network flow, form can be YUV or RGB.The input of voice data load module 101 audio reception data can be read in voice data from audio collecting device, disk or network flow, form can be PCM.Auxiliary data load module 102 receives the input of auxiliary data, can read in auxiliary data from disk or network flow, and auxiliary data comprises captions, picture, label, audio frequency, video etc.Auxiliary data is used for retrieval and the editor to video data and voice data (that is, media file).When auxiliary data was used for the search of media file, such as comprising the information such as video presentation, edit session, broadcasting time, video presentation can be inputted by the user, and other search (such as edit session, broadcasting time) can be generated automatically by system.
The video data of 103 pairs of inputs of video encoder carries out Video coding, can adopt present existing form to the coding of video data, such as H.264 waiting.The voice data of 104 pairs of inputs of audio coder carries out audio coding, can adopt present existing form to the coding of voice data, such as AAC etc.The auxiliary data of 105 pairs of inputs of auxiliary data scrambler is carried out auxiliary data coding, can adopt self-defining scrambler to encode to the coding of auxiliary data.
Synthesis module 106 is according to predefined file layout standard, and video data, voice data and auxiliary data after the time-based stamp will be encoded synthesize a media file.The form of the media file after synthetic is shown in Fig. 9 B, comprise file header, audio stream, video flowing and auxiliary data flow, and the form of auxiliary data comprises supplementary head, search information, edit file, caption information, label information, auxiliary audio frequency information and auxiliary video information shown in Fig. 9 A.At last, the file output after synthesizing can be saved as file to disk, also can pass through procotol (such as RTSP) real-time release to network.
The embodiment of media file-playing system
Fig. 5 shows the principle of the embodiment of media file-playing system of the present invention, and Fig. 6 shows the operational scheme of the system of this embodiment.Please be simultaneously referring to Fig. 5 and Fig. 6, the media file-playing system of present embodiment comprises: media file load module 200, document analysis module 201, Video Decoder 202, audio decoder 203, auxiliary data decoder 204, isochronous controller 205, video output module 206, audio frequency output module 207, supplementary output module 208.
Annexation between these modules is: media file load module 200 threaded file parsing modules 201, document analysis module 201 connects respectively Video Decoder 202, audio decoder 203 and auxiliary data decoder 204.Video Decoder 202, audio decoder 203 and auxiliary data decoder 204 all are connected to isochronous controller 205, and isochronous controller 205 is connected respectively to video output module 206, audio frequency output module 207 and supplementary output module 208.
Media file load module 200 reads a media file, and media file comprises synchronous video data, voice data and the auxiliary data of time-based stamp.From disk or network, read in the data of media file.The video data, voice data and the auxiliary data that comprise the time-based information synchronization in the media file.Auxiliary data is used for retrieval and the editor to video data and voice data (that is, media file).When auxiliary data is used for the search of media file, such as comprising the information such as video presentation, edit session, broadcasting time.
The form of synthetic media file is shown in Fig. 9 B, comprise file header, audio stream, video flowing and auxiliary data flow, and the form of auxiliary data comprises supplementary head, search information, edit file, caption information, label information, auxiliary audio frequency information and auxiliary video information shown in Fig. 9 A.
Document analysis module 201 decomposites video data, voice data and auxiliary data according to predefined file layout standard from media file.
202 pairs of video datas that decomposite of Video Decoder are decoded.203 pairs of voice datas that decomposite of audio decoder are decoded.204 pairs of auxiliary datas that decomposite of auxiliary data decoder are decoded.Video data is decoded the unpacked data of output YUV or rgb format.Voice data is decoded the unpacked data of output PCM form.Auxiliary data is decoded the unpacked data of output user-defined format.
The synchronous operation that isochronous controller 205 pairs of decoded video datas, voice data and auxiliary datas are exported.If the time that certain data arrives is larger than the output time of its setting, need these data are abandoned and do not export.
Video information after 206 outputs synchronously of video output module.Audio-frequency information after 207 outputs synchronously of audio frequency output module.Auxiliary data after 208 outputs synchronously of supplementary output module.
The synchronously operation of output has: for video output, the unpacked data of YUV or rgb format is shown on the concrete display device; For audio frequency output, the unpacked data of PCM form is outputed on the concrete audio output apparatus; Export for auxiliary data, incompressible auxiliary data is exported according to its form, if supplementary is captions, picture, label, video, the mode of reference video output is exported, if supplementary is audio frequency, then the mode of reference audio output is exported.If auxiliary data is search information, the mode that can be used as the output of customer interaction information reference video is exported.
Media file editor and the integrated system of broadcast
Fig. 7 shows media file editor of the present invention and plays the principle of integrated system.See also Fig. 7, the integral system of present embodiment comprises Play System user interface layer, Play System key-course, editing system user interface layer, editing system key-course, editing system data input layer, multimedia middleware layer, multimedia output layer, synthetic output layer, editing system hardware and Play System hardware.
The Play System user interface layer offers the interface that the user operates Play System, provides such as the input medium file, begins to play, stops to play, suspends broadcast, fast-forward play, fast reverse play, jumps to the operation such as ad-hoc location broadcast.
The Play System key-course encapsulates the bottom module of Play System, provides unified interface to call for the Play System user interface layer.
The editing system user interface layer offers the interface that the user operates editing system, comprises input material, montage material, begins to encode, suspends coding, finishes coding, begins to synthesize, suspends and synthesize, finish the operations such as synthetic.
The editing system key-course encapsulates the bottom module of editing system, provides unified interface to call for the editing system user interface layer.
Editing system data input layer is realized the function of input material, comprises from file, network flow or other audio-video acquisition equipment input material.
The multimedia middleware layer is realized multi-media decoding and encoding, the file synthesis/functions such as decomposition.
The multimedia output layer outputs to decoded audio, video data in the concrete audio frequency and video hardware device.
Synthetic output layer outputs to synthetic data in disk or the network.
Editing system hardware for example is editing system audio frequency and video input equipments, and the function of video acquisition, audio collection is provided.
Play System hardware for example is Play System audio frequency and video output devices, and the function that video shows, audio frequency is play is provided.
The multimedia middleware layer adopts the form exploitation of assembly, on Windows, can use the DirectShow standard, on embedded system, can adopt OpenMax IL standard, its principle as shown in Figure 8, the multimedia middleware layer comprises video encoder, audio coder, auxiliary data scrambler, file synthesis device, Video Decoder, audio decoder, auxiliary data decoder and file resolver.
Video encoder comes in to encode to video data, can adopt present existing form, as H.264 waiting.Audio coder is encoded to voice data, can adopt present existing form, such as AAC etc.The auxiliary data scrambler is encoded to auxiliary data, adopts the custom coding device to encode.
The voice data that the file synthesis device encodes each road, video data, auxiliary data is synthesized according to self-defining file layout.Need to consider each road information synchronization information in the time of synthetic.
Video Decoder is decoded to video data, the unpacked data of output YUV or rgb format.Audio decoder is decoded to voice data, the unpacked data of output PCM form.Auxiliary data decoder is decoded to auxiliary data, the unpacked data of output user-defined format.
The file resolver goes out coded audio data according to self-defining file layout Standard Decomposition from the data of reading in, video data encoder, and the auxiliaring coding data are given respectively corresponding demoder and are decoded.
The application of video website searching analysis
The searching analysis flow process of video website as shown in figure 10, at first the input medium file data namely reads in media file from the video website server.Then from media file data, analyze search information, from the document flow of reading in, analyze search information, obtain the information such as video presentation, edit session, broadcasting time.Analyze again supplementary, generate search information, the search engine system of video information is analyzed these search information, final updating is to video website backstage file system, search information is write in the backstage file system of search engine, can preserve by write into Databasce, also can be according to the user-defined format writing in files.
The application of the playback of media files on the video website
Figure 11 shows the application that media file of the present invention is play in video website.See also Figure 11, at first, from disk or network, read in media file.From the file that reads in, go out coded audio data, video data encoder, auxiliaring coding data according to self-defining file layout Standard Decomposition again, give respectively corresponding demoder and decode.
In the Video Decoder device, video data is decoded the unpacked data of output YUV or rgb format.In audio decoder, voice data is decoded the unpacked data of output PCM form.In auxiliary data decoder, auxiliary data is decoded the unpacked data of output user-defined format.
Then, by isochronous controller voice data, video data and the auxiliary data of decoding are carried out according to timestamp synchronously.If the time that certain data arrives is larger than the output time of its setting, then needs these data are abandoned, and do not export.
For the data after synchronously, the unpacked data of YUV or rgb format is shown on the concrete display device, the unpacked data of PCM form is outputed on the concrete audio output apparatus, incompressible auxiliary data is exported according to its form.If supplementary is captions, picture, label, video, the way of output that then shows according to video is exported, if supplementary is audio frequency, the way of output of then playing according to audio frequency is exported.If auxiliary data is search information, the mode that can be used as the output of customer interaction information reference video is exported.
At last, with the search such as broadcasting time information updating in media file.
Above-described embodiment provides to those of ordinary skills and realizes or use of the present invention; those of ordinary skills can be in the situation that does not break away from invention thought of the present invention; above-described embodiment is made various modifications or variation; thereby protection scope of the present invention do not limit by above-described embodiment, and should be the maximum magnitude that meets the inventive features that claims mention.