Background technology
Along with the develop rapidly of the Internet, network audio video transmission flow is also increasing, network audio-video stream has been deep into daily life, a lot of convenient and change is brought to our life, and constantly there is Image Database that is huge, different content, also seem to bring many puzzlements to us, online a large amount of bad image has a lot of murder by poisoning to numerous young people.In order to purify network, safeguard that social harmony is stabilized, in the urgent need to supervising the audio-video frequency content on network.
Internet video supervision needs to carry out content analysis process to audio and video streams simultaneously, need accelerator card can process mass data fragment, these fragments belong to some video flowings, but only have first video flowing fragment with the metadata information required for decoding, the header of namely usually said audio/video flow.When decoding non-first fragment of audio/video flow, need the header recovering this audio/video flow, so that subsequent decoding.Along with the develop rapidly of the Internet, due to himself superiority of MP4 file format, the MP4 as a container can encapsulate the audio, video data of Multi-encoding form, and the proportion shared in network audio-video stream of MP4 is also increasing.
On existing the Internet there is the audio-video document of multiple format in audio/video flow, and for video file, common encapsulation format has FLV, ASF, AVI, MPG, RM, MP4 etc.Form is planted based on the first five in earlier network audio/video flow, in the burst process of audio/video flow, the header size of audio/video flow is more fixing, when decoding non-first audio/video flow fragment, the Save and restore ratio of header is easier to, as passed through shared drive or message passing mechanism.Along with the develop rapidly of the Internet, the proportion of MP4 shared by network audio-video stream is also increasing, due to himself feature of MP4 encapsulation format, the header size of MP4 is no longer fixing, but increase along with the increase of the size of a road MP4 session, namely MP4 video file is larger, when the burst process of MP4 audio/video flow, needs the header of Save and restore also larger.If the size of MP4 video file does not have the upper limit in theory, the size of header does not just have the upper limit yet, and the strategy in original like this fixed size space for the process of audio/video flow header is just no longer applicable.
For the problem in correlation technique, at present effective solution is not yet proposed.
Summary of the invention
For the problem in correlation technique, the present invention proposes a kind of audio and video data processing method based on many core platforms and device.
Technical scheme of the present invention is achieved in that
According to an aspect of the present invention, a kind of audio and video data processing method based on many core platforms is provided.
The method comprises:
The header of audio/video flow is obtained by the decode first fragment of described audio, video data of the first core processor;
Header is stored in corresponding memory space by the type according to header;
Described audio, video data is divided into MP4 form, non-MP4 form;
According to the header corresponding with the audio, video data of described MP4 or non-MP4 form, the audio, video data of described MP4 or non-MP4 form is decoded.
Wherein, header is stored in corresponding memory space by the type according to header, comprising:
When the type of header is the header of non-MP4 form, the header of described non-MP4 form is stored in the structure of a fixed size and preserves;
When the type of header is the header of MP4 form, the header of described MP4 form is stored in the local process space by pre-defined algorithm.
In addition, the data slot of described audio, video data is obtained by the mode of poll.
Wherein, the method comprises further:
Described structure is obtained by shared drive, and described structure can be shared by the core processor of described many core platforms;
The audio, video data of header to described non-MP4 form according to the described non-MP4 form preserved in described structure is decoded.
Wherein, decoded by the audio, video data of core processor to described MP4 form removed in described many core platforms beyond described first core processor.
Additionally provide a kind of audio, video data processing unit based on many core platforms according to a further aspect in the invention, comprising:
First decoder module, the first fragment for described audio, video data of being decoded by the first core processor obtains the header of audio/video flow;
Memory module, for being stored in corresponding memory space according to the type of header by header;
Sort module, for being divided into MP4 form, non-MP4 form by described audio, video data;
Second decoder module, for according to the header corresponding with the audio, video data of described MP4 or non-MP4 form, decodes to the audio, video data of described MP4 or non-MP4 form.
Wherein, described memory module is further used for:
When the type of header is the header of non-MP4 form, the header of described non-MP4 form is stored in the structure of a fixed size and preserves;
When the type of header is the header of MP4 form, the header of described MP4 form is stored in the local process space by pre-defined algorithm.
Wherein, this device also comprises:
Acquisition module, for obtaining the data slot of described audio, video data by the mode of poll.
Wherein, described structure is obtained by shared drive, and described structure can be shared by the core processor of described many core platforms;
Described second decoder module is further used for, and the audio, video data of header to described non-MP4 form according to the described non-MP4 form preserved in described structure is decoded.
Wherein, described second decoder module is further used for, and is decoded by the audio, video data of core processor to described MP4 form removed in described many core platforms beyond described first core processor.
The decoding of the decoding of the audio-video document of MP4 encapsulation format with the audio-video document of other encapsulation format well combines by the present invention, and the header of MP4 form to be kept in the memory space of its correspondence thus to accomplish controlled, the needs of burst decoding when can well meet internet video supervision by said method.
Embodiment
Below in conjunction with the accompanying drawing in the embodiment of the present invention, be clearly and completely described the technical scheme in the embodiment of the present invention, obviously, described embodiment is only the present invention's part embodiment, instead of whole embodiments.Based on the embodiment in the present invention, the every other embodiment that those of ordinary skill in the art obtain, all belongs to the scope of protection of the invention.
According to embodiments of the invention, provide a kind of audio and video data processing method based on many core platforms, in the process realizing this programme, inventor finds that the many core CPU of many core processing platform Tilera used in the present invention can provide 64 processor cores on a single chip, each core can independent operating, communicate with message passing mechanism by shared drive, for the parallel computation of extensive multichannel audio-video frequency stream provides computing platform.
As shown in Figure 1, comprise according to the audio and video data processing method based on many core platforms of the embodiment of the present invention:
Step S101, obtains the header of audio/video flow by the first fragment of the first core processor decoding audio, video data;
Step S103, header is stored in corresponding memory space by the type according to header;
Step S105, is divided into MP4 form, non-MP4 form by audio, video data;
Step S107, according to the header corresponding with the audio, video data of MP4 or non-MP4 form, decodes to the audio, video data of MP4 or non-MP4 form.
Wherein, header is stored in corresponding memory space by the type according to header, comprising:
When the type of header is the header of non-MP4 form, the header of non-MP4 form is stored in the structure of a fixed size and preserves;
When the type of header is the header of MP4 form, the header of MP4 form is stored in the local process space by pre-defined algorithm.
In addition, the present invention obtains the data slot of audio, video data by the mode of poll.
Further, structure is obtained by shared drive, and structure can be shared by the core processor of many core platforms;
The audio, video data of header to non-MP4 form according to the non-MP4 form preserved in structure is decoded.
Wherein, decoded by the audio, video data of core processor to MP4 form removed in many core platforms beyond the first core processor.
Technical scheme for a better understanding of the present invention, the as shown in Figure 2 schematic diagram of the processing method of the audio, video data based on many core platforms of the present invention, in a specific embodiment, implementation procedure of the present invention is as follows:
(1) in many core platforms, all cores all follow a fixed process binding, and the initial stage completes necessary initialization follow-up journey by initial cores and binds;
(2) with the audio frequency and video fragment data of core accepts band decoding, be divided into two classes: first fragment of audio/video flow and non-first fragment, and be inserted into two different task queues to be decoded;
(3) to decode specially audio/video flow first fragment by a core, after parsing the header of audio/video flow, the header of non-MP4 adopts the structure of a fixed size to preserve, these structure resources from shared drive, later for the process of other cores; The header for MP4 of resolving if judge, directly returns.And whether foundation is that task to be decoded is divided into two class MP4stream and NonMP4stream by MP4 audio/video flow;
(4) non-first fragment in (2) and MP4stream and NonMP4stream in (3) are sent to residue core and go decoding, and we call decoding cores these cores;
(5) according to the characteristic of audio/video flow, after first fragment successfully returns, follow-up fragment just can be sent to decoding end, so based on (3) parsing about audio/video flow header, non-first fragment also will carry out the classification of MP4 and non-MP4.For ensureing the sequential of MP4 with the decoding of non-MP4, we adopt the mode of repeating query to obtain two generic tasks in decoding cores;
(6) non-MP4 audio/video flow obtains header from the structure be kept at shared drive, carries out the recovery of header, and then the further fragments of non-first fragment of decoding.The decoding of non-MP4 audio/video flow has nothing to do with concrete decoding cores;
(7) decoding of MP4 audio/video flow is based upon on the basis of the such relation of single MP4 session merchandiser one decoding cores one_to_one corresponding.Resolve MP4 header, be kept in the local process space by a certain special algorithm, again the header of this session is recovered and adjustment when follow-up non-first fragment decoding;
(8) after all fragments of a certain session are all decoded or this session is aging, then the header release in the header of non-MP4 of shared drive or own process space is preserved;
(9) special output core is finally had to be exported by decoded result.
Wherein, the Processing Algorithm of MP4 header is as follows:
Because the header of MP4 can change with the change of MP4 file size, can to the decoding of MP4 based on this particularity, especially the header of MP4 does process separately.Shared memory space is extremely limited, selects local process space to make the shelf space of MP4 header.Because MP4 header size there is not the upper limit, for avoiding Installed System Memory to exhaust, the shelf space for MP4 header does controlled distribution and recycling.Each decoding cores has the resource pool being independently used for preserving MP4 header, and these resource pools are joined by large I one by one and the same internal memory fragment forms.By some important we, the header of MP4 is called that the data block of index forms, after certain index of parsing, according to its size, we apply for some internal memory fragments from the resource pool of the header of current process, and record the address of each internal memory fragment, are preserved by this index.After the decoding during continuous MP4 audio frequency and video fragment, this index is recovered and adjusts.After all decoded or this session is aging until all fragments of this MP4 session, then these internal memory fragments recorded are returned to the resource pool that current process preserves header.
According to embodiments of the invention, additionally provide a kind of audio, video data processing unit based on many core platforms.
As shown in Figure 3, according to the audio, video data processing unit based on many core platforms of the embodiment of the present invention, comprising:
First decoder module 31, for obtaining the header of audio/video flow by the first fragment of the first core processor decoding audio, video data;
Memory module 32, for being stored in corresponding memory space according to the type of header by header;
Sort module 33, for being divided into MP4 form, non-MP4 form by audio, video data;
Second decoder module 34, for according to the header corresponding with the audio, video data of MP4 or non-MP4 form, decodes to the audio, video data of MP4 or non-MP4 form.
Wherein, memory module 32 is further used for:
When the type of header is the header of non-MP4 form, the header of non-MP4 form is stored in the structure of a fixed size and preserves;
When the type of header is the header of MP4 form, the header of MP4 form is stored in the local process space by pre-defined algorithm.
Wherein, this device also comprises:
Acquisition module (not shown), for obtaining the data slot of audio, video data by the mode of poll.
Wherein, structure is obtained by shared drive, and structure can be shared by the core processor of many core platforms;
Second decoder module 34 is further used for, and the audio, video data of header to non-MP4 form according to the non-MP4 form preserved in structure is decoded.
Wherein, the second decoder module 34 is further used for, and is decoded by the audio, video data of core processor to MP4 form removed in many core platforms beyond the first core processor.
Numerous core processor can be carried out classification process by many core platforms of the present invention as required, namely the multiple core processors in many core platforms can work alone the work that also can cooperatively interact, and staff can distribute multiple processor according to actual needs.
The decoding of the decoding of the audio-video document of MP4 encapsulation format with the audio-video document of other encapsulation format well combines by the present invention, and the header of MP4 form to be kept in the memory space of its correspondence thus to accomplish controlled, the needs of burst decoding when can well meet internet video supervision by said method.
The foregoing is only preferred embodiment of the present invention, not in order to limit the present invention, within the spirit and principles in the present invention all, any amendment done, equivalent replacement, improvement etc., all should be included within protection scope of the present invention.