CN1968250A

CN1968250A - Video code stream checking method, system and device

Info

Publication number: CN1968250A
Application number: CNA2006100813110A
Authority: CN
Inventors: 罗忠
Original assignee: Huawei Technologies Co Ltd
Current assignee: Huawei Technologies Co Ltd
Priority date: 2006-05-16
Filing date: 2006-05-16
Publication date: 2007-05-23
Anticipated expiration: 2026-05-16
Also published as: WO2007131445A1; CN1968250B

Abstract

The invention relates to a video flow inquiring method, system and relative devices, wherein said invention utilizes the relationship between basic layer and strengthen layer of code flow multilayer structure of interlayer code mechanism, to generate inquire code flow; the inquire code flow comprises basic layer code flow, and/or strengthen layer code flow; using special inquirer to inquire and transfer inquire code flow; when the special inquirer inquires out the harmful content in code flow, it will cut off the inquire code flow transmission to the user; user can not decode and rebuild original video flow, to control the transmission of harmful content. The invention can be used to in live broadcast mode and store content play mode.

Description

A kind of video code stream checking method, system and equipment

Technical field

The present invention relates to multimedia communication technology, in particularly a kind of multimedia communication process, a kind of video code stream checking method, system and relevant device of realizing based on hierarchical coding.

Background technology

Streaming Media (Streaming Media) is as a kind of basic multimedia communication form, derived numerous multimedia communication service forms: video conferencing/video telephone, IPTV (Internet ProtocolTelevision, Internet Protocol Television), VOD (Video on Demand, video request program), instant messaging etc.Therefore Streaming Media will become the basic communication form on the NGN (Next Generation Network, next generation network).Especially in recent years the domestic and international rapid rising of IPTV business, the application of Streaming Media on network is also in development rapidly.

Class business on the Streaming Media is a function so that the video/audio content to be provided all such as IPTV and VOD.The scope of content is boundless, comprises movie and video programs, news, sports tournament, concert etc.Each country, especially China pay much attention to for the safety and the monitoring of content always, and relevant law is all arranged.From protection minor angle, also there is relevant regulation various countries.Simultaneously, at operator/ISP (Internet Service Provider, ISP) and content supplier there such demand is arranged also.The domestic operation of soon carrying out IPTV on a large scale, so at first a problem is how to guarantee effective contents supervision and filtration, realizes the filtration of harmful content.Do not address this problem, IPTV operation at home will not known where to begin, and national correlation department also can not distribution of license plates may.Therefore, the solution of this problem has great significance for the development that promotes the IPTV industry.For content safety, common understanding comprises two aspects:

1,, prevents that content from not had the user of authority to receive for the protection of content;

See TV programme etc. such as preventing to steal.For this class invasion, a lot of mature technologies are arranged, such as encrypting (Encryption) and scrambling (Scrambling), authentication and digital copyright management DRM (DigitalRight Management) etc.;

2, for the strick precaution of the invasion of harmful and illegal contents, protection to as if the object attacked of content, normally audient.

This need carry out real-time examination for content, use for reference present way in broadcast television industry, mainly be (generally all to be according to DVB-T the TV programme code stream, Digital Video Broadcasting-Terrestrial transmits, the compressed format of video/audio is MPEG-2, MPEG=Moving Picture ExpertsGroup, an International Standards Organization) in the path that is sent to user television set/STB from program source (such as satellite) the inspection node is set, carries out manual review.Certainly along with development of technology, the examination of some content can have system to finish automatically or semi-automatic (man-machine combination) finished.In a single day manual review finds that programme content has problem, will take measures, stop the transmission of this program stream, and (in most cases) substitute the harmful program of interrupted content with an interim program that substitutes, announce or the like such as public service ads or captions.Certainly artificial judgment need be gathered the context of content, and reaction and disposal need the regular hour, therefore must have a deferred mount that this time-delay is provided, such as 5 seconds.

So-called information filtering, handle for some attribute of content exactly and judge, these contents attributes can comprise: the name of content supplier, the URL of content (Universal Resource Locator Universal Resource Locator, network address is the important URL of a class), the IP address of content providing server etc., and Media Stream is with packet header (packet header) information of the packet under the packet encapsulation situation, the information in the bag etc.As can be seen, this processing and to filter also be that level according to from the superficial to the deep carries out.

Prior art one mainly is the surface according to content, perhaps is called the shallow-layer feature and carries out information filtering.Wherein most typical example is url filtering, its principle is as shown in Figure 1: content filtering equipment is between the core net and edge access network on the network, be from the critical point, only way between the Media Stream arrival receiving terminal of content source so, in practice, can with the agency of enterprise network, NAT (Network AddressTranslator, network address translation devices)/FW (Firewall, fire compartment wall) is placed on same network site, situation for the broadband home user, can and BAS (Broadband Administration System, the broadband management system)/BRAS (Broadband Registration and Admission System, broadband registration and access system), DISLAM is placed on same position, perhaps be placed on the POP (Point ofPresence exists a little) of ISP.

Oneself has internal database filter plant, has the information of a plurality of content origin urls, just can judge according to this database whether a part of content source is harmful, and shielding is harmful to content source, the harmless content source of letting pass.Simultaneously, the content classification service provider that much provides the third party to serve in addition, their database is abundant and professional more, and content filtering equipment also can be connected with this third party service provider, uses their service to carry out url filtering.

There are the following problems for prior art one:

1, the wrong problem of killing: according to url filtering, may fall harmless information filtering, provide Video-on-demand such as the website that has, wherein some program is harmful to, but some is healthy film, only can't distinguish according to URL;

2, mistake is put problem: some URL may may also can go wrong (pretended to be its network address by assault, illegal attempt etc. perhaps oneself is arranged) because be considered to the good website of qualification in scalar system;

3, adopt url filtering, yet need third-party rating system usually, such rating system has, and the grading service provider of some charge provides the grading service specially.But their result can not entirely accurate and the limit network on all contents.And the content on the network also is often to change, and any one rating system also can not in time be caught up with these variations.

For requiring very high application scenarios, such as the public's that orients towards the whole country IPTV, if in case the especially political sensitive content of harmful content is invaded successfully, the harm that causes is huge.It is perfectly safe to accomplish, it all is insecure therefore adopting the filtration of shallow hierarchy.Must adopt the inferior information filtering of bottommost layer, i.e. the filtration of video/audio data itself such as the identification for image, is discerned face of harmful scene (violence, pornographic etc.) wherein, harmful Word message (captions), specific personage or the like.

Reach very high filtration accuracy, must be deep into the darkest level, be i.e. content-data itself.This aspect belongs to present research focus, deep packet filtering DPF (Deep Packet Filtering).

Prior art two depth D PF set based on artificial depth content, in this case, content filtering equipment can be decoded and content play is come out and (be supposed that encryption is not a problem for Media Stream, because the problem of encrypting can require by the Lawful Interception of communication equipment to solve), for manual monitoring person's examination.If finding has problem, the supervisor takes immediate steps, and cuts off harmful content, switches to one section harmless content simultaneously such as public service ads etc.A quite jumbo delay apparatus must be arranged after content filtering equipment certainly, postpone harmful content, give certain judgement of monitor staff and reaction disposal time (such as 5 seconds).Certainly, this process also can adopt automatic or man-machine in conjunction with automanual method realization at present, and it realizes principle as shown in Figure 2.

The basic thought of prior art two is right, and in the field of broadcast televisions practical application for many years, effect still is good.But be used for the Content Advisor of the streaming medium content service on IP (Internet Protocol) network, also need to carry out considerable improvement and perfect.Problem mainly contains:

1, the situation of IP network is more more complex on structure and network topology than television network broadcast, and program is also much more, if the program of being examined all is sent to network center, then takies the too much communication resource;

2, the content source of IP network is many, program is many, if the decoding processing of all concentrating when examining, then the process of decoding amount is too big, and too high to the capacity requirement of examination equipment, existing equipment can't satisfy.

Layered video coding (layered video coding) is a kind of video data stream to be carried out the compressed in layers Methods for Coding, main thought is a plurality of coding layers of output, most important part is a basic layer (base layer), on the basic layer a plurality of enhancement layers (enhancement layer) are arranged, basic layer can separately send (can walk different network paths) with enhancement layer.At receiving terminal, basic layer can go out basic layer video by independent decoding and reconstituting, but enhancement layer must depend on basic layer and/or the enhancement layer below it, could go out each self-corresponding video by decoding and reconstituting.At receiving terminal, decoding and reconstituting comes out basic layer and each enhancement-layer video code stream be according to being superposeed by concrete hierarchy encoding method predetermined rule, thereby obtain total video code flow.Hierarchical coding has a lot of benefits, the most significant be can the enhanced flow media business for the adaptability of variety of network conditions (static difference and dynamic the variation), can export basic its bit rate of layer such as program source is 384kpbs, with its bit rate of enhancement layer be 768kbps, if user's access bandwidth is greater than 1024kbps so, just can receive basic layer and add enhancement layer,, can only receive basic layer if less than 1024kbps.There is the situation of static difference in this (heterogeneous) network that is heterogeneous.The more general situation of network is dynamic change, such as because of network congestion, causes the reduction of bandwidth, and originally the user can receive basic layer and enhancement layer, but after bandwidth reduces, can only receive basic layer.But the user still can see if do not adopt hierarchical coding, to have only a layer, and after possible bandwidth reduced, the user just can't watch.

As shown in Figure 3, Fig. 3 has provided the principle schematic of layered video coding, and this figure has summarized current various main hierarchical coding technology.In general, three kinds of different hierarchical coding mechanism are arranged:

1, time hierarchical coding

Abandon some coded frame, reduce the coding frame number in the unit interval, just so-called frame per second, thus reach the purpose that reduces bit rate.If these frames are so-called bi-directional predicted frames (also being called the B frame), these B frames can be formed an enhancement layer so, because the Code And Decode of B frame relies on other frame, such as intracoded frame (I frame) and encoded predicted frame (P frame), but other frame does not rely on the B frame.Therefore just meet the dependence between basic layer and the enhancement layer.Therefore the method for formation time hierarchical coding is: basic layer coding only comprises I frame and P frame; Between these I frames and P frame, produce the B frame then, thereby form enhancement layer, can add the B frame of any amount as required by bi-directional predicted method.

2, SNR hierarchical coding

The coding quality of basic layer can be lower, thereby cause lower heavily enough signal noise ratios (SNR=Signal-to-Noise Ratio) of image.In general, slightly quantize, can reduce the SNR of basic layer, obtain lower bit rate by increasing quantization parameter (QP=QuantizationParameter).And the reconstruct residual error of basic layer is admitted to enhancement layer and encodes.In enhancement layer, if carry out intraframe coding, obtain the EI frame, if carry out inter prediction encoding, obtain the EP frame.

3, space layered coding

Space layered coding and SNR hierarchical coding are similar, just in that residual looking into forms before enhancement layer coding imports data by the reconstruct of basic layer, carry out the amplification on the space, are called up-sampling (Upsampling) in the technical language of video coding.Ordinary circumstance causes image to amplify k in the horizontal and vertical directions separately _h, k _vDoubly.Certainly, generally speaking, k _h, k _vEquate.Otherwise anamorphose is out of proportion.But in special applications, can be unequal.Typical application is that the image of basic layer is amplified 2 times separately in level and vertical direction, if basic layer is the QCIF image, enhancement layer is exactly the CIF image so.

Corresponding to Fig. 3,, the decomposition transform T of coding side and the synthetic conversion R of decoding end have been introduced in order to summarize above three kinds of mechanism.As long as conversion T and R that definition is suitable, different hierarchical codings all is applicable to this basic framework.Such as for space layered coding, decomposition transform T asks difference to obtain the reconstruct residual error exactly, and then carries out up-sampling; And synthetic conversion R at first carries out up-sampling to the reconstructed frame of low layer and superposes with the reconstructed frame of high level.

More than three kinds of basic mechanism, be widely used in ITU-T H.263/H.263+, H.264, in the main video compression coding standard such as MPEG-2, MPEG-4.On these three kinds of fundamental mechanisms, also has other mutation technology, such as so-called FGS (Fine Granularity Scalability, the fine granulation hierarchical coding), not coding method at enhancement layer exactly according to routine, but after carrying out DCT (Discrete Cosine Transform discrete cosine transform) for the motion prediction residual error, encode according to bit-planes (bit plane) mode, obtain can for the optional position block can both normal decoder so-called embedded bitstream, thereby can provide very thin bandwidth granularity and better network condition adaptability for enhancement layer.But because the fine granulation hierarchical coding needs calculation of complex, so its practicality is lower.

Summary of the invention

The invention provides a kind of video code stream checking method, system and relevant device, to solve in the prior art how to realize the problem that the video code flow depth content is examined based on the hierarchical coding technology.

For addressing the above problem, the invention provides following technical scheme:

A kind of video code stream checking method, it is basic layer bit stream and one deck enhancement layer bitstream at least that content source is carried out hierarchical coding with the original video code stream, when the described video code flow of transmission, described method comprises the steps:

A, described basic layer bit stream is transmitted to examination equipment, sends to user's receiving terminal after described enhancement layer bitstream is postponed to handle;

B, examination equipment examine in the described basic layer bit stream whether comprise harmful content, should be transmitted to user's receiving terminal by basic layer bit stream more if not, transmit this basic layer bit stream otherwise cut off to user's receiving terminal.

Wherein: during live-mode, in the described steps A, content source directly sends or transmits the corresponding code stream of hierarchical coding; Among the described step B, examination equipment video quality as required, the coding quality of the corresponding raising in content of announcement source/described basic layer bit stream of reduction, perhaps when transmitting this basic layer bit stream, the enhancement layer bitstream that is used to strengthen examination effect and accuracy is transmitted in increase/minimizing.

During store content play mode, in the described steps A, described basic layer bit stream that described content source will produce earlier and enhancement layer bitstream store corresponding basic layer bit stream track and enhancement layer track respectively into; Streaming media server is read described basic layer bit stream and is transmitted to examination equipment from described basic layer bit stream track, reads described enhancement layer bitstream and postpone to send to user terminal after the processing; Among the described step B, examination equipment video quality as required, notification streams media server are when transmitting this basic layer bit stream, and the enhancement layer bitstream that is used to strengthen examination effect and accuracy is transmitted in increase/minimizing.

The checking method of examination equipment comprises: the described basic layer bit stream image and import automatic identification equipment of decoding, the related content that comprises in harmful content in harmful content data base that automatic identification equipment will prestore and the described basic layer bit stream image compare analyzes the automatic identification that is harmful to content; And/or described basic layer bit stream image is shown to the manual identified that the supervisor is harmful to content.

When manual identified is carried out simultaneously with automatic identification, inconsistent if the recognition result of the two occurs, then preferentially carry out automatic identification equipment or supervisor's court verdict.

Perhaps, when manual identified is carried out simultaneously with automatic identification, by automatic identification equipment and supervisor respectively according to preset rule, for the harmful content that identifies provides corresponding harmful degree score value, then two score values are weighted the court verdict of finally being carried out after the processing, when only receiving the score value that the harmful content of a side for identification provides, the score value that acquiescence the opposing party provides for this content is zero.

Concrete described weighted method is:

S _I＝(W _M×S _M+W _H×S _H)/(W _M+W _H)

Wherein, W _MAnd W _HExpression automatic identification equipment and supervisor's weights, W _MAnd W _HBetween relative size represented degree of belief to recognition result, S _MAnd S _HIf the mark that provides of automatic identification equipment and supervisor respectively is S _IGreater than a set-point, then court verdict is harmful, otherwise court verdict is harmless, W _M, W _HWith set-point rule of thumb value setting respectively.

In the described method, when examination equipment cut off to be transmitted described basic layer bit stream, start and transmit standby harmless video code flow.

Described method also comprises simultaneously: examination equipment is recorded and is preserved the basic layer bit stream of set period and is used to strengthen the enhancement layer bitstream of examination effect and accuracy.

Described method also comprises simultaneously: the identification situation that will be harmful to content is recorded in the daily record and generates Log Report.

In the described method, when user's receiving terminal receives all enhancement layer bitstreams and basic layer bit stream, the described original video code stream of decoding and reconstituting.

When based on the space delamination method for video coding, content source is carried out hierarchical coding after according to preset proportion the original video code stream being dwindled processing earlier again.

The present invention also provides a kind of video code stream checking system, and comprising: video input apparatus, video encoder and streaming media server also comprise examination equipment;

Described video encoder is basic layer bit stream and at least one enhancement layer bitstream with the hierarchical coding of the original video code stream of video input apparatus collection, and described basic layer bit stream is transmitted to examination equipment by communication network, simultaneously all enhancement layer bitstreams are sent to user's receiving terminal by communication network after postponing to handle; Examination equipment examines in the described basic layer bit stream whether comprise harmful content, should be transmitted to user's receiving terminal by basic layer bit stream by communication network more if not, transmits this basic layer bit stream otherwise cut off to user's receiving terminal.

Wherein, described video encoder comprises:

Encoder, encode described basic layer bit stream and enhancement layer bitstream directly are transmitted to examination equipment with described basic layer bit stream;

First Postponement module postpones described enhancement layer bitstream to send to user's receiving terminal after the processing.

Described examination equipment also comprises: communication module, communicate to connect video encoder, examination equipment video quality as required, the coding quality of the corresponding raising of notice video encoder/described basic layer bit stream of reduction, perhaps when transmitting this basic layer bit stream, the enhancement layer bitstream be used to strengthen examination effect and accuracy is transmitted in increase/minimizing, and the enhancement layer bitstream that will be used to strengthen examination effect and accuracy simultaneously is transmitted to user's receiving terminal after postponing to handle.

Described video code stream checking system also comprises: the video content data storehouse, preserve described basic layer bit stream and enhancement layer bitstream, wherein, described basic layer bit stream is kept in the basic layer bit stream track of setting, and described enhancement layer bitstream is kept in the enhancement layer bitstream track of setting; Described streaming media server comprises: the code stream read module and second Postponement module, the code stream read module is read described basic layer bit stream and is transmitted to examination equipment from described basic layer bit stream track, from described enhancement layer bitstream track, read enhancement layer bitstream and postpone to handle by described second Postponement module after send to user's receiving terminal.

Described examination equipment also comprises: communication module, communicate to connect the video encoder streaming media server, examination equipment video quality as required, the notification streams media server is when transmitting this basic layer bit stream, the enhancement layer bitstream be used to strengthen examination effect and accuracy is transmitted in increase/minimizing, and the enhancement layer bitstream that will be used to strengthen examination effect and accuracy simultaneously is transmitted to user's receiving terminal after postponing to handle.

Described examination equipment comprises:

The 3rd Postponement module postpones to be transmitted to user's receiving terminal after the processing to all code streams that receive;

The examination module, all code streams that examination receives comprise output control signal corresponding when being harmful to content in examining out this code stream;

Switch module is connected after described the 3rd Postponement module;

Main control module is connected between described examination module and the switch module, triggers according to described control signal to disconnect described switch module.

Specifically comprise manual identified submodule and/or automatic recognin module in the described examination module, wherein:

Specifically comprise in the described manual identified submodule: decoding unit, enhancement process unit, display unit and command reception unit, all code stream images that described decoding unit decodes receives, described image is shown to the supervisor after by the enhancement process cell processing on display unit, supervisor's manual identified goes out to comprise in this image when being harmful to content, exports described control signal by the described main control unit of described command reception unit triggers;

Specifically comprise in the described automatic recognin module: automatic identification unit and harmful content data base, automatic identification unit is analyzed the automatic identification that be harmful to content according to the related content that comprises in the harmful content in described harmful content data base and all code streams of receiving, and when identifying harmful content automatically the described main control unit of triggering export described control signal.

When comprising manual identified submodule and automatic recognin module in the described examination module simultaneously, also comprise in this examination module: review mode switches the mould piece, selects to start described manual identified submodule and/or automatic recognin module according to the control of main control module; And the judgement submodule, receive the output signal of described command reception unit and automatic identification unit simultaneously, and whether trigger described main control unit according to the judgement principle judgement of setting and export described control signal.

Also comprise following primary structure in the described examination module:

Judgement principle sub module stored is connected between described judgement submodule and the main control module, and storage is by the judgement principle of main control module input.

Content is recorded module, connects described the 3rd Postponement module, is used to record all code streams of set period the 3rd Postponement module output.

Described examination equipment also comprises:

Logger module connects other module or submodule respectively, is used for generating and exporting the daily record of described examination equipment running status;

The decision support knowledge base connects main control module, is used to store specific or interim harmful content literal, facial image and relevant laws and regulations file.

The video replacing content library is connected described switch module, stores harmless alternate video content.

The present invention also is provided for a kind of video encoder, streaming media server and the examination equipment of said system simultaneously.

Beneficial effect of the present invention is as follows:

Technical solution of the present invention has been utilized the relation of basic layer and enhancement layer in the code stream sandwich construction of hierarchical coding mechanism, produce the examination code stream, examination comprises basic layer bit stream with code stream, perhaps can also comprise the part enhancement layer bitstream, examination of examination equipment and forwarding examination code stream by special use, this special use examination equipment comprises in code stream when being harmful to content examining out examination, cut-out is transmitted the examination code stream to the user, user side can't the decoding and reconstituting original video stream, thereby reach the purpose of the harmful content propagation of control; Less based on the examination that the hierarchical coding technology forms with the data volume of code stream, can not cause burden to system because of repeating process;

Further, can in hierarchical coding the bit rate of each layer be set flexibly, as long as total bit rate reaches certain level, the quality of the video code flow of reconstruct just can reach higher level.Therefore,, can be provided with basic layer bit stream lower, reach the effect of further mitigation system burden in order to reduce the expense that examination causes with code stream;

Technical solution of the present invention can be respectively applied for live-mode and store content play mode, for the existing streaming media service of realizing based on the hierarchical coding technology provides general depth content examination scheme, guaranteed the content safety of streaming media service such as present IPTV, Digital Television, effectively prevented of the diffusion of harmful content by streaming media service.

Description of drawings

The principle schematic that Fig. 1 filters for existing content-based URL;

Fig. 2 is the degree of depth examination principle schematic of existing streaming medium content;

Fig. 3 is the layered video coding principle schematic;

Fig. 4 is for realizing the auditing system structural representation of the technology of the present invention design;

Fig. 5 and Fig. 6 are a kind of structural representation of examination equipment of the present invention;

Fig. 7 is the hierarchical coding principle schematic of the technology of the present invention method when realizing based on the space layered coding technology.

Embodiment

The present invention is based on existing hierarchical coding technology.Produce the examination code stream by basic layer, examine for the examination code stream by examination equipment, examine when no problem, examination equipment is transmitted this basic layer bit stream to receiving user's receiving terminal again, and enhancement layer bitstream directly sends to user's receiving terminal then without examination equipment, user terminal receives after all basic layers add enhancement layer, can carry out the total video code flow of decoding and reconstituting and watch program, user terminal can not receive basic layer, just can't watch program by decoding and reconstituting.

The auditing system structural representation of realization the technology of the present invention design mainly comprises relevant device, content data base, the streaming media server of content source as shown in Figure 4.Examination equipment and user's receiving device, wherein:

Solid line partly is the examination realization flow of live-mode, under the live-mode, the elementary stream that content source directly will be examined usefulness directly sends to by streaming media server and sends to Content Advisor equipment, residue strengthens the code stream delayed delivery and gives user's receiving terminal, do not comprise when being harmful to content in the examination equipment examination elementary stream, again elementary stream is transmitted to user's receiving terminal, if do not comprise harmful content in the elementary stream, then elementary stream is not transmitted to user's receiving terminal, like this, when user's receiving terminal can not receive elementary stream, can't decoding and reconstituting original video code stream.In this process, examination equipment can be as required video quality, the coding quality of the corresponding raising in content of announcement source/basic layer bit stream of reduction, perhaps when transmitting this basic layer bit stream, the enhancement layer bitstream that is used to strengthen examination effect and accuracy is transmitted in increase/minimizing.If the latter comprises also in the code stream that is applied to mutually examine that a part strengthens code stream.

Dotted portion is the examination realization flow of store content play mode, and under the store content play mode, basic layer bit stream that content source will produce earlier and enhancement layer bitstream store corresponding basic layer bit stream track and enhancement layer track respectively into; When streaming media server is play this video content again, from basic layer bit stream track, read basic layer bit stream and be transmitted to examination equipment, read enhancement layer bitstream and postpone to handle after send to user terminal; Equally, in this process, examination equipment video quality as required, the notification streams media server is when transmitting this basic layer bit stream, the enhancement layer bitstream that is used to strengthen examination effect and accuracy is transmitted in increase/minimizing, comprises also in the code stream that is applied to mutually examine that a part strengthens code stream.

Describe the concrete structure of following several equipment components that this system mainly comprises below in detail:

One, content source

Content source facility comprises video input apparatus and the video encoder of gathering video code flow, this video input apparatus is generally video camera, technical conceive of the present invention realizes based on the hierarchical coding technology, thus video encoder based on the hierarchical coding technology original video code stream is encoded to elementary stream and one deck enhancing code stream at least.

Still consult shown in Figure 4, video encoder mainly comprise following structure:

Encoder, encoder is basic layer bit stream and one deck enhancement layer bitstream at least with the original video stream hierarchical coding;

During live-mode, directly export basic layer bit stream to streaming media server, by first Postponement module output enhancement layer bitstream after postponing to handle by communication network; Perhaps directly export basic layer bit stream and part enhancement layer bitstream, postpone to handle the remaining enhancement layer bitstream of back output by first Postponement module to streaming media server.

The code stream that streaming media server will be examined usefulness is transmitted to examination equipment, and the code stream of non-examination usefulness is directly sent to user's receiving terminal.

Certainly, under the live-mode, can simultaneously video content be stored in the content data base, deposit method is: elementary stream is left in the basic layer bit stream track of setting, enhancement layer bitstream is left in the enhancement layer layer bit stream track of setting.

Two, streaming media server

Under live-mode, streaming media server is only born forwarding capability, does not postpone to handle, and is playing under the memory contents pattern, and streaming media server need be to the processing that postpones of non-examination usefulness.

Still consult shown in Figure 4, streaming media server comprises the video code flow read module, also comprise second Postponement module that connects the video code flow read module, the video code flow read module reads from the content data base that the outside connects based on the basic layer bit stream of hierarchical coding and one deck enhancement layer bitstream at least, then:

Directly the basic layer bit stream of output is exported enhancement layer bitstream by second Postponement module through postponing to handle the back; Perhaps

Directly output basic layer bit stream and part enhancement layer bitstream exported remaining enhancement layer bitstream by second Postponement module through postponing to handle the back.

Three, examination equipment

As Fig. 5, based on the examination equipment of hierarchical coding realization video code flow Content Advisor, comprising: communication module communicates to connect other network equipments; Also comprise:

The 3rd Postponement module receives basic layer bit stream or basic layer bit stream and part enhancement layer bitstream, and all code streams that receive are postponed to be transmitted to user's receiving terminal after the processing;

Examination module, examination connect all code streams that postpone forwarding, comprise output control signal corresponding when being harmful to content in examining out code stream;

Switch module is connected after the 3rd Postponement module;

Main control module is connected between examination module and the switch module, triggers the cut-off switch module according to control signal.

As shown in Figure 6, specifically comprise manual identified submodule and/or automatic recognin module in the examination module:

Specifically comprise in the manual identified submodule: all code stream images that decoding unit, enhancement process unit, display unit and command reception unit, decoding unit decodes receive.By be shown to the supervisor after the enhancement process cell processing on display unit, supervisor's manual identified goes out to comprise in this image when being harmful to content, by command reception unit triggers main control unit output control signal;

Automatically specifically comprise in the recognin module: automatic identification unit and harmful content data base, automatic identification unit is analyzed the automatic identification that be harmful to content according to the related content that comprises in the harmful content in harmful content data base and all code streams that receive, and triggers main control unit automatically and export control signal when identifying harmful content.

When comprising manual identified submodule and automatic recognin module in the examination module simultaneously, also comprise in this examination module:

The review mode switching submodule selects to start manual identified submodule and/or automatic recognin module according to the control of main control module;

The judgement submodule receives the output signal of command reception unit and automatic identification unit simultaneously, and whether triggers main control unit according to the judgement principle judgement of setting and export control signal;

Judgement principle sub module stored is connected between judging module and the main control module, and storage is by the judgement principle of main control module input;

The decision support knowledge base connects main control module, is used to store specific or interim harmful content literal, facial image, terrorist's image of for example just having received etc., and the relevant rules file of having sent out.

Examination equipment also comprises: content is recorded module, connects the output of the 3rd Postponement module, is used to record all code streams of set period the 3rd Postponement module output.

Examination equipment also comprises: logger module, connect other module or submodule respectively, and be used for generating and the daily record of output examination equipment running status.

Examination equipment also comprises: the video replacing content library, be connected switch module, and store harmless alternate video content.

The auditing system that above-mentioned video encoder, streaming media server and examination equipment are formed as shown in Figure 4, it is basic layer bit stream and one deck enhancement layer bitstream at least that content source is carried out hierarchical coding with the original video code stream, under live-mode, the flow process of technical solution of the present invention embodiment comprises the steps:

1, content source is transmitted to examination equipment with basic layer bit stream, sends to user's receiving terminal after enhancement layer bitstream is postponed to handle;

2, examination equipment examines whether comprise harmful content in the basic layer bit stream, should be transmitted to user's receiving terminal by basic layer bit stream more if not, transmits this basic layer bit stream otherwise cut off to user's receiving terminal.

And, during live-mode, examination equipment video quality as required, the coding quality of the corresponding raising in content of announcement source/basic layer bit stream of reduction, perhaps when transmitting this basic layer bit stream, the enhancement layer bitstream that is used to strengthen examination effect and accuracy is transmitted in increase/minimizing.

Under store content play mode, basic layer bit stream that content source will produce earlier and enhancement layer bitstream store corresponding basic layer bit stream track and enhancement layer track respectively into; Streaming media server is read basic layer bit stream and is transmitted to examination equipment from basic layer bit stream track, read enhancement layer bitstream and postpone to send to user terminal after the processing;

Examination equipment can be as required video quality, the notification streams media server is when transmitting this basic layer bit stream, increases/minimizings forwarding is used to strengthen the enhancement layer bitstream of examining effect and accuracy.

Wherein, the checking method of examination equipment comprises:

The related content that comprises in the decode basic layer bit stream image and import automatic identification equipment, the harmful content in harmful content data base that automatic identification equipment will prestore and basic layer bit stream image compares analyzes the automatic identification that is harmful to content; And/or

Basic layer bit stream image is shown to the manual identified that the supervisor is harmful to content.

When manual identified is carried out simultaneously with automatic identification, by automatic identification equipment and supervisor respectively according to preset rule, for the harmful content that identifies provides corresponding harmful degree score value, then two score values are weighted the court verdict of finally being carried out after the processing, when only receiving the score value that the harmful content of a side for identification provides, the score value that acquiescence the opposing party provides for this content is zero.

Above-mentioned weighted method is:

S _I＝(W _M×S _M+W _H×S _H)/(W _M+W _H)

Among the present invention, it is one of following that harmful content comprises at least: harmful image, harmful stack literal or symbol, specific facial image.

Among the present invention, when examination equipment cut off to be transmitted basic layer bit stream, start and transmit standby harmless video code flow.

Examination equipment can also be recorded and preserve the basic layer bit stream of set period and be used to strengthen the enhancement layer bitstream of examination effect and accuracy; And the identification situation that will be harmful to content is recorded in the daily record and generates Log Report.

Technical solution of the present invention mainly realizes based on existing three kinds of main flow hierarchical coding technology:

1, based on time layered video coding method

For basic layer lower frame per second is set, such as 5 frame per seconds (5fps), encoding according to intra-frame encoding mode and inter prediction encoding pattern produces basic layer, and then according to the bi-predictive mode coding, produces enhancement layer.Make enhancement layer add that basic layer reaches 25fps (for pal mode), 30fps (for the TSC-system formula).

2, based on SNR layered video coding method

The influencing factors such as quantization parameter of the basic layer coding of control make the bit rate of basic layer be lower than R _b(such as 128kbps) forms first enhancement layer again, perhaps is called main enhancement layer (Prime EnhancementLayer).Bit rate is R _Pe(such as 384kbps) on main enhancement layer basis, forming several auxiliary enhancement layers (Secondary Enhancement Layer) then, plays bit rate and is respectively R _Se1, R _Se2..., R _Sek....

Because basic layer is in this aspect, only be for the examination of this program is provided, as long as it is just passable to see essential informations such as image outline clearly, bit rate is low as far as possible, the user receives basic layer separately and can't satisfy and watch the program requirement, therefore must could normally watch program in conjunction with main enhancement layer.If the network bandwidth is abundant certainly, the user can accept more even whole enhancement layers, watches the program of high definition.In above example, R can be set _b=128kbps, R _Pe=384kbps, R _Se1=512kbps.If the user is ADSL (Asymmetric Digital Subscriber Line) access band user so, can guarantee 512kbps, then can receive basic layer and main enhancement layer.If the user is Ethernet access band user, then can receive basic layer and whole enhancement layer, obtain best video quality.

3, based on the space delamination method for video coding

In order to reduce the bit rate of basic layer, in examination code stream generation scheme based on space layered coding, the present invention at first dwindles for original video data stream, and technical term is called down-sampling (Down-sampling), causes image to dwindle k in the horizontal and vertical directions separately _h, k _vDoubly.The benefit of doing like this is further to reduce the bit rate of examination code stream, such as under SNR layering situation, if 128kbps is the bit rate that the video quality of examination needs can be provided, if we are in space layered coding, at first image is reduced to original 1/4 size (level and vertical direction reduce by 2 times separately), the video quality of examination needs should just can be provided about 32kbps so.Principle as shown in Figure 7.Wherein conversion U carries out up-sampling after representing to ask difference again.Conversion D represents down-sampling.

In streaming media service, two kinds of broadcast and on-the-spot broadcasting (live) that most important business service pattern is a memory contents (storage).

1, memory contents is play situation

Video content is stored in the content data base with the video stream media document form through compressed encoding in advance, in video file inside, basic layer bit stream can be placed on respectively on two different media track with enhancement layer bitstream, principle is consulted shown in Figure 4, at first produce basic layer encoding code stream and each enhancement layer coding code stream, be written to video file then and store in the content data base by video encoder.

When needs are play, streaming media server reads the video data of examination track and enhancement layer track respectively from video file, the delay link that enhancement data can be set by a time of delay then gets on sending to network, and the examination code stream directly sends, and does not need through postponing.

Streaming media server (streaming server) sense data from file, pack and send these bags according to the supplementary in the file, supplementary generally provides the rule about packing, with a plurality of media track (media track in the file, such as video is a track, track of audio frequency, track of text or the like) synchronizing information between, thereby streaming media server knows that in any one special time period [t, t+ Δ t] should be how which track reading which and how many data from packs and send.In this case, video encoder is responsible for carrying out hierarchical coding, generates video file then.Streaming media server is responsible for reading of data from video file, according to the indication of the supplementary transmission of packing:

(1) extracts examination with the transmission of packing of the data of code stream, send to examination equipment;

Here, examination may only comprise basic layer bit stream with code stream or comprise basic layer bit stream and the enhancement layer bitstream of part enhancing examination effect and accuracy, increase or reduce the concrete quantity of enhancement layer bitstream according to the indication of examination equipment.

(2) enhancement layer bitstream that the extracts non-examination usefulness transmission of packing directly sends to user terminal;

(3) keep time synchronized between each media track.

(4) require to send leading in time lead of enhancement layer bitstream with the code stream packing, such as 5 seconds or 10 seconds for examination.In a single day this is that especially manual review needs certain lead because the examination code stream is Duoed than enhancement layer bitstream to be examined through a checking process, so that examine harmful content, has the abundant time to take measures.This lead can be realized by enhancement layer bitstream being postponed the corresponding time, because so-called shifting to an earlier date with backwardness all is relative.

2, live situation

Video flowing after video encoder directly will be encoded makes a distinction packing with code stream (elementary stream or elementary stream and part strengthen code stream) and non-examination with code stream (whole or remainder strengthens code stream) according to examination and sends to streaming media server, and is just passable as long as streaming media server is transmitted.Encoder postpones time of setting by a delay link when the non-examination video coding code stream of output, re-send to network and get on, and examination directly sends with code stream, does not need through postponing.

In a single day Content Advisor equipment receive the examination video flowing from streaming media server, will examine it, and examination can have two kinds of basic modes:

1, manual review: be exactly by manually examining, rely on the experience of audit crew, whether harmfully judging by the examination content according to the laws and regulations of country or some principles of social convention for the video examination content of decoding and be presented on the screen.

2, examination automatically: carry out automatic Content Advisor by machine.A variety of methods are arranged, handle, discern specific harmful scene, such as the violence pornographic etc. such as the identification by video lens; Also can wait the identification of carrying out literal such as captions, judge the Word message that whether exists content harmful by the stack literal that extracts in the video; Also can carry out the identification of people's face and find whether to have specific personage to appear in the picture, such as reaction propaganda of terrorist, Taiwan independence separatist force etc. etc.These are examined automatically, under up-to-date technological progress promotes, have reached its maturity and practicality, and a lot of methods, location and identification such as the stack literal need not decoded and just can carry out for the examination code stream, so can raise the efficiency greatly, and its practicality further improves.

Certainly sometimes, also can carry out the examination of manual review and machine and carry out simultaneously, again its result be merged.Insecure the time, may need both to merge simultaneously such as a kind of recognition result.Such as, if the examination code stream is second-rate, display image is fuzzy, when human eye can't clearly be distinguished, and can be by machine; Also has a kind of situation, think passable people's face such as an artificial audit crew having occurred, but can't take a decision as to whether the object that needs control, discern by automatic examination, if this people's face is arranged in the decision support knowledge base, just can accurately discern its name attribute etc.

Result for two kinds of examinations will merge, and can adopt multiple mode to carry out the fusion of information.Such as methods such as weighted averages.

After having carried out examination, if find that content is no problem, just allow the examination code stream pass through, be forwarded to user terminal, user terminal carries out decoding and reconstituting in conjunction with a plurality of enhancement layers of receiving, obtains the content video that can watch.

If examination finds to have problem, so need be disconnected for harmful content card, thus the program audience of user terminal will interrupt, in order to remedy the problem of " blank screen ".Content Advisor equipment can be adopted and send the substitution video content to user terminal, such as public service ads etc., from the video replacing content library in the Content Advisor equipment.Also to record for harmful content simultaneously, prepare against the later stage and trace law evidence obtaining etc.

Obviously, those skilled in the art can carry out various changes and modification to the present invention and not break away from the spirit and scope of the present invention.Like this, if of the present invention these are revised and modification belongs within the scope of claim of the present invention and equivalent technologies thereof, then the present invention also is intended to comprise these changes and modification interior.

Claims

1, a kind of video code stream checking method, it is basic layer bit stream and one deck enhancement layer bitstream at least that content source is carried out hierarchical coding with the original video code stream, when the described video code flow of transmission, described method comprises the steps:

2, the method for claim 1 is characterized in that, during live-mode,

In the described steps A, content source directly sends or transmits the corresponding code stream of hierarchical coding;

Among the described step B, examination equipment video quality as required, the coding quality of the corresponding raising in content of announcement source/described basic layer bit stream of reduction, perhaps when transmitting this basic layer bit stream, the part enhancement layer bitstream that is used to strengthen examination effect and accuracy is transmitted in increase/minimizing.

3, the method for claim 1 is characterized in that, during store content play mode,

In the described steps A, described basic layer bit stream that described content source will produce earlier and enhancement layer bitstream store corresponding basic layer bit stream track and enhancement layer track respectively into; Streaming media server is read described basic layer bit stream and is transmitted to examination equipment from described basic layer bit stream track, reads described enhancement layer bitstream and postpone to send to user terminal after the processing;

Among the described step B, examination equipment video quality as required, notification streams media server are when transmitting this basic layer bit stream, and the enhancement layer bitstream that is used to strengthen examination effect and accuracy is transmitted in increase/minimizing.

4, the method for claim 1 is characterized in that, among the step B, the checking method of examination equipment comprises:

Decode described basic layer bit stream image and import automatic identification equipment, the related content that comprises in harmful content in harmful content data base that automatic identification equipment will prestore and the described basic layer bit stream image compares analyzes the automatic identification that is harmful to content; And/or

Described basic layer bit stream image is shown to the manual identified that the supervisor is harmful to content.

5, filter method as claimed in claim 4 is characterized in that, and is inconsistent if the recognition result of the two occurs when manual identified is carried out simultaneously with automatic identification, then preferentially carries out automatic identification equipment or supervisor's court verdict.

6, filter method as claimed in claim 4, it is characterized in that, when manual identified is carried out simultaneously with automatic identification, by automatic identification equipment and supervisor respectively according to preset rule, for the harmful content that identifies provides corresponding harmful degree score value, then two score values are weighted the court verdict of finally being carried out after the processing, when only receiving the score value that the harmful content of a side for identification provides, the score value that acquiescence the opposing party provides for this content is zero.

7, filter method as claimed in claim 6 is characterized in that, described weighted method is:

S _I＝(W _M×S _M+W _H×S _H)/(W _M+W _H)

8, filter method as claimed in claim 1 is characterized in that, it is one of following that described harmful content comprises at least: harmful image, harmful stack literal or symbol, specific facial image.

9, filter method as claimed in claim 1 is characterized in that, among the described step B, when examination equipment cut off to be transmitted described basic layer bit stream, starts and transmits standby harmless video code flow.

10, as claim 2 or 3 described filter methods, it is characterized in that described method also comprises simultaneously: examination equipment is recorded and is preserved the basic layer bit stream of set period and is used to strengthen the enhancement layer bitstream of examination effect and accuracy.

11, filter method as claimed in claim 1 is characterized in that, described method also comprises simultaneously: the identification situation that will be harmful to content is recorded in the daily record and generates Log Report.

12, the method for claim 1 is characterized in that, when comprising the steps: also after the step B that user's receiving terminal receives all enhancement layer bitstreams and basic layer bit stream, and the described original video code stream of decoding and reconstituting.

13, the method for claim 1 is characterized in that, when based on the space delamination method for video coding, content source is carried out hierarchical coding after according to preset proportion the original video code stream being dwindled processing earlier again.

14, a kind of system of video code flow Content Advisor comprises: video input apparatus, video encoder and streaming media server is characterized in that described video code stream checking system also comprises examination equipment;

Described video encoder is basic layer bit stream and at least one enhancement layer bitstream with the hierarchical coding of the original video code stream of video input apparatus collection, and described basic layer bit stream is transmitted to examination equipment by communication network, all enhancement layer bitstreams are postponed to handle simultaneously the back and sending to user's receiving terminal by communication network; Examination equipment examines in the described basic layer bit stream whether comprise harmful content, should be transmitted to user's receiving terminal by basic layer bit stream by communication network more if not, transmits this basic layer bit stream otherwise cut off to user's receiving terminal.

15, system as claimed in claim 14 is characterized in that, described video encoder comprises:

16, system as claimed in claim 15, it is characterized in that, described examination equipment also comprises: communication module, communicate to connect video encoder, examination equipment video quality as required, the coding quality of the corresponding raising of notice video encoder/described basic layer bit stream of reduction, perhaps when transmitting this basic layer bit stream, the enhancement layer bitstream be used to strengthen examination effect and accuracy is transmitted in increase/minimizing, and the enhancement layer bitstream that will be used to strengthen examination effect and accuracy simultaneously is transmitted to user's receiving terminal after postponing to handle.

17, system as claimed in claim 14 is characterized in that,

Described video code stream checking system also comprises: the video content data storehouse, preserve described basic layer bit stream and enhancement layer bitstream, wherein, described basic layer bit stream is kept in the basic layer bit stream track of setting, and described enhancement layer bitstream is kept in the enhancement layer bitstream track of setting;

Described streaming media server comprises: the code stream read module and second Postponement module, the code stream read module is read described basic layer bit stream and is transmitted to examination equipment from described basic layer bit stream track, from described enhancement layer bitstream track, read enhancement layer bitstream and postpone to handle by described second Postponement module after send to user's receiving terminal.

18, system as claimed in claim 17, it is characterized in that, described examination equipment also comprises: communication module, communicate to connect the video encoder streaming media server, examination equipment video quality as required, the notification streams media server is when transmitting this basic layer bit stream, and the enhancement layer bitstream that is used to strengthen examination effect and accuracy is transmitted in increases/minimizings, and will be used to strengthen the enhancement layer bitstream of examining effect and accuracy simultaneously and postpone to be transmitted to user's receiving terminal after the processing.

19, as one of any described system of claim 14-18, it is characterized in that described examination equipment comprises:

Switch module is connected after described the 3rd Postponement module;

20, system as claimed in claim 19 is characterized in that, specifically comprises manual identified submodule and/or automatic recognin module in the described examination module, wherein:

21, system as claimed in claim 20 is characterized in that, when comprising manual identified submodule and automatic recognin module in the described examination module simultaneously, also comprises in this examination module:

Review mode switches the mould piece, selects to start described manual identified submodule and/or automatic recognin module according to the control of main control module; And

The judgement submodule receives the output signal of described command reception unit and automatic identification unit simultaneously, and whether triggers described main control unit according to the judgement principle judgement of setting and export described control signal.

22, system as claimed in claim 20 is characterized in that, also comprises judgement principle sub module stored in the described examination module, is connected between described judgement submodule and the main control module, and storage is by the judgement principle of main control module input.

As one of any described system of claim 14-18, it is characterized in that 23, described examination equipment also comprises one of following or combination in any:

Content is recorded module, connects described the 3rd Postponement module, is used to record all code streams of set period the 3rd Postponement module output;

The decision support knowledge base connects main control module, is used to store specific or interim harmful content literal, facial image and relevant laws and regulations file;

24, a kind of video encoder, comprise encoder, it is characterized in that, also comprise: described video encoder also comprises first Postponement module that connects described encoder, described encoder is based on the original video stream hierarchical coding being basic layer bit stream and one deck enhancement layer bitstream at least, then:

Directly the described basic layer bit stream of output is exported described enhancement layer bitstream by described first Postponement module through postponing to handle the back; Perhaps

Directly described basic layer bit stream of output and part enhancement layer bitstream postpone to handle the remaining enhancement layer bitstream of back output by described first Postponement module.

25, a kind of streaming media server, comprise the video code flow read module, it is characterized in that, described streaming media server also comprises second Postponement module that connects described video code flow read module, described video code flow read module reads from external data base based on the basic layer bit stream of hierarchical coding and one deck enhancement layer bitstream at least, then:

Directly the described basic layer bit stream of output is exported described enhancement layer bitstream by described second Postponement module through postponing to handle the back; Perhaps

Directly output described basic layer bit stream and part enhancement layer bitstream exported remaining enhancement layer bitstream by described second Postponement module through postponing to handle the back.

26, a kind of examination equipment, being carried out hierarchical coding by the original video code stream examined by content source is basic layer bit stream and one deck enhancement layer bitstream at least, described examination equipment comprises: communication module communicates to connect other network equipments; It is characterized in that,

The 3rd Postponement module receives described basic layer bit stream or basic layer bit stream and part enhancement layer bitstream, and all code streams that receive are postponed to be transmitted to user's receiving terminal after the processing;

Examination module, examination connect all code streams that postpone forwarding, comprise output control signal corresponding when being harmful to content in examining out described code stream;

Switch module is connected after described the 3rd Postponement module;

27, examination equipment as claimed in claim 26 is characterized in that, specifically comprises manual identified submodule and/or automatic recognin module in the described examination module, wherein:

28, examination equipment as claimed in claim 27 is characterized in that, when comprising manual identified submodule and automatic recognin module in the described examination module simultaneously, also comprises in this examination module:

The review mode switching submodule selects to start described manual identified submodule and/or automatic recognin module according to the control of main control module;

29, examination equipment as claimed in claim 28 is characterized in that, also comprises judgement principle sub module stored in the described examination module, is connected between described judging module and the main control module, and storage is by the judgement principle of main control module input.

As one of any described examination equipment of claim 26-29, it is characterized in that 30, described examination equipment also comprises one of following or combination in any: