CN101968797A - Inter-lens context-based video concept labeling method - Google Patents

Abstract

The invention provides an inter-lens context-based video concept labeling method, which comprises: acquiring a detector for each concept training labeled in a training set and inputting lenses in the training set and a test set into the concept detectors, wherein output lenses comprise prediction probabilities of corresponding concepts; assigning a target concept, dividing the lenses into different types according to the relationship between the concepts and target concepts of lenses in the training set, and extracting type characteristics; building a model according to the relationship between the lens type characteristics and type labels of the video lenses in the training set and the relationship between adjacent lens type labels, extracting the type characteristics of the video lenses in the test set, inputting into model, acquiring the probability of a certain type of a lens label, and using the probability as a prediction probability of the optimized target concept; and optimizing all concepts in turn, and labeling the lenses with the corresponding concepts if the optimized prediction probabilities are greater than a given threshold. In the method, as the different relationships between lenses in videos are taken into full consideration, the result of concept labeling can be optimized more effectively.

Description

A kind of based on contextual video notion mask method between camera lens

Technical field

The invention belongs to video content analysis and understand technical field, be specifically related to a kind of based on contextual video notion mask method between camera lens.

Background technology

In recent years, along with Internet technology and rapid development of multimedia, the video content that has occurred magnanimity on the network, video information in the face of magnanimity like this, how to utilize computing machine to come it is effectively managed, make the user can retrieve the content of wanting quickly and accurately, become a key issue that urgency is to be solved.Yet, exist difference between the video of human intelligible and the video of computer understanding: the mankind understand video from the angle of semanteme, and computing machine can only extract the low-level image feature in the video content, for example color, texture, shape and sound etc., can not understand semanteme wherein, therefore, exist a wide gap between the human understanding and the understanding of computing machine, be called semantic wide gap, semantic wide gap becomes the big obstacle that computing machine is analyzed automatically and managed video content.In order to shorten the distance of semantic wide gap, help computing machine to understand better and the managing video content, researchers have proposed video notion mark technology, this method at first defines the semantic concept dictionary, the semantic concept that comprises in the dictionary extensively covers the content that may occur in the video as far as possible, for one section video, can choose corresponding notion the main contents of video are described; After having defined the semantic concept dictionary, video notion mark technology is mainly studied the mapping relations of how setting up between video and the semantic concept, for each section video, it is mapped to corresponding semantic concept gets on, and come video content is described with these semantic concepts, can effectively shorten the distance of semantic wide gap.Fig. 1 has provided an example and has described, these notions are taken from the semantic concept dictionary of definition in the document " Lscom lexicon definitions and annotations version 1.0; dto challengeworkshop on large scale concept ontology for multimedia " (document is the technical report of Columbia university in 2006), can see that the semantic concept that video comprises can be described the main contents of video.When video is retrieved, can utilize corresponding notion to come video is carried out index, thereby realize effective management the magnanimity video.Therefore, video notion mark technology has crucial research and using value.

Existing video notion mark technology can be divided into two classes, and first kind method is directly set up the mapping relations of video low-level image feature and semantic concept.For example, in Columbia university technical report in 2007, A.Yanagawa and S.F.Chang etc. have extracted three kinds of low-level image features of video in document " Columbia university ' s baseline detectors for 374 LSCOM semantic visual concepts ", to each notion, obtain three based on these three kinds of features training respectively and detect son, when a certain camera lens is marked, these three detection are exported a prediction probability respectively, and the mean value of getting these three probable values at last comprises the probability of corresponding concepts as this camera lens.Second class utilizes the adjacent video camera lens further to optimize the result of notion mark in relation semantically on the basis of first kind method.Generally speaking, specifically pay close attention to a certain given notion, when being referred to as target concept, we are divided into four classes to the relation between the camera lens in the video: consistance, positive correlation, negative correlation and irrelevant.Wherein, consistance is meant, if comprise target concept in a camera lens, then can increase the probability that its adjacent camera lens comprises target concept; Positive correlation is meant, if the notion and the target concept positive correlation that comprise in camera lens then can increase the probability that comprises target concept in its adjacent camera lens; Negative correlation is meant, if the notion and the target concept negative correlation that comprise in camera lens then can reduce the probability that comprises target concept in its adjacent camera lens; Irrelevant being meant, notion that comprises in the camera lens and target concept are irrelevant, and this camera lens for whether comprising target concept in its adjacent camera lens can not exert an influence so.On ACM Multimedia Conference in 2008, M.F.Weng and Y.Y.Chuang utilize the adjacent camera lens of video in semantically consistance in document " Multi-cue fusion for semantic video indexing ", and the result of semantic concept mark is optimized.Yet this method has only been considered cinestrip in semantically consistance, and does not consider other relations, and therefore for example positive correlation and negative correlation etc. have the room for improvement of further raising.

Summary of the invention

At the deficiencies in the prior art, the present invention proposes a kind ofly based on contextual video notion mask method between camera lens, be used for video lens is labeled as relevant semantic concept, to reach the purpose that video content is described.This method has taken into full account the various relations of camera lens in the video, comprises consistance, positive correlation, negative correlation and irrelevant etc., can further optimize the result of video notion mark, thereby help understanding and management to video content.

For reaching above purpose, technical scheme of the present invention is as follows:

A kind of based on contextual video notion mask method between camera lens, may further comprise the steps:

1) given video training set with artificial annotation results, for each semantic concept training that marks obtains concept detection, video lens in training set and the test set to be marked is input to concept detection, exports the prediction probability that comprises corresponding concepts in each camera lens;

2) with a certain given semantic concept as target concept, according to the relation of notion that comprises in the training set video lens and target concept, be shot classification dissimilar, and extract the lens type feature;

3) based on step 2) in shot classification and the type feature of extraction, mutual relationship between the camera lens in the training set video is carried out modeling, the relational model that makes up is represented the type feature of a certain camera lens and the relation between the type mark, represent the relation between the type mark of adjacent camera lens simultaneously, extract the type feature of test set video lens, the input relational model obtains the probability that camera lens is labeled as a certain type, and the prediction probability after optimizing as target concept thus;

4) repeating step 2) and step 3), notions all in test set are all as till the optimised mistake of target concept, at this moment, if the probability that camera lens comprises certain notion then marks this camera lens with this notion greater than given threshold value.

Described step 1) is divided into positive sample and negative sample to training set according to artificial annotation results, utilizes the method training of machine learning to obtain sorter then, and this sorter is promptly as detection of corresponding concepts.

Described step 2) camera lens is divided into four types: target concept camera lens, positive correlation notion camera lens, negative correlation notion camera lens, and unrelated concepts camera lens.

Four class camera lenses are defined as follows: comprise target concept in the target concept camera lens; Positive correlation notion camera lens does not comprise target concept, but comprises and the positively related notion of target concept; Negative correlation notion camera lens does not comprise target concept and positively related notion thereof, but comprises the notion with the target concept negative correlation; The unrelated concepts camera lens neither comprises target concept, does not also comprise the notion with target concept positive correlation or negative correlation.

Given a certain target concept, notion and target concept positive correlation represent that this notion and target concept semantically are being mutually related; Notion and target concept negative correlation represent that this notion and target concept are to repel mutually semantically.

Described step 2) in, type feature extracts according to the prediction probability of concept detection output, and expression is shot classification dissimilar probability.

In the described step 3), the camera lens that belongs to a video in the training set is arranged according to the time order and function relation, obtained shot sequence, training obtains the relational model between the camera lens based on shot sequence.

The given threshold value of described step 3) is 0.5.

Effect of the present invention is: compare with existing method, the present invention can obtain video notion annotation results more accurately, thus give full play to notion be labeled in video content analysis and understand in vital role.Why the present invention has the foregoing invention effect, and its reason is: the present invention has taken into full account the relation of the difference between the camera lens in the video, comprises consistance, positive correlation, negative correlation and irrelevant, can more effectively optimize the result of notion mark.

Description of drawings

Fig. 1 is a video notion mark synoptic diagram;

Fig. 2 is the schematic flow sheet of the inventive method.

Embodiment

The present invention is described in further detail below in conjunction with the drawings and specific embodiments.

As shown in Figure 2, of the present invention a kind of based on contextual video notion mask method between camera lens, specifically comprise following steps:

(1) obtains concept detection for each the semantic concept training in the training set

Need to be divided into positive sample and negative sample to training set according to artificial annotation results, utilize the method training of machine learning to obtain sorter then, this sorter is promptly as detection of corresponding concepts.Present embodiment adopts the method that proposes in the document " Columbia university ' s baseline detectors for 374 LSCOM semantic visualconcepts " (author is A.Yanagawa and S.F.Chang) in the Columbia university technical report in 2007 to obtain concept detection.This method is at first extracted three kinds of low-level image features to video lens: i.e. color moment feature, Gabor textural characteristics, and edge histogram feature; Obtain three based on these three kinds of features training respectively then and detect son, for a pending camera lens, these three detection are exported a prediction probability respectively, and the mean value of getting these three prediction probabilities comprises the probability of corresponding concepts as this camera lens.Therefore, in the method, detection of each notion detects son by three and forms.

Symbol definition:, earlier some symbols that will use are described in order hereinafter to clearly demonstrate method used in the present invention.

The camera lens note that belongs in the same video is made S={s ₁, s ₂..., s _n, wherein, the subscript of camera lens is according to the time order and function order assignment of camera lens in video: camera lens s _I-1At camera lens s _iBefore, camera lens s _I+1At camera lens s _iAfterwards, n is the number that comprises camera lens in the video.

Use L={l ₁, l ₂..., l _nThe artificial mark of camera lens among the expression S, wherein, Camera lens s in the expression training set _iArtificial mark,

Value be 1 expression camera lens s _iComprise notion c _j,

Value be 0 expression this camera lens s _iDo not contain notion c _jM is the number of notion in the semantic dictionary.Use L _TrainArtificial concept's mark of expression training set.

Use P={P ¹, P ²..., P ^mRepresent the prediction probability of concept detection to camera lens among the S, wherein,

Be notion c _jThe prediction probability of detection output, Be notion c _jDetection to camera lens s _iPrediction probability.Use P _TrainAnd P _TestRepresent the prediction probability of concept detection respectively to camera lens output in training set and the test set.

Use Y={y ¹, y ²..., y ^mThe type of camera lens among the expression S, wherein,

Be with c _jDuring for target concept, according to notion that comprises in the camera lens and c _jRelation camera lens is carried out sorting result, Be camera lens s _iType.Use Y _TrainThe type mark of expression training set.

Use X={x ¹, x ²..., x ^mThe type feature of camera lens among the expression S, wherein, Be with notion c _jDuring for target concept, to the type feature of camera lens extraction.

Be camera lens s _iType feature.Use X _TrainAnd X _TestThe type feature of representing camera lens in training set and the test set respectively.

With

Expression utilizes the result after relational model between camera lens is optimized the prediction probability of camera lens among the S,

Be to notion c _jThe result of prediction probability after being optimized,

Be to notion c _jAt camera lens s _iOn the result of prediction probability after being optimized.With

Camera lens is through the prediction probability after optimizing in the expression test set.

(2) for a certain given semantic concept c _j, be referred to as target concept, according to notion that is comprised in the camera lens and the relation between the target concept camera lens in the training set is classified.

We are divided into four classes to the camera lens in the training set: the target concept camera lens is designated as

Positive correlation notion camera lens is designated as

Negative correlation notion camera lens is designated as

With the unrelated concepts camera lens, be designated as

Four class camera lenses are defined as follows: comprise notion c in the target concept camera lens _jPositive correlation notion camera lens does not comprise notion c _j, but comprise and c _jPositively related notion; Negative correlation notion camera lens does not comprise notion c _jAnd positively related notion, but comprise and c _jThe notion of negative correlation; The unrelated concepts camera lens neither comprises notion c _j, also do not comprise and c _jThe notion of positive correlation or negative correlation.Here these four types are designated as

(3) pay close attention to target concept c _j, the type feature of camera lens in extraction training set and the test set.In the present embodiment, we are based on concept detection the prediction probability of video lens are extracted type feature, also can adopt method for distinguishing, and for example the low-level image feature based on video lens extracts type feature etc.

Camera lens s _iType feature be the proper vector of a four-dimension, be designated as

Each dimension value is by following formula definition:

Formula one: $x_{i,\mathrm{consist}}^j={P_i}^j$

Formula two: $x_{i,\mathrm{pos}}^j=(1-{P_i}^j)\×(1-\underset{c_h\∈{\mathrm{POS}}_j}{\mathrm{\Π}}(1-{P_i}^h))$

Formula three: $x_{i,\mathrm{neg}}^j=(1-{P_i}^j)\×\underset{c_h\∈{\mathrm{POS}}_j}{\mathrm{\Π}}(1-{P_i}^h)\×(1-\underset{c_h\∈{\mathrm{NEG}}_j}{\mathrm{\Π}}(1-{P_i}^h))$

Formula four: $x_{i,\mathrm{irrel}}^j=(1-{P_i}^j)\×\underset{c_h\∈{\mathrm{POS}}_j}{\mathrm{\Π}}(1-{P_i}^h)\×\underset{c_h\∈{\mathrm{NEG}}_j}{\mathrm{\Π}}(1-{P_i}^h))$

Wherein, P _i ^jBe notion c _jDetection to camera lens s _iOutput valve, i.e. camera lens s _iComprise notion c _jPrediction probability.POS _jBe and notion c _jPositively related notion set, NEG _jBe and notion c _jThe notion set of negative correlation.Notion and c _jPositive correlation represents that this notion usually and c _jIn same camera lens, occur; Notion and c _jNegative correlation is represented this notion c that gets along well substantially _jIn same camera lens, occur.Formula five and formula six have provided POS _jAnd NEG _jDefinition, satisfy the notion c of formula five _hBelong to set POS _j, satisfy the notion c of formula six _hBelong to set NEG _j

Formula five: ${\mathrm{NormMI}}_{\mathrm{pos}}(c_h,c_j)>{\mathrm{Th}}_{\mathrm{pos}}^j$

Formula six: ${\mathrm{NormMI}}_{\mathrm{neg}}(c_h,c_j)>{\mathrm{Th}}_{\mathrm{neg}}^j$

Wherein, NormMI _Pos(c _h, c _j) and NormMI _Neg(c _h, c _j) value provide by formula seven, eight, nine, ten, 11: NormMI _Pos(c _h, c _j) value big more, the expression c _hWith c _jPositively related degree is strong more, NormMI _Neg(c _h, c _j) value big more, the expression c _hWith c _jThe degree of negative correlation is strong more. With

Be respectively a threshold value, in the present embodiment, we

Be taken as all NormMI _Pos(c _h, c _j) in the sixth-largest value (h ≠ j), like this, satisfy the notion c of formula five _hThere are and only have five,

Value also taked same method.

Formula seven: NormMI _Pos(c _h, c _j)=MI _Pos(c _h, c _j)/min{H (c _h), H (c _j)

Formula eight: NormMI _Neg(c _h, c _j)=MI _Neg(c _h, c _j)/min{H (c _h), H (c _j)

Formula nine:

Formula ten:

Formula 11: $H\left(c_j\right)=-\underset{l^j\∈\{-\mathrm{1,1}\}}{\mathrm{\Σ}}p\left(l^j\right)\log p\left(l^j\right)$

Wherein, l ^jValue be 1 expression notion c _jL appears in camera lens ^jValue be 0 expression notion c _jIn camera lens, do not occur.P (l ^j) be notion c _jThe prior probability that whether occurs in camera lens can be by statistics training centralized concept c _jThe number of times that occurs obtains.

(4) the target concept annotation results is optimized

The present invention adopts condition random field, and (Conditional Random Field CRF) carries out modeling for the mutual relationship between the camera lens in the video.In addition, also can utilize method for distinguishing to carry out modeling, for example Markov random field etc.Condition random field is on the International of calendar year 2001 Conference on Machine Learning, proposed in document " Conditional random fields:Probabilistic models for segmenting and labelingsequence data " by people such as J.Lafferty, this method is used to solve the sequence labelling problem.In condition random field, the type of node i mark y _iThe type feature that not only depends on this node also depends on the annotation results of its neighborhood of nodes.In the present invention, the camera lens that belongs to same video is arranged in a shot sequence according to time sequencing, each camera lens will be noted as

One of these four types, like this, the problem of camera lens mark can well solve with condition random field with regard to can be regarded as a sequence labelling problem, and camera lens is noted as type

Probability then be counted as and comprise notion c in the camera lens _jProbability, can be used as notion c _jPrediction probability after the optimization, owing to the dependence of having considered between the camera lens, the probable value after the optimization is more accurate, provides the definition of condition random field below:

Make X and Y be respectively two random vectors, (V E) is a non-directed graph to G=, and v ∈ V is a node among the figure, and v is used for corresponding stochastic variable y _v, the variable among so vectorial Y can be corresponding one by one by the node among the G.As variable y _vSatisfy when scheme Markov property that G represents that node concerns about the conditional probability distribution of X, we claim＜X, Y〉be a condition random field, i.e. P (y _v| X, y _w, w ≠ v)=P (y _v| X, y _w, w～v), wherein, w～v represents that node w and v are neighborhood of nodes in figure G.

In practical problems, X represents the type feature of node usually, and Y is the type mark of node.(author is J.Lafferty as document " Conditional random fields:Probabilistic models for segmenting and labelingsequence data ", A.McCallum and F.Pereira, be published on the International Conference on Machine Learning of calendar year 2001) described in, under the prerequisite of given X, the conditional probability of Y is provided by formula 12:

Formula 12: $P\left(Y\right|X)=\frac{1}{Z}\exp (\underset{e\∈E}{\mathrm{\Σ}}I(e,Y|e,X,\mathrm{\λ})+\underset{v\∈V}{\mathrm{\Σ}}A(v,Y|v,X,\mathrm{\μ}))$

Wherein, D={e, v} are the subgraphs of G, and Y|D is a stochastic variable relevant with subgraph D among the Y; A is the auto-correlation potential function of condition random field, is used for to the type feature of node and the relationship modeling between the type mark; I is the simple crosscorrelation potential function, is used for the relation between the different node type mark is carried out modeling; Z is used for to result of calculation normalization, makes value value between 0 to 1 of P (Y|X).In particular problem, in case the definition of given auto-correlation potential function and simple crosscorrelation potential function, the process of utilizing condition random field to mark can be divided into two stages: training and prediction, and the training stage is to parameter θ=(λ μ) learns; After obtaining θ, the target of forecast period will find a mark sequence Y exactly ^*, make P (Y|X) get maximal value, shown in formula 13:

Formula 13: $Y^{*}=\underset{Y}{\arg \max }P\left(Y\right|X,\mathrm{\θ})$

About the concrete study of condition random field with forecasting process can (author be J.Lafferty referring to document " Conditional random fields:Probabilistic models for segmenting and labeling sequence data ", A.McCallum and F.Pereira are published on the International Conference on Machine Learning of calendar year 2001).

In the present invention, two potential functions in our the following definite condition random field, i.e. auto-correlation potential function (AssociationPotential) and simple crosscorrelation potential function (Interactive Potential):

The auto-correlation potential function is used for representing the type feature of camera lens and the relation between the type mark, for target concept c _j, definition auto-correlation potential function is as follows:

Formula 14: $A(s_i,y^j,x^j)=\left\{\begin{array}{cc}{x}_{i,\mathrm{consist}}^j& y_i^j=T_{\mathrm{consist}}^j\\ x_{i,\mathrm{pos}}^l& y_i^j=T_{\mathrm{pos}}^j\\ x_{i,\mathrm{neg}}^j& y_i^j=T_{\mathrm{neg}}^j\\ x_{i,\mathrm{irrel}}^j& y_i^j=T_{\mathrm{irrel}}^j\end{array}\right.$

As above-mentioned definition,

Be with c _jCamera lens s during for objective function _iType mark,

It is the type mark of n camera lens in the video.

Be the type feature of n camera lens,

Be camera lens s _iType feature.

The simple crosscorrelation potential function is used for representing the mutual relationship between the dissimilar lens type marks, for target concept c _j, definition auto-correlation potential function is as follows:

Formula 15: $I(y_i^j,y_{i+1}^j,x^j)=\underset{t^{\′},t\∈T^j}{\mathrm{\Σ}}{\mathrm{\λ}}_{t^{\′},t}\mathrm{\δ}(y_i^j=t^{\′})\mathrm{\δ}(y_{i+1}^j=t)$

Wherein, δ (.) is an indicator function, and when the condition of parenthetic statement is a true time, the value of this function is 1, otherwise is 0.

Be based on target concept c _jThe lens type of definition.

After the definition that obtains auto-correlation potential function and simple crosscorrelation potential function, can obtain relational model M between camera lens by training set study _j, the video that marks for needs extracts the type feature of camera lens, input model M according to the method in the step (3) _j, can obtain camera lens is labeled as type

Marginal probability, as notion c _jPrediction probability after the optimization is if the probability after optimizing is then used notion c greater than threshold value 0.5 _jCamera lens is marked.

Below experimental result show that compare with existing method, the present invention can obtain notion annotation results more accurately, thereby give full play to the huge effect of video notion in being labeled in video content analysis and understanding.

Present embodiment has adopted the data set of TRECVID 2006 to evaluate and test, and TRECVID is the authoritative activity of assessment and test by the video frequency searching field of American National Standard Technical Board tissue.This data set is divided into two parts: training set and test set.Comprise 85 hours news video program in the training set, related to Arabic news, Chinese news and English news.These news programs are made up of 137 videos, are split into 43907 camera lenses.(author is A.Yanagawa and S.F.Chang etc. to document " Columbia university ' s baseline detectorsfor 374 lscom semantic visual concepts ", be a technical report of Columbia university) camera lens in these training sets has been carried out artificial mark, each camera lens is noted as corresponding notion, and present embodiment has adopted the labeled data of these data as training set.Test set has comprised 259 videos, and these videos are split into 79484 camera lenses, and we, evaluate and test on 20 notions that TRECVID 2006 officials adopt after these camera lenses are handled with different notion mask methods.We have tested following 3 kinds of methods and have contrasted as experiment:

I. document " Columbia university ' s baseline detectors for 374 LSCOM semantic visualconcepts " (author A.Yanagawa and S.F.Chang, be the technical report of Columbia university in 2007) in method, this method adopts support vector machine as sorter, directly sets up the mapping relations between video low-level image feature and the semantic concept;

II. document " Association and temporal rule mining for post-processing of semanticconcept detection in video " (author K.H.Liu and M.F.Weng, be published on the IEEETransactions on Multimedia in 2008) in method, this method adopts rule-based method that the notion annotation results is optimized on the basis of method I;

III. document " Multi-cue fusion for semantic video indexing " (author M.F.Weng and Y.Y.Chuang, be published on the ACM Multimedia Conference in 2008) in method, this method is utilized in the video to come the prediction probability of notion is optimized in consistance semantically between the adjacent camera lens on the basis of method I;

IV. the method for present embodiment.

Document " Estimating average precision with incomplete and imperfectjudgments " is adopted in experiment, and (author is E.Yilmaz and J.A.Aslam, be published on the ACM Conference onInformation and Knowledge Management in 2006) in the InfAP that proposes as evaluation metrics, the value of InfAP is big more, illustrates that the result of notion mark is just good more.

The contrast of table 1 experimental result

	InfAP
		Method I	0.0948
Method II	0.1108
		Method III	0.1112
IV of the present invention	0.1252

As can be seen from Table 1, IV of the present invention has obtained best notion annotation results, control methods I and IV of the present invention, the context of not using among the method I between camera lens comes the notion annotation results is optimized, and in IV of the present invention, utilize the mutual relationship between the camera lens can improve the performance that notion marks.Control methods II, III and IV of the present invention, method II are based on the method for rule, and it is always limited to sum up the rule of coming out, has limitation, only consider the consistance between the adjacent camera lens among the method III, and do not consider other relations between the camera lens therefore have further improved space.The present invention utilizes condition random field that the mutual relationship between the video lens is carried out modeling, comprises consistance, and positive correlation and negative correlation etc. have been improved the accuracy rate that notion marks greatly.

Obviously, those skilled in the art can carry out various changes and modification to the present invention and not break away from the spirit and scope of the present invention.Like this, if of the present invention these are revised and modification belongs within the scope of claim of the present invention and equivalent technologies thereof, then the present invention also is intended to comprise these changes and modification interior.

Claims (8)

1. one kind based on contextual video notion mask method between camera lens, may further comprise the steps:

1) given video training set with artificial annotation results, for each semantic concept training that marks obtains concept detection, video lens in training set and the test set to be marked is input to concept detection, exports the prediction probability that comprises corresponding concepts in each camera lens;

2) with a certain given semantic concept as target concept, according to the relation of notion that comprises in the training set video lens and target concept, be shot classification dissimilar, and extract the lens type feature;

3) based on step 2) in shot classification and the type feature of extraction, mutual relationship between the camera lens in the training set video is carried out modeling, the relational model that makes up is represented the type feature of a certain camera lens and the relation between the type mark, represent the relation between the type mark of adjacent camera lens simultaneously, extract the type feature of test set video lens, the input relational model obtains the probability that camera lens is labeled as a certain type, and the prediction probability after optimizing as target concept thus;

4) repeating step 2) and step 3), notions all in test set are all as till the optimised mistake of target concept, at this moment, if the probability that camera lens comprises certain notion then marks this camera lens with this notion greater than given threshold value.

2. video notion mask method as claimed in claim 1, it is characterized in that, described step 1) is divided into positive sample and negative sample to training set according to artificial annotation results, utilizes the method training of machine learning to obtain sorter then, and this sorter is promptly as detection of corresponding concepts.

3. video notion mask method as claimed in claim 1 is characterized in that described step 2) camera lens is divided into four types: target concept camera lens, positive correlation notion camera lens, negative correlation notion camera lens, and unrelated concepts camera lens.

4. video notion mask method as claimed in claim 3 is characterized in that, given a certain target concept, and four class camera lenses are defined as follows: comprise target concept in the target concept camera lens; Positive correlation notion camera lens does not comprise target concept, but comprises and the positively related notion of target concept; Negative correlation notion camera lens does not comprise target concept and positively related notion thereof, but comprises the notion with the target concept negative correlation; The unrelated concepts camera lens neither comprises target concept, does not also comprise the notion with target concept positive correlation or negative correlation.

5. video notion mask method as claimed in claim 4 is characterized in that, given a certain target concept, and notion and target concept positive correlation represent that this notion and target concept semantically are being mutually related; Notion and target concept negative correlation represent that this notion and target concept are to repel mutually semantically.

6. video notion mask method as claimed in claim 1 is characterized in that described step 2) in, type feature extracts according to the prediction probability of concept detection output, and expression is shot classification dissimilar probability.

7. video notion mask method as claimed in claim 1, it is characterized in that, in the described step 3), the camera lens that belongs to a video in the training set is arranged according to the time order and function relation, obtain shot sequence, training obtains the relational model between the camera lens based on shot sequence.

8. video notion mask method as claimed in claim 1 is characterized in that, the given threshold value of described step 3) is 0.5.

Images