A kind of face video search method and system
Technical field
The present invention relates to field of video retrieval, more particularly to a kind of face video search method and system.
Background technology
With developing rapidly for multimedia technology and computer networking technology, video is increasingly becoming the main flow of Information Communication and carries
One of body.People's problems faced has no longer been the scarcity of video content, but in the face of vast as the open sea video information, how soon
Speed, the content for efficiently finding oneself needs.Wherein, in social public security field, video monitoring system becomes maintenance society
Public security, strengthens an important component part of social management.Face video is retrieved, and is become in public security user monitoring system and is compeled to be essential
Ask.As current most popular video search technique, the either Video content retrieval based on uncompressed domain and based on compression domain
Video content retrieval, this conventional design pattern, not using face retrieval the characteristics of, so as to affect face video retrieve skill
The efficiency of art.
The content of the invention
The purpose of the embodiment of the present invention is to propose a kind of face video search method, it is intended to solve existing face video inspection
The low problem of rope technical efficiency.
The embodiment of the present invention is achieved in that a kind of face video search method, the method comprising the steps of:
Step A:Judge current search video present frame pictJudgement parameter partWhether it is 1, if then entering step B,
Otherwise, into step E;
Step B:Present frame is scanned for using the first video search pattern;
Step C:If the next frame of current search video present frame is present, t=t+1 is made, and by current search video
The next frame of present frame is set to current search video present frame, subsequently into step D;Otherwise, terminate;T represents search video
The frame number of sequence, the initial value of t is 1;
Step D:If there is no sbkt(i, j)=1, then into step E;Otherwise enter step G.
sbkt(i, j) represents bkt(i, j) identification parameter, bkt(i, j) represents pictThe i-th row jth row solution code block;
Step E:If current search video present frame pictFor infra-frame prediction frame, then tp is madet=bkh*bkw;Otherwise, count
Calculate tpt=sum (sign (bkt(i, j) | condition 2) | 1≤i≤bkh and 1≤j≤bkw);
Step F:If tpt=0, then first, all sbk are sett(i, j)=0, subsequently into step C;Otherwise, if
tpt>=0.9*bkh*bkw, then into step B;Otherwise, then into step G;Bkw, bkh represent that respectively a two field picture divides in bulk
After, columns and line number of the image in units of block;
Step G:Present frame is scanned for using the second video search pattern, then, into step C;
First video search pattern is comprised the following steps:
Decoding current search video present frame, obtains decoding image;
It is handled as follows to decoding all solution code blocks of image:If bkt(i, j) predictive mode is sub-block prediction pattern, then
Into subdivision determinating mode;Otherwise, into rough segmentation determinating mode;
Unified current search region and the resolution of search target, then, with unified resolution current search area are scaled
Domain and search target are to same size;
First to the region of search of current decoding image, characteristics of image is extracted;Then contrasted with search target,
Match somebody with somebody, complete the search to current search video present frame;
By the matching result of current search video present frame, each solution code block of current search video present frame is known
Other parameter identification;
Wherein, sbkt(i, j)=sign (bkt(i, j) | condition 3), condition 3 is represented:bkt(i, j) matches target.
The another object of the embodiment of the present invention is to propose a kind of face video searching system, and the system includes:
First judging treatmenting module, for judging current search video present frame pictJudgement parameter partWhether it is 1,
If the first video searching apparatus are then entered, otherwise into scene handoff parameter computing module;
Wherein, partRepresent pictJudgement parameter,pictRepresent current search video t
Frame, t represents the frame number of search video sequence, and the initial value of t is 1;Condition 1 is represented:T=1OrpicT be infra-frame prediction frame ortpt≥
0.9*bkh*bkw;tptFor scene handoff parameter, tpt=sum (sign (bkt(i, j) | condition 2) | 1≤i≤bkh and 1≤j≤
bkw);sum(Variable|Condition) represent that the variable to meeting condition is sued for peace;Condition 2
Represent:bkt(i, j) is for intra-frame prediction block or including at least an infra-frame prediction sub-block;bkt(i, j) represents pictThe i-th row
Jth solution code block bkw, bkh represent that respectively a two field picture is divided after in bulk, columns and line number of the image in units of block;
First video searching apparatus, for being scanned for present frame using the first video search pattern;
Second judging treatmenting module, for judging that the next frame of current search video present frame whether there is, if so, then makes
T=t+1, and the next frame of current search video present frame is set to into current search video present frame, sentence subsequently into the 3rd
Disconnected processing module, otherwise terminates;
3rd judging treatmenting module, for judging whether to there is sbkt(i, j)=1, if not existing, into scene
Handoff parameter computing module, otherwise into the second video searching apparatus;
Scene handoff parameter computing module, if for judging current search video present frame pictFor infra-frame prediction frame,
Then make tpt=bkh*bkw;Otherwise calculate tpt=sum (sign (bkt(i, j) | condition 2) | 1≤i≤bkh and 1≤j≤bkw);
4th judging treatmenting module, for judging whether tpt=0, if then arranging all sbkt(i, j)=0, Ran Houjin
Enter the second judging treatmenting module;Otherwise, if judging tpt>=0.9*bkh*bkw, then into the first video searching apparatus;Otherwise,
Then enter the second video searching apparatus;
Second video searching apparatus, for being scanned for present frame using the second video search pattern, subsequently into
Two judging treatmenting modules;
First video searching apparatus include:
Decoding image collection module, for decoding current search video present frame, obtains decoding image;
Predictive mode determination module, if for judging bkt(i, j) predictive mode is sub-block prediction pattern, then into thin
Divide decision maker;Otherwise, into rough segmentation decision maker;
First size unified modules, are connected with predictive mode determination module, for unifying current search region with search mesh
Target resolution, then, with unified resolution scaling current search region and search target to same size;
First object image search module, to the region of search of current decoding image, characteristics of image is extracted for first;So
Contrasted with search target afterwards, matched, completed the search to current search video present frame;
First identification parameter mark module, for pressing the matching result of current search video present frame, regards to current search
Each solution code block of frequency present frame is identified parameter identification;
Wherein, sbkt(i, j)=sign (bkt(i, j) | condition 3), sbkt(i, j) represents bktThe identification ginseng of (i, j)
Number;Condition 3 is represented:bkt(i, j) matches target.
Beneficial effects of the present invention
The present invention proposes a kind of face video search method, and the inventive method determines key frame by the information of uncompressed domain
Region of search, the then motion by compression domain and information of forecasting obtain Tracing region search, so as to less video search
Data volume and operand, lift the ageing of video search;Additionally, the characteristics of this method is also directed to face retrieval, by reducing
Region of search, reduces amount of calculation;By pretreatment, the accuracy rate of search is lifted.
Description of the drawings
Fig. 1 is a kind of face video search method flow chart of the preferred embodiment of the present invention;
Fig. 2 is the method flow diagram of Step1 in Fig. 1;
Fig. 3 is a kind of face video searching system structure chart of the preferred embodiment of the present invention;
Fig. 4 is the first video searching apparatus structure chart in Fig. 3;
Fig. 5 is subdivision decision maker structure chart in Fig. 4;
Fig. 6 is rough segmentation decision maker structure chart in Fig. 4;
Fig. 7 is the second video searching apparatus structure chart in Fig. 3.
Specific embodiment
In order that the objects, technical solutions and advantages of the present invention become more apparent, it is right below in conjunction with drawings and Examples
The present invention is further elaborated, and for convenience of description, illustrate only the part related to the embodiment of the present invention.Should manage
Solution, the specific embodiment that this place is described is used only for explaining the present invention, not to limit the present invention.
The embodiment of the present invention proposes a kind of face video search method and system, and present invention method is by uncompressed
The information in domain determines the region of search of key frame, then the motion by compression domain and information of forecasting, obtains Tracing region search,
So as to the data volume and operand of less video search, the ageing of video search is lifted;Additionally, this method is also directed to face inspection
The characteristics of rope, by reducing region of search, reduce amount of calculation;By pretreatment, the accuracy rate of search is lifted.
Embodiment one
Fig. 1 is a kind of face video search method flow chart of the preferred embodiment of the present invention;The method comprising the steps of:
Step:0:Judge parameter partFor 1, then into Step1, otherwise, into Step4.
Wherein, partRepresent pictJudgement parameter,pictRepresent current search video t frames (i.e.
Current search video present frame), t represents the frame number of search video sequence, and the initial value of t is 1;Condition 1 is represented:T=1Or
picT be infra-frame prediction frame ortpt≥0.9*bkh*bkw;tptFor scene handoff parameter, tpt=sum (sign (bkt(i, j) | condition 2) | 1
≤ i≤bkh and 1≤j≤bkw);sum(Variable|Condition) represent that the variable to meeting condition is sued for peace;
Condition 2 is represented:bkt(i, j) is for intra-frame prediction block or including at least an infra-frame prediction sub-block;bkt(i, j) represents pict's
(size of block is 16x16 (standard such as H264), 64x64 (HEVC) to i-th row jth solution code block, and when block Further Division, these are little
Sized blocks are referred to as sub-block), bkw, bkh represent that respectively a two field picture is divided after in bulk, columns and row of the image in units of block
Number;
Step1:Using the first video search pattern, present frame is scanned for.
First video search pattern (Fig. 2 is the method flow diagram of Step1 in Fig. 1):
Step11:Decoding current search video present frame, obtains decoding image.
Step12:According to the characteristics of recognition of face, to decoding image region of search delimited;I.e. to decoding all decodings of image
Block is handled as follows:If bkt(i, j) predictive mode has made Further Division for sub-block prediction pattern, i.e. block, then enter subdivision
Determinating mode;Otherwise, into rough segmentation determinating mode.Subdivision determinating mode:
Step A1:Using each pixel in block as colour of skin decision-point, colour of skin judgement is made to the colour of skin decision-point, if
It is the colour of skin to meet colour of skin decision-point, then the block skin pixel point number adds 1.
Step A2:As skin pixel point number is more than the 14th threshold value in fruit block, then judge that the block puts face video under and searches
Rope region, otherwise, the block puts non-face video search region under.14th upper threshold is the pixel quantity sum of block, under
The half of the pixel quantity sum of optional piece of limit.
Rough segmentation determinating mode:
Step B1:Using in units of pixel average in block as colour of skin decision-point, i.e., with all pixels point in block accordingly minute
Value of the average of amount as each color model component.
Step B2:Colour of skin judgement is done to colour of skin decision-point, if the colour of skin decision-point is the colour of skin, face video is put under and is searched
Rope region;Otherwise, the block puts non-face video search region under.
In the subdivision determinating mode and rough segmentation judgment model, if colour of skin decision-point must be while meet following 6 for the colour of skin
Condition:
Require 1;Thres1<b-g<Thres2, require 2:Thres3<r-g<Thres4*Wr、
Require 3:Gup<g<Gdown, require 4:Thres5<Wr, require 5:Thres6<Co<Thres7、
Require 6:Thres8<energyUV<Thres9&&U*Thres10<V&&U*Thres11>V or Thres12<
energyUV<Thres13
Wherein, Thresjj, jj ∈ [1,13] are respectively the first to the 13rd threshold value, and the first to the 13rd threshold value is according to reality
Border situation sets itself;Based on normalizing RGB models,Obtain normalization rgb color component r, g, b;Color
Harmonious parameter Wr=(r-1/3)2+(g-1/3)2;Build green component upper bound model Gup=aupr2+bupr+cup, wherein aup,
bup,cupFor model parameter, Gdown=adownr2+bdownr+cdown;Wherein adown,bdown,cdownFor model parameter;Based on model
YUV modelsObtain color energyY is
Luminance component, U, V represent respectively two chromatic components of YUV models;Based on YCoCg modelsObtain
Co, Co are YCgCo model color component values;
In first video search pattern, region of search includes face video region of search and non-face video search region;
The colour of skin judges point methods, it is also possible to disclosed any method in the industry.
Step13:Unified current search region and the resolution of search target, then, are scaled current with unified resolution
Region of search and search target are to same size.
Step14:First, to the region of search of current decoding image, characteristics of image is extracted;Then carry out with search target
Contrast, matching, completes the search to current search video present frame.
Wherein, the extraction characteristics of image and search target are contrasted, and matching process can be with correspondence video search field
Interior disclosed any one method, will not be described here.
Step15:By the matching result of current search video present frame, current search video present frame each is decoded
Block is identified parameter identification.
Wherein, sbkt(i, j)=sign (bkt(i, j) | condition 3), sbkt(i, j) represents bktThe identification ginseng of (i, j)
Number;Condition 3 is represented:bkt(i, j) matches target.
Step2:If the next frame of current search video present frame is present, t=t+1 is made, and by current search video
The next frame of present frame is set to current search video present frame, subsequently into Step3;Otherwise, terminate.
Step3:If there is no sbkt(i, j)=1, then into Step4;Otherwise enter Step6.
Step4:If pictFor infra-frame prediction frame, then tp is madet=bkh*bkw;Otherwise, tp is calculatedt=sum (sign (bkt
(i, j) | condition 2) | 1≤i≤bkh and 1≤j≤bkw).
Step5:If tpt=0, then first, all sbk are sett(i, j)=0, subsequently into Step2;Otherwise, if
tpt>=0.9*bkh*bkw, then into Step1;Otherwise, then into Step6.
Step6:Using the second video search pattern, present frame is scanned for, then, into Step2.
Second video search pattern:
Step61:If bkt(i, j) is intra-frame prediction block, then decode the block, then delimits the block for region of search;It is no
Then,
If spbkt(i, j)=1, then arrange sbkt(i, j)=1, that is, represent current Block- matching target;Otherwise, then arrange
sbkt(i, j)=0, that is, represent that current block mismatches target.
Wherein, spbkt(i, j) represents bktThe identification parameter of the reference block of (i, j).
Step62:Pretreatment is carried out to current search region, that is, unifies the resolution in current search region and search target,
Then, the size with unified resolution scaling current search region with search target to as.
Step63:First, to region of search, characteristics of image is extracted, is then contrasted with search target, matched, completed
Search to current search video present frame.
Wherein, the extraction characteristics of image and search target are contrasted, and matching process can be with correspondence video search field
Interior disclosed any one method, will not be described here.
Step64:The matching result of code block is solved by region of search, to solving code block parameter identification is identified.
Embodiment two
Fig. 3 is a kind of face video searching system structure chart of the preferred embodiment of the present invention;The system includes:
First judging treatmenting module, for judging current search video present frame pictJudgement parameter partWhether it is 1,
If the first video searching apparatus are then entered, otherwise into scene handoff parameter computing module;
Wherein, partRepresent pictJudgement parameter,pictRepresent current search video t frames (i.e.
Current search video present frame), t represents the frame number of search video sequence, and the initial value of t is 1;Condition 1 is represented:T=1Or
picT be infra-frame prediction frame ortpt≥0.9*bkh*bkw;tptFor scene handoff parameter, tpt=sum (sign (bkt(i, j) | condition 2) | 1
≤ i≤bkh and 1≤j≤bkw);sum(Variable|Condition) represent that the variable to meeting condition is sued for peace;
Condition 2 is represented:bkt(i, j) is for intra-frame prediction block or including at least an infra-frame prediction sub-block;bkt(i, j) represents pict's
(size of block is 16x16 (standard such as H264), 64x64 (HEVC) to i-th row jth solution code block, and when block Further Division, these are little
Sized blocks are referred to as sub-block), bkw, bkh represent that respectively a two field picture is divided after in bulk, columns and row of the image in units of block
Number;
First video searching apparatus, for being scanned for present frame using the first video search pattern;
Second judging treatmenting module, for judging that the next frame of current search video present frame whether there is, if so, then makes
T=t+1, and the next frame of current search video present frame is set to into current search video present frame, sentence subsequently into the 3rd
Disconnected processing module, otherwise terminates.
3rd judging treatmenting module, for judging whether to there is sbkt(i, j)=1, if not existing, into scene
Handoff parameter computing module, otherwise into the second video searching apparatus;
Scene handoff parameter computing module, if for judging current search video present frame pictFor infra-frame prediction frame,
Then make tpt=bkh*bkw;Otherwise calculate tpt=sum (sign (bkt(i, j) | condition 2) | 1≤i≤bkh and 1≤j≤bkw).
4th judging treatmenting module, for judging whether tpt=0, if then arranging all sbkt(i, j)=0, Ran Houjin
Enter the second judging treatmenting module;Otherwise, if judging tpt>=0.9*bkh*bkw, then into the first video searching apparatus;Otherwise,
Then enter the second video searching apparatus.
Second video searching apparatus, for being scanned for present frame using the second video search pattern, subsequently into
Two judging treatmenting modules;
Further, Fig. 4 is the first video searching apparatus structure chart in Fig. 3, and first video searching apparatus include:
Decoding image collection module, for decoding current search video present frame, obtains decoding image;
Predictive mode determination module, if for judging bkt(i, j) predictive mode is sub-block prediction pattern, then into thin
Divide decision maker;Otherwise, into rough segmentation decision maker.
First size unified modules, are connected with predictive mode determination module, for unifying current search region with search mesh
Target resolution, then, with unified resolution scaling current search region and search target to same size;
First object image search module, to the region of search of current decoding image, characteristics of image is extracted for first;So
Contrasted with search target afterwards, matched, completed the search to current search video present frame.
First identification parameter mark module, for pressing the matching result of current search video present frame, regards to current search
Each solution code block of frequency present frame is identified parameter identification.
Wherein, sbkt(i, j)=sign (bkt(i, j) | condition 3), sbkt(i, j) represents bktThe identification ginseng of (i, j)
Number;Condition 3 is represented:bkt(i, j) matches target.
Further, Fig. 5 is subdivision decision maker structure chart in Fig. 4;
Subdivision decision maker, including block skin pixel point counting module and the first face video search region division module,
Block skin pixel point counting module, as colour of skin decision-point, sentences for using each pixel in block to the colour of skin
Fixed point makees colour of skin judgement, if it is the colour of skin to meet colour of skin decision-point, the block skin pixel point number adds 1;
First face video search region division module, is connected with block skin pixel point counting module, if for judging
Skin pixel point number is more than the 14th threshold value in block, then judge that the block puts face video region of search under, and otherwise, the block puts under
It is non-for face video search region.
14th upper threshold is the pixel quantity sum of block, the half of the pixel quantity sum that optional piece of lower limit.
Fig. 6 is rough segmentation decision maker structure chart in Fig. 4;
Rough segmentation decision maker, including block colour model component value computing module and the second face video region of search division mould
Block,
Block colour model component value computing module, as colour of skin decision-point, uses for using in units of pixel average in block
Value of the average of all pixels point respective component as each color model component in block;
Second face video region of search division module, is connected, for skin with block colour model component value setting module
Color decision-point does colour of skin judgement, if the colour of skin decision-point is the colour of skin, puts face video region of search under;Otherwise, the block is drawn
Enter non-for face video search region.
In the subdivision determinating mode and rough segmentation judgment model, if colour of skin decision-point must be while meet following 6 for the colour of skin
Condition:
Require 1;Thres1<b-g<Thres2, require 2:Thres3<r-g<Thres4*Wr、
Require 3:Gup<g<Gdown, require 4:Thres5<Wr, require 5:Thres6<Co<Thres7、
Require 6:Thres8<energyUV<Thres9&&U*Thres10<V&&U*Thres11>V or Thres12<
energyUV<Thres13
Wherein, Thresjj, jj ∈ [1,13] are respectively the first to the 13rd threshold value, and the first to the 13rd threshold value is according to reality
Border situation sets itself;Based on normalizing RGB models,Obtain normalization rgb color component r, g, b;Color balance
Property parameter Wr=(r-1/3)2+(g-1/3)2;Build green component upper bound model Gup=aupr2+bupr+cup, wherein aup,bup,
cupFor model parameter, Gdown=adownr2+bdownr+cdown;Wherein adown,bdown,cdownFor model parameter;Based on model YUV
ModelObtain color energyY is brightness point
Amount, U, V represent respectively two chromatic components of YUV models;Based on YCoCg models
Co is obtained, Co is YCgCo model color component values;The colour of skin judges point methods, it is also possible to disclosed any method in the industry.
Further, Fig. 7 is the second video searching apparatus structure chart in Fig. 3, and second video searching apparatus include:
Module delimited in second region of search, if for judging bkt(i, j) is intra-frame prediction block, then decode the block, then
The block delimited for region of search;Otherwise, if spbkt(i, j)=1, then arrange sbkt(i, j)=1, that is, represent current Block- matching
Target;Otherwise, then sbk is sett(i, j)=0, that is, represent that current block mismatches target.Wherein, spbkt(i, j) represents bkt(i,
The identification parameter of reference block j).
Second size unified modules, delimit module and be connected with the second region of search, pre- for carrying out to current search region
Process, that is, unify the resolution in current search region and search target, then, with unified resolution scaling current search region
With size of the search target to as.
Second target image search module, for first, to region of search, extracts characteristics of image, then with search target
Contrasted, matched, completed the search to current search video present frame.
Wherein, the said extracted characteristics of image and search target are contrasted, and matching process can be with correspondence video search
Disclosed any method, will not be described here in field.
Second identification parameter mark module, for solving the matching result of code block by region of search, is identified to solving code block
Parameter identification.
It will be understood by those skilled in the art that realize that all or part of step in above-described embodiment method is can
With what is completed by programmed instruction related hardware, described program can be stored in a computer read/write memory medium,
Described storage medium can be ROM, RAM, disk, CD etc..
Presently preferred embodiments of the present invention is the foregoing is only, not to limit the present invention, all essences in the present invention
Any modification, equivalent and improvement made within god and principle etc., should be included within the scope of the present invention.