CN102509070A - Video-based human face area tracking method for counting people paying close attention to advertisement - Google Patents

Abstract

The invention discloses a video-based human face area tracking method for counting people paying close attention to advertisement. The method comprises the following steps of: detecting and positioning the human face area; marking the mutual relationship state of the human face area; tracking the human face in different states; counting the residence time of the successfully tracked human face area; and giving the target according to the counting result. According to the invention, the people paying close attention to the advertisement are identified through accurate tracking of the target when the targets are shielded by each other.

Description

A kind of human face region tracking of the concern advertisement demographics based on video

Technical field

The invention belongs to the image processing method technical field, relate to a kind of human face region tracking of the concern advertisement demographics based on video.

Background technology

Multimedia advertising form Along with computer technology, development of internet technology, its form of expression; And the broadcast place is more and more diversified; In order to add up the attention rate situation of advertisement, the place of in time adjusting ad content and laying advertisement need be added up the crowd density of paying close attention to advertisement.And can not have stronger advantage because of it to paying close attention to crowd's characteristics of interference based on the ad attention analytic system of video content analysis.In ad attention analytic system,, be the key that realizes its correct statistics to the correct tracking of the people's face target in the sequence of frames of video based on video content analysis.

Summary of the invention

The human face region tracking that the purpose of this invention is to provide a kind of concern advertisement demographics based on video can be blocking each other between target under the situation, and through the tracking to target, accurately the number of advertisement is paid close attention in identification.

The technical scheme that the present invention adopted is, a kind of human face region tracking of the concern advertisement demographics based on video is characterized in that concrete steps are following:

The detection of step 1, human face region and location:

Step 1.1, employing skin color detection method obtain area of skin color in the video pictures, and the area of skin color definition that obtains is people's face connected domain;

Step 1.2, front face detect:

People's face connected domain to step 1.1 obtains is carried out SHAPE DETECTION, in detecting people's face connected domain, comprises three cavities, and satisfy two cavities be arranged in parallel within another be positioned at below the time, judge that this people's face connected domain comprises a front face zone; When detecting people's face connected domain and comprise 3N-1 or 3N cavity, N is a positive integer, and N >=2, judges that this people's face connected domain comprises N human face region;

Step 1.3, front face area identification:

Front face zone to step 1.2 obtains uses minimum boundary rectangle to identify respectively, and specifically using the upper left corner coordinate of each boundary rectangle is [x ₀ ^(k), y ₀ ^(k)] and lower right corner coordinate be [x ₁ ^(k), y ₁ ^(k)], be designated { [x ₀ ^(k), y ₀ ^(k)], [x ₁ ^(k), y ₁ ^(k)], k=1,2 ..., N};

The area in front face that calculation procedure 1.2 obtains zone when the people's face that calculates is long-pending when not satisfying preset scope, is discharged this human face region target, as continuing tracing object;

Step 2, the front face testing result that step 1.2 is obtained are carried out human face region mutual relationship status indicator:

When people's face connected domain is a front face zone, judge that this human face region and other human face regions do not have adhesion each other; When people's face connected domain comprises N human face region, judge between this N human face region to have adhesion that N is a positive integer, and N >=2;

Step 3, face tracking

Step 3.1, human face region do not have the face tracking of adhesion each other

When judging that to 2 human face regions are no adhesion in present frame and former frame through step 1, with this human face region in the present frame position { [x ₀ ^(t), y ₀ ^(t)], [x ₁ ^(t), y ₁ ^(t)], and in the former frame position { [x ₀ ^(t-1), y ₀ ^(t-1)], [x ₁ ^(t-1), y ₁ ^(t-1)] compare, when satisfying x ₁ ^(t)>=x ₀ ^(t-1), and y ₁ ^(t)>=y ₀ ^(t-1)The time; Or satisfy x ₁ ^(t)>=x ₁ ^(t-1), and y ₁ ^(t)>=y ₁ ^(t-1)The time; Or satisfy x ₁ ^(t)≤x ₀ ^(t-1), and y ₁ ^(t)≤y ₁ ^(t-1)The time; Or satisfy x ₀ ^(t)>=x ₁ ^(t-1), and y ₀ ^(t)>=y ₁ ^(t-1)The time, judge this human face region is followed the tracks of successfully;

The face tracking that step 3.2, a plurality of human face region exist the state of adhesion not change

When judging that to 2 all there is adhesion in human face regions through step 1 in present frame and former frame, and people's face connected domain at this human face region place empty number and cavity position in present frame and former frame be when constant, and judgement to this face tracking successfully;

Step 3.3, the face tracking when human face region becomes adhesion

Judge that to 2 human face region is no adhesion in former frame through step 1; And when there was adhesion in present frame, each the angular coordinate analysis to present frame human face region place people's face connected domain comprises obtained N the human face region that this connected domain comprises; N is a positive integer; And N >=2 near being criterion to the maximum with overlapping area, are specified the human face region of a certain human face region for following the tracks of with movement velocity in this N human face region; And judge tracking success to this human face region, other human face region in this N human face region is defined as emerging human face region to be tracked;

Face tracking when step 3.4, human face region become no adhesion

Judge that to 2 there is adhesion in human face region in former frame through step 1, and when present frame was no adhesion, a plurality of human face regions that present frame is obtained were defined as emerging human face region to be tracked;

Step 4, the successful human face region residence time of statistical trace, the human face region target that residence time is shorter than certain threshold value is got rid of, not as continuing tracing object.

In the step 1.1, skin color detection method is:

The coloured image of input is { R _{M * n}, G _{M * n}, B _{M * n}, represent that the size of its red, green, blue three Color Channels is m * n, obtain the Face Detection result according to following formula:

Y(i，j)＝0.299·R(i，j)+0.587·G(i，j)+0.114·B(i，j)，i＝1，2，...，m，j＝1，2，...，n；

L(i，j)＝R(i，j)+G(i，j)+B(i，j)，

$r(i,j)=\frac{R(i,j)}{L(i,j)},$ $g(i,j)=\frac{G(i,j)}{L(i,j)},$ $b(i,j)=\frac{B(i,j)}{L(i,j)},$ i＝1，2，...，m，j＝1，2，...，n，

$\mathrm{Lab}(i,j)=\left\{\begin{array}{cc}\begin{array}{c}1\\ 0\end{array}& \begin{array}{c}{\mathrm{<figure-callout\; id="1"\; label="Th\; R"\; filenames="HDA0000098136780000021.png,HDA0000098136780000022.png"\; state="\{\{state\}\}">Th</figure-callout>}}_R\&le;r(i,j)\&le;{\mathrm{Th}}_{R2},{\mathrm{<figure-callout\; id="1"\; label="Th\; G"\; filenames="HDA0000098136780000021.png,HDA0000098136780000022.png"\; state="\{\{state\}\}">Th</figure-callout>}}_G\&le;g(i,j)\&le;{\mathrm{Th}}_{G2}\\ g(i,j)<r(i,j)\&le;k\&CenterDot;g(i,j),Y(i,j)\&GreaterEqual;{\mathrm{Th}}_Y\\ \mathrm{other}\end{array}\end{array}\right.,$ i＝1，2，...，m，j＝1，2，...，n

Wherein, (i, each connected domain j)=1 is an area of skin color to Lab, Th _R1, th _R2, Th _G1, Th _G2, Th _yBe the judgment threshold of the colour of skin, k for the adjustment factor, R (i, j), G (i, j) and B (i j) is video image red, green, blue tristimulus values.The span of the judgment threshold of each colour of skin is: 0.35≤Th _R1≤0.37,0.68≤Th _R2≤0.69,0.24≤Th _G1≤0.25,0.39≤Th _R2≤0.40,0.2≤Th _Y≤0.3,1.3≤k≤1.4.

The present invention and the contrast of existing tracking have the following advantages:

1, only the positive face of the spectators that face billboard is detected and follows the tracks of, can ignore the pedestrian that the side face is passed by automatically.

2, provide the default value scope that effective people's face distance of paying close attention to advertisement is filtered, as from billboard passerby too far away, though positive face, can be ignored because distance is too far away to advertisement; Or,, can not think that also it is paying close attention to ad content owing to too near from billboard to the street cleaner that billboard is cleaned.

3, for the people's face that blocks each other, can be corresponding with which people's appearance of previous frame through judging that its regional matching degree with previous frame people face is distinguished the detected people's face of present frame.

4, can adapt to voluntarily for the human face region of human face region of of short duration appearance (owing to people's face detects the situation that mistake is known or passerby causes towards reasons such as billboard flash across just) or of short duration disappearance (because mistake is known or human face region situation such as is blocked in a flash by other moving object just), and provide adjustable a plurality of parameter to change the continuous degree of tracking (time of the of short duration appearance of human face region or the time of disappearance etc.) of human face region for different statistic or track demand.

Description of drawings

Fig. 1 concerns synoptic diagram for the position that human face region among the present invention does not have adhesion;

Fig. 2 exists the position of adhesion to concern synoptic diagram for human face region among the present invention each other;

Fig. 3 concerns synoptic diagram for human face region among the present invention is no adhesion in present frame and former frame position;

Fig. 4 all exists the position of adhesion to concern synoptic diagram in present frame and former frame for human face region among the present invention;

Fig. 5 concerns synoptic diagram for human face region among the present invention becomes the position that has adhesion by no adhesion;

Fig. 6 concerns synoptic diagram for human face region among the present invention by the position that exists adhesion to become no adhesion.

Embodiment

Below in conjunction with accompanying drawing and embodiment the present invention is elaborated.

The human face region tracking of a kind of concern advertisement demographics based on video of the present invention, concrete steps are following:

The detection of step 1, human face region and location:

Step 1.1, employing skin color detection method obtain area of skin color in the video pictures, and the area of skin color definition that obtains is people's face connected domain.

Wherein, the monitor video coloured image of input is { R _{M * n}, G _{M * n}, B _{M * n}, represent that the size of its red, green, blue three Color Channels is m * n.Skin color detection method is:

Obtain the Face Detection result according to following formula:

Y(i，j)＝0.299·R(i，j)+0.587·G(i，j)+0.114·B(i，j)，i＝1，2，...，m，j＝1，2，...，n，

L(i，j)＝R(i，j)+G(i，j)+B(i，j)，

$r(i,j)=\frac{R(i,j)}{L(i,j)},$ $g(i,j)=\frac{G(i,j)}{L(i,j)},$ $b(i,j)=\frac{B(i,j)}{L(i,j)},$ i＝1，2，...，m，j＝1，2，...，n，

$\mathrm{Lab}(i,j)=\left\{\begin{array}{cc}\begin{array}{c}1\\ 0\end{array}& \begin{array}{c}{\mathrm{<figure-callout\; id="1"\; label="Th\; R"\; filenames="HDA0000098136780000021.png,HDA0000098136780000022.png"\; state="\{\{state\}\}">Th</figure-callout>}}_R\&le;r(i,j)\&le;{\mathrm{Th}}_{R2},{\mathrm{<figure-callout\; id="1"\; label="Th\; G"\; filenames="HDA0000098136780000021.png,HDA0000098136780000022.png"\; state="\{\{state\}\}">Th</figure-callout>}}_G\&le;g(i,j)\&le;{\mathrm{Th}}_{G2}\\ g(i,j)<r(i,j)\&le;k\&CenterDot;g(i,j),Y(i,j)\&GreaterEqual;{\mathrm{Th}}_Y\\ \mathrm{other}\end{array}\end{array}\right.,$ i＝1，2，...，m，j＝1，2，...，n，

Wherein, (i, each connected domain j)=1 is an area of skin color to Lab, Th _R1, Th _R2, Th _G1, Th _G2, Th _yBe the judgment threshold of the colour of skin, k is the adjustment factor.

Because present embodiment mainly is applicable to artificial lead domestic with the yellow race, so the span of each colour of skin judgment threshold is: 0.35≤Th _R1≤0.37,0.68≤Th _R2≤0.69,0.24≤Th _G1≤0.25,0.39≤Th _R2≤0.40,0.2≤Th _Y≤0.3,1.3≤k≤1.4.

Step 1.2, front face detect:

The people who pays close attention to advertisement is defined as the people who is just seeing in the face of view of advertisement, therefore, detected front face zone is estimated as the people who pays close attention to advertisement.People's face connected domain to step 1.1 obtains is carried out SHAPE DETECTION, in detecting people's face connected domain, comprises three cavities, and satisfy two cavities be arranged in parallel within another be positioned at below the time, judge that this people's face connected domain comprises a front face zone; When detecting people's face connected domain and comprise 3N-1 or 3N cavity, N is a positive integer, and N >=2, judges that this people's face connected domain comprises N human face region.

According to watching position relation attentively between target and advertisement,, confirm as two front faces and block together each other if connected domain comprises five (allowing one of them people's a eye to be blocked) or six when empty.Connected domain comprises 3N-1 (at this moment, one of them people's a eye is blocked by other people) or 3N cavity, and then showing has N the mutual adhesion of people's appearance, blocks together.

Step 1.3, front face area identification:

Front face zone to step 1.2 obtains uses minimum boundary rectangle to identify respectively, and specifically using the upper left corner coordinate of each boundary rectangle is [x ₀ ^(k), y ₀ ^(k)] and lower right corner coordinate be [x ₀ ^(k), y ₁ ^(k)], be designated { [x ₀ ^(k), y ₀ ^(k)], [x ₁ ^(k), y ₁ ^(k)], k=1,2 ..., N}.

The area in front face that calculation procedure 1.2 obtains zone when the people's face that calculates is long-pending when not satisfying preset scope, is discharged this human face region target, as continuing tracing object.Wherein, the size of preset range is set in system according to actual conditions (like effective people's face distance of definite concern advertisement).If this people's face is long-pending during greater than preset range, show that it is the street cleaner etc. of wiping advertisement; If people's face is long-pending during less than preset range, show it not within the coverage of paying close attention to advertisement, more than in two kinds of situations personage's target be not effective tracing object all, do not follow the tracks of.

Step 2, the front face testing result that step 1.2 is obtained are carried out human face region mutual relationship status indicator:

When people's face connected domain is a front face zone, judge that this human face region and other human face regions do not have adhesion each other, two human face regions as shown in Figure 1 do not have adhesion, and target A that it is represented respectively and target B do not have adhesion each other.

When people's face connected domain comprises N human face region, judge between this N human face region to have adhesion that N is a positive integer, and N >=2.There is adhesion in two human face regions as shown in Figure 2, and exist between target A that it is represented respectively and the target B and block, at this moment, these two human face regions blocking an angular coordinate of losing and can infer out each other through other three coordinates.

Step 3, face tracking

Step 3.1, human face region do not have the face tracking of adhesion each other

As shown in Figure 3, when judging that to 2 people's face connected domains are no adhesion in present frame (i.e. t frame) and former frame (i.e. t-1 frame) through step 1, with this human face region in the present frame position { [x ₀ ^(t), y ₀ ^(t)], [x ₁ ^(t), y ₀ ^(t)], and in the former frame position { [x ₀ ^(t-1), y ₀ ^(t-1)], [x ₁ ^(t-1), y ₁ ^(t-1)] compare A ^T-1The human face region position of expression former frame, A ^tX is satisfied through working as in four positions that expression present frame human face region position possibly occur therefore ₁ ^(t)>=x ₀ ^(t-1), and y ₁ ^(t)>=y ₀ ^(t-1)The time; Or satisfy x ₁ ^(t)>=x ₁ ^(t-1), and y ₁ ^(t)>=y ₁ ^(t-1)The time; Or satisfy x ₁ ^(t)≤x ₀ ^(t-1), and y ₁ ^(t)≤y ₁ ^(t-1)The time; Or satisfy x ₀ ^(t)>=x ₁ ^(t-1), and y ₀ ^(t)>=y ₁ ^(t-1)The time, judge this human face region is followed the tracks of successfully.

The face tracking that step 3.2, a plurality of human face region exist the state of adhesion not change

When judging that to 2 all there is adhesion in human face regions through step 1 in present frame and former frame, and people's face connected domain at this human face region place empty number and cavity position in present frame and former frame be when constant, and judgement to this face tracking successfully.

As shown in Figure 4, A ^T-1And A ^tRepresent the position of same human face region respectively, B at former frame (i.e. t-1 frame) and present frame (i.e. t frame) ^T-1And B ^tRepresent the position of another human face region respectively, A at former frame and present frame ^T-1And B ^T-1Stick together and belong to same people's face connected domain in former frame, this people's face connected domain does not all change in the hole of present frame number and cavity position.Therefore judge two target following successes that this people's face connected domain is comprised.

Step 3.3, the face tracking when human face region becomes adhesion

Judge that to 2 human face region is no adhesion in former frame through step 1; And when there was adhesion in present frame, each the angular coordinate analysis to present frame human face region place people's face connected domain comprises obtained N the human face region that this connected domain comprises; N is a positive integer; And N >=2 near being criterion to the maximum with overlapping area, are specified the human face region of a certain human face region for following the tracks of that meets above criterion most with movement velocity in this N human face region; And judge tracking success to this human face region, again other human face region in this N human face region is defined as emerging human face region to be tracked.

As shown in Figure 5, A ^T-1And A ^tRepresent the position of same human face region respectively, B at former frame (i.e. t-1 frame) and present frame (i.e. t frame) ^tRepresent the position of another human face region, at former frame A at present frame ^T-1There is not adhesion with other human face regions, A in present frame ^tAnd B ^tThere is adhesion and belongs to same people's face connected domain, in this people's face connected domain, be chosen in the A of former frame ^T-1The region overlapping area is big, and immediate one of movement velocity direction is former tracking target, and another then is labeled as emerging human face region to be tracked.In like manner, when blocking adhesion each other with a plurality of targets, also with speed near and overlapping area be criterion to the maximum.

Wherein, the movement velocity of target is the speed calculation of the upper left corner coordinate of this target area.

Face tracking when step 3.4, human face region become no adhesion

Judge that to 2 there is adhesion in human face region in former frame through step 1, and when present frame was no adhesion, a plurality of human face regions that present frame is obtained were defined as emerging human face region to be tracked.

As shown in Figure 6, A ^T-1And A ^tRepresent the position of a human face region A respectively, B at former frame (i.e. t-1 frame) and present frame (i.e. t frame) ^T-1And B ^tRepresent another human face region B respectively in the position of former frame and present frame, at present frame, respectively with the movement velocity direction nearer be that criterion judges that two targets separately are respectively emerging human face region to be tracked: A target and B target.

Step 4, the successful human face region residence time of statistical trace, the human face region target that residence time is shorter than certain threshold value is got rid of, not as continuing tracing object.

Claims (3)

1. human face region tracking based on the concern advertisement demographics of video is characterized in that concrete steps are following:

The detection of step 1, human face region and location:

Step 1.1, employing skin color detection method obtain area of skin color in the video pictures, and the area of skin color definition that obtains is people's face connected domain;

Step 1.2, front face detect:

People's face connected domain to step 1.1 obtains is carried out SHAPE DETECTION, in detecting people's face connected domain, comprises three cavities, and satisfy two cavities be arranged in parallel within another be positioned at below the time, judge that this people's face connected domain comprises a front face zone; When detecting people's face connected domain and comprise 3N-1 or 3N cavity, N is a positive integer, and N >=2, judges that this people's face connected domain comprises N human face region;

Step 1.3, front face area identification:

Front face zone to step 1.2 obtains uses minimum boundary rectangle to identify respectively, and specifically using the upper left corner coordinate of each boundary rectangle is [x ₀ ^(k), y ₀ ^(k)] and lower right corner coordinate be [x ₁ ^(k), y ₁ ^(k)], be designated { [x ₀ ^(k), y ₀ ^(k)], [x ₁ ^(k), y ₁ ^(k)], k=1,2 ..., N};

The area in front face that calculation procedure 1.2 obtains zone when the people's face that calculates is long-pending when not satisfying preset scope, is discharged this human face region target, as continuing tracing object;

Step 2, the front face testing result that step 1.2 is obtained are carried out human face region mutual relationship status indicator:

When people's face connected domain is a front face zone, judge that this human face region and other human face regions do not have adhesion each other; When people's face connected domain comprises N human face region, judge between this N human face region to have adhesion that N is a positive integer, and N >=2;

Step 3, face tracking

Step 3.1, human face region do not have the face tracking of adhesion each other

When judging that to 2 human face regions are no adhesion in present frame and former frame through step 1, with this human face region in the present frame position { [x ₀ ^(t), y ₀ ^(t)], [x ₁ ^(t), Y ₁ ^(t)], and in the former frame position { [x ₀ ^(t-1), y ₀ ^(t-1)], [x ₁ ^(t-1), y ₁ ^(t-1)] compare, when satisfying x ₁ ^(t)>=x ₀ ^(t-1), and y ₁ ^(t)>=y ₀ ^(t-1)The time; Or satisfy x ₁ ^(t)>=x ₁ ^(t-1), and y ₁ ^(t)>=y ₁ ^(t-1)The time; Or satisfy x ₁ ^(t)≤x ₀ ^(t-1), and y ₁ ^(t)≤y ₁ ^(t-1)The time; Or satisfy x ₀ ^(t)>=x ₁ ^(t-1), and y ₀ ^(t)>=y ₁ ^(t-1)The time, judge this human face region is followed the tracks of successfully;

The face tracking that step 3.2, a plurality of human face region exist the state of adhesion not change

When judging that to 2 all there is adhesion in human face regions through step 1 in present frame and former frame, and people's face connected domain at this human face region place empty number and cavity position in present frame and former frame be when constant, and judgement to this face tracking successfully;

Step 3.3, the face tracking when human face region becomes adhesion

Judge that to 2 human face region is no adhesion in former frame through step 1; And when there was adhesion in present frame, each the angular coordinate analysis to present frame human face region place people's face connected domain comprises obtained N the human face region that this connected domain comprises; N is a positive integer; And N >=2 near being criterion to the maximum with overlapping area, are specified the human face region of a certain human face region for following the tracks of with movement velocity in this N human face region; And judge tracking success to this human face region, other human face region in this N human face region is defined as emerging human face region to be tracked;

Face tracking when step 3.4, human face region become no adhesion

Judge that to 2 there is adhesion in human face region in former frame through step 1, and when present frame was no adhesion, a plurality of human face regions that present frame is obtained were defined as emerging human face region to be tracked;

Step 4, the successful human face region residence time of statistical trace, the human face region target that residence time is shorter than certain threshold value is got rid of, not as continuing tracing object.

2. according to the human face region tracking of the described concern advertisement demographics based on video of claim 1, it is characterized in that skin color detection method is in the step 1.1:

The coloured image of input is { R _{M * n}, G _{M * n}, B _{M * n}, represent that the size of its red, green, blue three Color Channels is m * n, obtain the Face Detection result according to following formula:

Y(i，j)＝0.299·R(i，j)+0.587·G(i，j)+0.114·B(i，j)，i＝1，2，...，m，j＝1，2，...，n；

L(i，j)＝R(i，j)+G(i，j)+B(i，j)，

$r(i,j)=\frac{R(i,j)}{L(i,j)},$ $g(i,j)=\frac{G(i,j)}{L(i,j)},$ $b(i,j)=\frac{B(i,j)}{L(i,j)},$ i＝1，2，...，m，j＝1，2，...，n，

$\mathrm{Lab}(i,j)=\left\{\begin{array}{cc}\begin{array}{c}1\\ 0\end{array}& \begin{array}{c}{\mathrm{Th}}_{R1}\&le;r(i,j)\&le;{\mathrm{Th}}_{R2},{\mathrm{Th}}_{G1}\&le;g(i,j)\&le;{\mathrm{Th}}_{G2}\\ g(i,j)<r(i,j)\&le;k\&CenterDot;g(i,j),Y(i,j)\&GreaterEqual;{\mathrm{Th}}_Y\\ \mathrm{other}\end{array}\end{array}\right.,$ i＝1，2，...，m，j＝1，2，...，n

Wherein, (i, each connected domain j)=1 is an area of skin color to Lab, Th _R1, Th _R2, Th _G1, Th _G2, Th _yBe the judgment threshold of the colour of skin, k for the adjustment factor, R (i, j), G (i, j) and B (i j) is video image red, green, blue tristimulus values.

3. according to the human face region tracking of the described concern advertisement demographics based on video of claim 2, it is characterized in that the span of the judgment threshold of said each colour of skin is: 0.35≤Th _R1≤0.37,0.68≤Th _R2≤0.69,0.24≤Th _G1≤0.25,0.39≤Th _R2≤0.40,0.2≤Th _Y≤0.3,1.3≤k≤1.4.

Images