CN108171724A - A kind of method realized vision and follow movement - Google Patents
A kind of method realized vision and follow movement Download PDFInfo
- Publication number
- CN108171724A CN108171724A CN201711476216.5A CN201711476216A CN108171724A CN 108171724 A CN108171724 A CN 108171724A CN 201711476216 A CN201711476216 A CN 201711476216A CN 108171724 A CN108171724 A CN 108171724A
- Authority
- CN
- China
- Prior art keywords
- target
- image
- value
- window
- pixel
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Withdrawn
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10016—Video; Image sequence
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20024—Filtering details
- G06T2207/20032—Median filtering
Abstract
The invention discloses a kind of methods realized vision and follow movement, include the following steps：34) it is realized using median filtering algorithm and acquisition image is handled；35) after-treatment is carried out to image using expansion algorithm；36) target image is handled three times using erosion algorithm；37) CAMSHIFT algorithms are carried out to be iterated target and carry out target following according to iteration result；38) tracking target area is subjected to R, G, B mean value computation, every 50 frame carries out Kalman's weighted filtering and calculates, and movement is followed according to the completion of result of calculation driving motor.The present invention provides it is a kind of can quickly exclusive PCR and the method that can adapt to dynamic object, allow the robot to effectively follow adaptation moving target, and exclude other and interfere and realize that vision follows.The accuracy that robot follows is improved, increases the stability that robot follows realization, reduces tracking error.
Description
Technical field
The present invention relates to robotic technology field, more particularly to a kind of method realized vision and follow movement.
Background technology
As robot automtion develops, robot identification is followed and is increasingly attracted attention, but with following target
Movement, robot are caused possible None- identified post exercise to follow target or produced to identification very much by the interference of various environment
Raw interference, therefore in the prior art, lack it is a kind of can quickly exclusive PCR and the method that can adapt to dynamic object, make machine
Device people can effectively follow adaptation moving target, and exclude other and interfere and realize that vision follows.
Therefore those skilled in the art, which are dedicated to developing, a kind of quickly exclusive PCR and can adapt to dynamic object
Method allows the robot to effectively follow adaptation moving target, and excludes other and interfere and realize that vision follows.
Invention content
In view of the drawbacks described above of the prior art, the technical problems to be solved by the invention are to provide one kind can be quick
Exclusive PCR and the method that can adapt to dynamic object allow the robot to effectively follow adaptation moving target, and exclude other
It interferes and realizes that vision follows.
To achieve the above object, the present invention provides a kind of method realized vision and follow movement, include the following steps：
34) it is realized using median filtering algorithm and acquisition image is handled；
35) after-treatment is carried out to image using expansion algorithm；
36) target image is handled three times using erosion algorithm；
37) CAMSHIFT algorithms are carried out to be iterated target and carry out target following according to iteration result；
38) tracking target area being subjected to R, G, B mean value computation, every 50 frame carries out Kalman's weighted filtering and calculates,
Movement is followed according to the completion of result of calculation driving motor.
Preferably, it in step 34), is handled according to the following steps：
341) th is set as thresholding：M is the length pixel value of entire identification window, and n is the width pixel of entire identification window
Value,
When m, n are odd number,
When m, n wherein at least one are even number, then
342) by the target window detected, each pixel value is ranked up, and establishes the histogram H of window pixel, really
Fixed wherein intermediate value mnm, brightness is write down less than or equal to mnmNumber of pixels nmn。
343) for each pixel p of most left brightnessgli, do：
H(pgli')=H (pgli)-1
nnmgli'=nnmgli-1
Wherein, H (pgli) for calculate before pgliGrey scale pixel value；
H(pgli') for calculate after pgliGrey scale pixel value；
nnmgli'It is less than or equal to m for the brightness after calculatingnmNumber of pixels；
nnmgliIt is less than or equal to m for the brightness before calculatingnmNumber of pixels；
344) for each pixel p of right column brightnessgri, do：
H(pgri')=H (pgri)+1
nnmgri'=nnmgri-1
Wherein, H (pgri) for calculate before pgriGrey scale pixel value；
H(pgri') for calculate after pgriGrey scale pixel value；
nnmgri'It is less than or equal to m for the brightness after calculatingnmNumber of pixels；
nnmgriIt is less than or equal to m for the brightness before calculatingnmNumber of pixels；
If 345) nnmgri'348)=th is then jumped to；
If 346) nnmgri'347) ＞ th are then jumped to；
If nnmgri'＜ th, according to the following formula to mnm、nnmIt is converted：
mnm'=mnm+1
nnm'=nnm+H[mnm]
With mnm'、nnm'Value repeats step 341) to 346)；
347) work as nnmgri'＞ th, according to the following formula to mnm、nnmIt is converted：
mnm”=mnm+1
nnm”=nnm+H[mnm]
With mnm”、nnm”Value repeats step 341) to 346)；
348) if the right side of window is not the right margin of image, jump to 344)；If the right side of window is not figure
349) right margin of picture then jumps to；
349) for each pixel p of most uplink brightnessgai, do：
H(pgai')=H (pgai)-1
nnmgai'=nnmgai-1
Wherein, H (pgai) for calculate before pgaiGrey scale pixel value；
H(pgai') for calculate after pgaiGrey scale pixel value；
nnmgai'It is less than or equal to m for the brightness after calculatingnmNumber of pixels；
nnmgaiIt is less than or equal to m for the brightness before calculatingnmNumber of pixels；
3410) for each pixel p of most downlink brightnessgri, do：
H(pgbi')=H (pgbi)-1
nnmgbi'=nnmgbi-1
Wherein, H (pgbi) for calculate before pgriGrey scale pixel value；
H(pgbi') for calculate after pgriGrey scale pixel value；
nnmgbi'It is less than or equal to m for the brightness after calculatingnmNumber of pixels；
nnmgbiIt is less than or equal to m for the brightness before calculatingnmNumber of pixels；
If 3411) nnmgbi'348)=th is then jumped to；
If 3412) nnmgbi'347) ＞ th are then jumped to；
If nnmgbi'＜ th, according to the following formula to mnm、nnmIt is converted：
mnm'=mnm+1
nnm'=nnm+H[mnm]
With mnm'、nnm'Value repeats step 349) to 3410)；
3413) work as nnmgbi'＞ th, according to the following formula to mnm、nnmIt is converted：
mnm”=mnm+1
nnm”=nnm+H[mnm]
With mnm”、nnm”Value repeats step 349) to 3410)；
If the bottom row of window is not the lower boundary of image, jump to 341)；
If the bottom row of window is the lower boundary of image, terminate.
Preferably, in step 35), the after-treatment of image is carried out according to the following steps：
351) with the structural element of 3X3, each pixel of bianry image, two covered with structural element with it are scanned
Value image does AND operation, if being all 0, the pixel of structural images is 0, is otherwise 1；
352) image result that all values are complete as after-treatment for 1 pixel is taken.
Preferably, in step 36), the processing three times of image is carried out according to the following steps：
361) with the structural element of 3X3, each pixel of bianry image, two covered with structural element with it are scanned
Value image does AND operation, if being all 1, the pixel of structural images is 1, is otherwise 0.
362) pixel that all values are 1 is taken as the image result handled three times.
Preferably, in step 37), the target iteration of image is carried out according to the following steps：
371) back projection is realized according to the following steps：
3711) image is transformed into HSV space from rgb space first；
3712) make histogram to H components therein, obtain color probability look-up table；
3713) by the value of pixel each in image with the probability that its color occurs to replace, obtain color probability distribution
Figure；
372) target is detected by light stream, finds out target window X；
373) CAMSHIFT iteration is realized according to the following steps,
3731) according to the following formula calculate zeroth order away from：
Wherein, rows is total line number of target window；
Clos is total columns of target window；
(xi,yi) coordinate for the point I in target image X；
Wherein, x, y are the coordinate of each pixel in each image；
3732) calculate single order away from：
3733) the barycenter C of target window X is calculated according to the following formula；
xc=M10/M00
yc=M01/M00
Wherein, (xc,yc) be barycenter C coordinate；
3734) the center M (x of moving target windowm,ym) to barycenter C (xc,yc), if displacement distance is more than preset
Fixed threshold, then repeatedly previous step calculates barycenter C', until the displacement distance between the center M of target window and barycenter C' is small
In preset fixed threshold；
When the number of loop computation reaches maximum value, stopping calculates, and feeds back the coordinate of current target window center M；
374) all frames of video are done into step 3731) to operation 3734), and by previous frame as a result, i.e. previous frame
The size of target window and center as the initial value of next frame target window, continue iteration.
Preferably, it in the step 38), is realized according to the following steps and tracking target area is subjected to R, G, B mean value meter
It calculates, every 50 frame carries out Kalman's weighted filtering and calculates：
381) every 50 frame recording follows the RGB mean values of target, and stores；
382) preceding 5 records calculated using step 381), will record distribution weight every time, obtain the number of preceding 5 records
Value collection and corresponding weight sets；
383) using the set of values and corresponding weight sets of preceding 5 records recorded in the step 382) according to karr
Graceful Filtering Formula algorithm calculates target RGB mean values next time, obtains prediction target rgb value during next 50 frame；
384) it with the prediction target rgb value according to step 383), finds with predicting the approximate target of target rgb value, determines it
To follow target.
Preferably, the method it is further comprising the steps of realize robot by the x in image coordinate, y is converted into scheming
Inconocenter is the coordinate of center of circle o, and converses first, second, third and fourth quadrant：
41) after tracking target, according to the following formula using target's center point M as target in a coordinate system effective
Point, and the coordinate system that converts；
xm0=round (xm-cols/2)
ym0=round (ym-cols/2)
Wherein (xm,ym) for conversion before image coordinate system target window central point M coordinate values；
Rows is total line number of image；
Cols is total columns of image；
(xm0,ym0) for conversion after point M' coordinate values；
And then position (the x by target point in a coordinate system 42)m0,ym0), motion planning and robot control order is conversed,
Driving motor realizes tracking target movement by robot.
The beneficial effects of the invention are as follows：The present invention provides one kind quickly exclusive PCR and can adapt to dynamic mesh
Calibration method allows the robot to effectively follow adaptation moving target, and excludes other and interfere and realize that vision follows.It improves
The accuracy that robot follows increases the stability that robot follows realization, reduces tracking error.
Description of the drawings
Fig. 1 is flow diagram of the present invention.
Fig. 2 is that the present invention combines the target follower method flow diagram that other methods are realized.
Fig. 3 is the flow diagram of target detection in Fig. 2.
Fig. 4 excludes flow diagram using HSV and profile condition exclusive PCR.
Fig. 5 is target follower method idiographic flow schematic diagram.
Fig. 6 is that target following step target action adapts to flow diagram in Fig. 2.
Specific embodiment
The invention will be further described with reference to the accompanying drawings and examples：
As shown in Figure 1, a kind of method realized vision and follow movement, includes the following steps：
34) processing to acquiring image is realized using median filtering algorithm；
35) after-treatment is carried out to image using expansion algorithm；
36) target image is handled three times using erosion algorithm；
37) CAMSHIFT algorithms are carried out to be iterated target and carry out target following according to iteration result；
38) tracking target area being subjected to R, G, B mean value computation, every 50 frame carries out Kalman's weighted filtering and calculates,
Exclude the light pollution interference of tracking target area.
Particularly, it in step 34), is carried out according to the following steps：
341) th is set as thresholding：M is the length pixel value of entire identification window, and n is the width pixel of entire identification window
Value,
When m, n are odd number,
When m, n wherein at least one are even number, then
342) by the target window detected, each pixel value is ranked up, and establishes the histogram H of window pixel, really
Fixed wherein intermediate value mnm, brightness is write down less than or equal to mnmNumber of pixels nmn。
343) for each pixel p of most left brightnessgli, do：
H(pgli')=H (pgli)-1
nnmgli'=nnmgli-1
Wherein, H (pgli) for calculate before pgliGrey scale pixel value；
H(pgli') for calculate after pgliGrey scale pixel value；
nnmgli'It is less than or equal to m for the brightness after calculatingnmNumber of pixels；
nnmgliIt is less than or equal to m for the brightness before calculatingnmNumber of pixels；
344) for each pixel p of right column brightnessgri, do：
H(pgri')=H (pgri)+1
nnmgri'=nnmgri-1
Wherein, H (pgri) for calculate before pgriGrey scale pixel value；
H(pgri') for calculate after pgriGrey scale pixel value；
nnmgri'It is less than or equal to m for the brightness after calculatingnmNumber of pixels；
nnmgriIt is less than or equal to m for the brightness before calculatingnmNumber of pixels；
If 345) nnmgri'348)=th is then jumped to；
If 346) nnmgri'347) ＞ th are then jumped to；
If nnmgri'＜ th, according to the following formula to mnm、nnmIt is converted：
mnm'=mnm+1
nnm'=nnm+H[mnm]
Wherein, H [mnm] it is intermediate value mnmGray scale；
With mnm'、nnm'Value repeats step 341) to 346)；
347) work as nnmgri'＞ th, according to the following formula to mnm、nnmIt is converted：
mnm”=mnm+1
nnm”=nnm+H[mnm]
With mnm”、nnm”Value repeats step 341) to 346)；
348) if the right side of window is not the right margin of image, jump to 344)；
If the right side of window is not the right margin of image, jump to 349)；
349) for each pixel p of most uplink brightnessgai, do：
H(pgai')=H (pgai)-1
nnmgai'=nnmgai-1
Wherein, H (pgai) for calculate before pgaiGrey scale pixel value；
H(pgai') for calculate after pgaiGrey scale pixel value；
nnmgai'It is less than or equal to m for the brightness after calculatingnmNumber of pixels；
nnmgaiIt is less than or equal to m for the brightness before calculatingnmNumber of pixels；
3410) for each pixel p of most downlink brightnessgri, do：
H(pgbi')=H (pgbi)-1
nnmgbi'=nnmgbi-1
Wherein, H (pgbi) for calculate before pgriGrey scale pixel value；
H(pgbi') for calculate after pgriGrey scale pixel value；
nnmgbi'It is less than or equal to m for the brightness after calculatingnmNumber of pixels；
nnmgbiIt is less than or equal to m for the brightness before calculatingnmNumber of pixels；
If 3411) nnmgbi'348)=th is then jumped to；
If 3412) nnmgbi'347) ＞ th are then jumped to；
If nnmgbi'＜ th, according to the following formula to mnm、nnmIt is converted：
mnm'=mnm+1
nnm'=nnm+H[mnm]
With mnm'、nnm'Value repeats step 349) to 3410)；
3413) work as nnmgbi'＞ th, according to the following formula to mnm、nnmIt is converted：
mnm”=mnm+1
nnm”=nnm+H[mnm]
With mnm”、nnm”Value repeats step 349) to 3410)；
If the bottom row of window is not the lower boundary of image, jump to 341)；
If the bottom row of window is the lower boundary of image, terminate.
After this method is handled, target can be made to obtain image more smooth.
Particularly, in step 35), the after-treatment of image is carried out according to the following steps：
351) with the structural element of 3X3, each pixel of bianry image, two covered with structural element with it are scanned
Value image does AND operation, if being all 0, the pixel of structural images is 0, is otherwise 1；
352) image result that all values are complete as after-treatment for 1 pixel is taken.
Particularly, in step 36), the processing three times of image is carried out according to the following steps：
361) with the structural element of 3X3, each pixel of bianry image, two covered with structural element with it are scanned
Value image does AND operation, if being all 1, the pixel of structural images is 1, is otherwise 0.
362) pixel that all values are 1 is taken as the image result handled three times.
By expansion and erosion algorithm, treated that image can reduce noise spot, prepares for follow-up follow.
Particularly, in step 37), the target iteration of image is carried out according to the following steps：
371) back projection is realized according to the following steps,
3711) image is transformed into HSV space from rgb space first；
3712) make histogram to H components therein, obtain color probability look-up table；
3713) by the value of pixel each in image with the probability that its color occurs to replace, obtain color probability distribution
Figure；
372) target is detected by light stream, finds out target window X；
373) CAMSHIFT iteration is realized according to the following steps,
3731) according to the following formula calculate zeroth order away from：
Wherein, rows is total line number of target window；
Clos is total columns of target window；
(xi,yi) coordinate for the point I in target image X；
Wherein, x, y are the coordinate of each pixel in each image；
3732) calculate single order away from：
3733) the barycenter C of target window X is calculated according to the following formula；
xc=M10/M00
yc=M01/M00
Wherein, (xc,yc) be barycenter C coordinate；
3734) the center M (x of moving target windowm,ym) to barycenter C (xc,yc), if displacement distance is more than preset
Fixed threshold, then repeatedly previous step calculates barycenter C', until the displacement distance between the center M of target window and barycenter C' is small
In preset fixed threshold；
When the number of loop computation reaches maximum value, stopping calculates, and feeds back the coordinate of current target window center M；
It can effectively be realized in movement by camshift and follow calculating.
374) all frames of video are done into step 3731) to operation 3734), and by previous frame as a result, i.e. previous frame
The size of target window and center as the initial value of next frame target window, continue iteration；
In the step 38), realized according to the following steps and tracking target area is subjected to R, G, B mean value computation, every 50 frame
It carries out Kalman's weighted filtering to calculate, excludes the light pollution interference of tracking target area：
381) every 50 frame recording follows the RGB mean values of target, and stores；
382) preceding 5 records calculated using step 381), will record distribution weight every time, obtain the number of preceding 5 records
Value collection and corresponding weight sets；
383) using the set of values and corresponding weight sets of preceding 5 records recorded in the step 382) according to karr
Graceful Filtering Formula algorithm calculates target RGB mean values next time, obtains prediction target rgb value during next 50 frame；Due to card
Kalman Filtering algorithm is the prior art, and details are not described herein again.
384) it with the prediction target rgb value according to step 383), finds with predicting the approximate target of target rgb value, determines it
To follow target.
384) step 381) is to, can effectively reducing influence of the environment to target jamming, such as illumination influences motion process
The realization of middle target.
In the step (4), according to the following steps realization robot by the x in image coordinate, y is converted into image
Center is the coordinate of center of circle o, and converses first, second, third and fourth quadrant：
41) after tracking target, according to the following formula using target's center point M as target in a coordinate system effective
Point, and the coordinate system that converts；
xm0=round (xm-cols/2)
ym0=round (ym-cols/2)
Wherein (xm,ym) for conversion before image coordinate system target window central point M coordinate values；
Rows is total line number of image；
Cols is total columns of image；
(xm0,ym0) for conversion after point M' coordinate values；
And then position (the x by target point in a coordinate system 42)m0,ym0), motion planning and robot control order is conversed,
Driving motor realizes tracking target movement by robot.
Coordinate system after this method converts can be more advantageous to identifying, the reality of motor function is followed conducive to robot
It is existing.
It is combined using the method that this realization vision follows movement with target detection, target identification and object feedback, Ke Yishi
Existing dynamic object follows, specific as follows：
Such as Fig. 2 to Fig. 6, a kind of method realized dynamic object and followed includes the following steps：
(1) target detection；
(2) target identification；
(3) target following；
(4) object feedback；
Wherein, the step (3) includes target jamming exclusion and target action adapts to.
Particularly, it is realized in the step (1) according to following steps：
(11) each frame image of image capture sensor is utilized；
(12) optical flow computation is carried out for front and rear two field pictures；
(13) optical flow field threshold calculations are carried out；
(14) moving target profile determines；
Wherein, in the step (12), optical flow computation is followed the steps below：
121) to each element offset amount n of image in front and rear two frameiIt is ranked up；
122) calculating of light stream field analysis test point is carried out according to the following formula：
Wherein,Image averaging light stream displacement for front and rear two frame；
niFor the element minimum offset in front and rear two field pictures；
rows:Picturedeep；
cols：Picturewide；
Wherein, in the step 222), with the following formula equivalence replacement original formula：
E=2* (n0+nrows*clos)/rows*cols
E be it is equivalent after front and rear two frame image averaging light stream displacement；
n0For the element minimum offset in front and rear two field pictures；
nrows*colsFor the element maximum offset in front and rear two field pictures；
rows:Picturedeep；
cols：Picturewide；
Wherein, in the step (13), optical flow field threshold calculations are carried out according to the following steps：
131) the offset variance of front and rear two field pictures is calculated according to the following formula；
Wherein：Image averaging light stream displacement for front and rear two frame；
Square difference of displacement for front and rear two field pictures；
132) effective light stream threshold value is determined according to the following equation：
Wherein：For effective light stream threshold value；
In the step (14), { n for meeting following equation is found outi, the region as moving target：
In the step (14), the skeleton pattern of moving target is calculated according to the following steps：
141) maximum inscribed circle of motion target area is intercepted according to the following formula：
(xi,yi) it is region { niOn vertex；
Wherein, A is motion target area area；
(xi,yi) it is region { niOn vertex ni；
(xi+1,yi+1) it is vertex niThe latter vertex ni+1Coordinate；
CxX-axis coordinate for motion target area polygon center；
CyY-axis coordinate for motion target area polygon center；
R is motion target area polygon maximum inscribed circle radius；
142) maximum inscribed circle in step (141) calculates the maximum of the circle and rectangle S is inscribed：
143) skeleton pattern of the rectangle as moving target is inscribed in the maximum of the inscribed circle in setting procedure (142).
Particularly, the step (1) is further comprising the steps of：
(15) the RGB mean values of moving region or moving region model are calculated according to the following formula, and are recorded：
WhereinFor the red component mean intensity in moving region；
RnFor nth elements red component mean intensity in motion target area；
For the green component mean intensity in moving region；
GnFor nth elements green component mean intensity in motion target area；
For the blue component mean intensity in moving region；
BnFor nth elements reddish blue component mean intensity in motion target area；
N is total number of components in moving region.
Particularly, in the step (2), target identification is realized according to following steps：
(21) according to preset profile value conditions (n1,n2,…ni…nn), to (n1,n2,…ni…nn) intersection is taken, it is right
It is not belonging to intersection { NiTarget excluded；Obtain object set { Xi}；
(22) judge the object set { X after excludingiIn element it is whether unique, in this way, it is determined that the unique objects is follow
Target；
If not, calculating all conditions value of target, and distribute each condition to weight, obtain weight (p1,
p2,…pi…pn), calculate the probability of objective contour according to the following formula according to weight：
Wherein, PxRepresent the probability of target x in step (22)；
piRepresent the weight of i-th of condition；
fx(ni) represent condition values of the target x under the conditions of n-th；Wherein, target x is closer to condition, then fx(ni) value
It is bigger.
(23) probability value for each target for meeting step (22) is calculated, and is ranked up according to size, is chosen most general
The target of rate value is used as and follows target；
Wherein, it is further comprising the steps of：
(24) it follows in motion process, according to Kalman filtering algorithm predict target location；
Wherein, future position according to the following formula：
xk=(2xk-1-xk-2)+Kgk(2xk-1-xk-2-VkxΔt)
Kgk=σk 2/σk 2+C2
Δμk-1=(1-Kgk-1)σk-1 2/2
yk=(2yk-1-yk-2)+Kgk(2yk-1-yk-2-VkyΔt)
Kgk=σk 2/σk 2+C2
Δμk-1=(1-Kgk-1)σk-1 2/2
Wherein, xkX-axis coordinate for the target prediction under K state；
xk-1For the practical x-axis coordinate of target under K-1 states；
xk-2For the practical x-axis coordinate of target under K-2 states；
ykY-axis coordinate for the target prediction under K state；
yk-1For the practical y-axis coordinate of target under K-1 states；
yk-2For the practical y-axis coordinate of target under K-2 states；
KgkFor the variation gain of k-1 to k-state, can be calculated according to above-mentioned formula；
VkxFor the movement speed in k-state target observed；
Δ t is time difference of the K-1 states to K state；
Δμk-1For the optimal deviation under k-1 states；
σkFor the predicted value Gaussian noise deviation under k-state；
(25) target and objective contour are carried out with reference to predicted position and carry out similitude judgement, according to similarity and position into
Row evaluation verifies the target to follow target：
Wherein, the verification to following target is realized according to the following formula：
Wherein, dist (Xk-1,Xk) for target X in K state value and the target similarity in K-1 state values；
X(k-1)iFor value of the target under k-1 states under the conditions of i-th；
XkiFor value of the target under k-state under the conditions of i；
Evaluation is realized according to the following formula：
PVerification=PVerification 1dist(Xk-1,Xk)+PVerification 2dist(PXIn advance, PXIt is practical to measure)
dist(PXIn advance,PXIt is practical to measure)=PXIt is practical to measure-PXIn advance/PXIt is practical to measure
Wherein, PVerificationFor the validation value of the target X under K state, it is verified if it is more than 50%；
PVerification 1For the X under K statekWith Xk-1Similarity weight；
PXIn advanceFor the X target locations under being predicted under K state；
PXIt is practical to measureX target locations for actual prediction under K state；
PVerification 2For the PX under K stateIn advanceWith PXIt is practical to measurePosition relationship weight；
Particularly, in step (3), realize that human hand detection multiple target excludes according to following steps：
31) all target circumscribed circles in detection range are reduced to external model of ellipse, if its long axis is a, short axle is
B, the identification gross area are S, and the area of target X is SX；
Determine condition one：a/b∈[1,4)；Determine condition two：Sx/S∈(1/20,1/8)；Take what two conditions met
Target { Xi, exclude other targets；
If 32) { XiIn be single-element, it is determined that the XiFor the target followed；
If there is multiple elements, then probability calculation is carried out according to the following formula：
Px=p1f(ax/bx)+p2*f(Sx/S)
Wherein, PxX targets to meet condition are to follow the probability of target；
p1It is set as the weight of condition one；
p2It is set as the weight of condition two；
f(ax/bx) it is set as ax/bxWith the relation function of one section extreme value of condition；
f(Sx/ S) it is set as Sx/ S and the relation function of two section extreme value of condition.
Relationship needs between relation function value and condition and range value are directly proportional, i.e., when closer to value range, relationship letter
Numerical value is bigger, and in the present embodiment, the linear function that relation function is set as to condition is as follows：
f(ax/bx(a of)=1/3x/bx)-1/3
f(Sx/ S)=40/3 (Sx/S)-2/3
The probability that it is target is calculated with can be realized according to the condition weight of above-mentioned function and distribution.
In other embodiment, can also by the function setup of relation function and condition for other with positive correlation
Function, to reach identical technique effect.
33) traversal calculates each destination probability value in step 32)；Sequence chooses the target of probability value maximum, really
Fixed its is to follow target.
It determines after following target, using the method for the technical program, realizes vision with amiable object feedback.
This method can be used in robot, and realization follows people or other mobile objects to move, and can make up certain machines
The defects of vision guided navigation function that device people does not have at present.
The technical program may be used different conditions and be evaluated, such as Fig. 4, with regard to being carried out using profile condition and HSV
Determining for single goal is realized after filter；In other embodiment, other conditions can also be set to be determined.
Fig. 5 is the particular flow sheet of target follower method in the present embodiment.
During work, using the hardware module of robot routine, such as camera, driving motor, control module, carry out by
Conventional programming is carried out according to the technical program to import so that robot can carry out data acquisition using camera first, control
Under the control of molding block, driving motor realization follows.
The specific mentality of designing of the technical program is：Target detection, target identification, target following, object feedback.
Target detection is followed to use light stream Farneback algorithms, which is intensive optical flow algorithm, is to image
In each point carry out optical flow computation.Optical flow algorithm is every in the image array by two frame same sizes before and after adjacent
Two values of one element are calculated, and represent amount of exercise of this pixel in x directions and y directions, the figure of this two frame respectively
The relational expression of picture is：
PrevImg (y, x)~nextImg (y+flow (y, x)) [1], x+flow (y, x) [0])
By the dynamic threshold design to intensive optical flow field, interference light stream point can be filtered out, obtains needing to follow mesh
Target optical flow field, and calculate the RGB and profile for following target.Then the target detected is carried out by color histogram
Analysis.
Target identification
If do not carried out when detection follows target in non-ideal environment or in detection process according to testing requirements,
May then will detect that it is non-follow target, in actual life, if user volume is bigger, then occur such case be ratio
More typical, then it carries out that target identification is followed just to be highly desirable after target detection, major part can be excluded in this way
External disturbance.
In target identification, often a large amount of tests, can to follow that target interfered to have other similar with human hand
Object, non-wake up follow the factors such as the hand, strong illumination, visibility of pathfinder are low etc., follow target too far, different
Under the influence of factor, there is different solutions, in recognizer is followed, mainly exclude shape and color and hand class yes
The interference problem of object and other human hands.Solution is as shown in Figure 1 and Figure 2：
Target following
Target following is to determine that unique follow take Camshift algorithms after target, which is：
Continuously Adaptive Mean-SHIFT.Its basic thought is that all picture frames of video sequence are all made
MeanShift operations, and using the result (i.e. the center of search window and window size) of previous frame as next frame
The initial value of the search window of MeanShift algorithms, so iteration continues.MeanShift is optimal for single picture searching
Iteration result, and camShift is then to handle for video sequence, and each frame picture in the sequence is all called
MeanShift finds optimal iteration result.It is handled just because of camShift for a video sequence, so as to protect
The size of window can constantly be adjusted by demonstrate,proving it, thus, when the size of target changes, the algorithm
Target area is adaptively adjusted to continue to track.
Object feedback
By the x in image coordinate, y is converted into the coordinate using picture centre as center of circle o, and converses first, second, third and fourth
Quadrant.After target is tracked by algorithm, using available point of target's center's point as target in a coordinate system, and convert
Coordinate system.Then the position by target point in a coordinate system converses motion planning and robot control order, robot is allowed to reach
Target movement is tracked, motion mode smoothly links up.
The preferred embodiment of the present invention described in detail above.It should be appreciated that those of ordinary skill in the art without
Creative work is needed according to the present invention can to conceive and makes many modifications and variations.Therefore, all technologies in the art
Personnel can be obtained under this invention's idea by logical analysis, reasoning, or a limited experiment on the basis of existing technology
Technical solution, all should be in the protection domain being defined in the patent claims.
Claims (7)
1. a kind of method realized vision and follow movement, it is characterized in that：Include the following steps：
34) it is realized using median filtering algorithm and acquisition image is handled；
35) after-treatment is carried out to image using expansion algorithm；
36) target image is handled three times using erosion algorithm；
37) CAMSHIFT algorithms are carried out to be iterated target and carry out target following according to iteration result；
38) tracking target area being subjected to R, G, B mean value computation, every 50 frame carries out Kalman's weighted filtering and calculates, according to
The completion of result of calculation driving motor follows movement.
2. the method that vision follows movement is realized as described in claim 1, it is characterized in that：In step 34), according to the following steps
It is handled：
341) th is set as thresholding：M is the length pixel value of entire identification window, and n is the width pixel value of entire identification window,
When m, n are odd number,
When m, n wherein at least one are even number, then
342) by the target window detected, each pixel value is ranked up, and establishes the histogram H of window pixel, is determined wherein
Intermediate value mnm, brightness is write down less than or equal to mnmNumber of pixels nmn。
343) for each pixel p of most left brightnessgli, do：
H(pgli')=H (pgli)-1
nnmgli'=nnmgli-1
Wherein, H (pgli) for calculate before pgliGrey scale pixel value；
H(pgli') for calculate after pgliGrey scale pixel value；
nnmgli'It is less than or equal to m for the brightness after calculatingnmNumber of pixels；
nnmgliIt is less than or equal to m for the brightness before calculatingnmNumber of pixels；
344) for each pixel p of right column brightnessgri, do：
H(pgri')=H (pgri)+1
nnmgri'=nnmgri-1
Wherein, H (pgri) for calculate before pgriGrey scale pixel value；
H(pgri') for calculate after pgriGrey scale pixel value；
nnmgri'It is less than or equal to m for the brightness after calculatingnmNumber of pixels；
nnmgriIt is less than or equal to m for the brightness before calculatingnmNumber of pixels；
If 345) nnmgri'348)=th is then jumped to；
If 346) nnmgri'347) ＞ th are then jumped to；
If nnmgri'＜ th, according to the following formula to mnm、nnmIt is converted：
mnm'=mnm+1
nnm'=nnm+H[mnm]
With mnm'、nnm'Value repeats step 341) to 346)；
347) work as nnmgri'＞ th, according to the following formula to mnm、nnmIt is converted：
mnm”=mnm+1
nnm”=nnm+H[mnm]
With mnm”、nnm”Value repeats step 341) to 346)；
348) if the right side of window is not the right margin of image, jump to 344)；
If the right side of window is not the right margin of image, jump to 349)；
349) for each pixel p of most uplink brightnessgai, do：
H(pgai')=H (pgai)-1
nnmgai'=nnmgai-1
Wherein, H (pgai) for calculate before pgaiGrey scale pixel value；
H(pgai') for calculate after pgaiGrey scale pixel value；
nnmgai'It is less than or equal to m for the brightness after calculatingnmNumber of pixels；
nnmgaiIt is less than or equal to m for the brightness before calculatingnmNumber of pixels；
3410) for each pixel p of most downlink brightnessgri, do：
H(pgbi')=H (pgbi)-1
nnmgbi'=nnmgbi-1
Wherein, H (pgbi) for calculate before pgriGrey scale pixel value；
H(pgbi') for calculate after pgriGrey scale pixel value；
nnmgbi'It is less than or equal to m for the brightness after calculatingnmNumber of pixels；
nnmgbiIt is less than or equal to m for the brightness before calculatingnmNumber of pixels；
If 3411) nnmgbi'348)=th is then jumped to；
If 3412) nnmgbi'347) ＞ th are then jumped to；
If nnmgbi'＜ th, according to the following formula to mnm、nnmIt is converted：
mnm'=mnm+1
nnm'=nnm+H[mnm]
With mnm'、nnm'Value repeats step 349) to 3410)；
3413) work as nnmgbi'＞ th, according to the following formula to mnm、nnmIt is converted：
mnm”=mnm+1
nnm”=nnm+H[mnm]
With mnm”、nnm”Value repeats step 349) to 3410)；
If the bottom row of window is not the lower boundary of image, jump to 341)；
If the bottom row of window is the lower boundary of image, terminate.
3. the method that vision follows movement is realized as described in claim 1, it is characterized in that：In step 35), according to the following steps
Carry out the after-treatment of image：
351) with the structural element of 3X3, each pixel of bianry image, the bianry image covered with structural element with it are scanned
AND operation is done, if being all 0, the pixel of structural images is 0, is otherwise 1；
352) image result that all values are complete as after-treatment for 1 pixel is taken.
4. the method that vision follows movement is realized as described in claim 1, it is characterized in that：In step 36), according to the following steps
Carry out the processing three times of image：
361) with the structural element of 3X3, each pixel of bianry image, the bianry image covered with structural element with it are scanned
AND operation is done, if being all 1, the pixel of structural images is 1, is otherwise 0.
362) pixel that all values are 1 is taken as the image result handled three times.
5. the method that vision follows movement is realized as described in claim 1, it is characterized in that：In step 37), according to the following steps
Carry out the target iteration of image：
371) back projection is realized according to the following steps：
3711) image is transformed into HSV space from rgb space first；
3712) make histogram to H components therein, obtain color probability look-up table；
3713) by the value of pixel each in image with the probability that its color occurs to replace, obtain color probability distribution figure；
372) target is detected by light stream, finds out target window X；
373) CAMSHIFT iteration is realized according to the following steps：
3731) according to the following formula calculate zeroth order away from：
Wherein,rowsTotal line number for target window；
closTotal columns for target window；
(xi,yi) coordinate for the point I in target image X；
Wherein, x, y are the coordinate of each pixel in each image；
3732) calculate single order away from：
3733) the barycenter C of target window X is calculated according to the following formula；
xc=M10/M00
yc=M01/M00
Wherein, (xc,yc) be barycenter C coordinate；
3734) the center M (x of moving target windowm,ym) to barycenter C (xc,yc), if displacement distance is more than preset fixed threshold
Value, then repeatedly previous step calculates barycenter C', until the displacement distance between the center M of target window and barycenter C' is less than preset
Fixed threshold；
When the number of loop computation reaches maximum value, stopping calculates, and feeds back the coordinate of current target window center M；
374) all frames of video are done into step 3731) to operation 3734), and by previous frame as a result, i.e. previous frame target
The size of window and center as the initial value of next frame target window, continue iteration.
6. the method that vision follows movement is realized as described in claim 1, it is characterized in that：
In the step 38), realized according to the following steps and tracking target area is subjected to R, G, B mean value computation, every 50 frame carries out
Kalman's weighted filtering calculates：
381) every 50 frame recording follows the RGB mean values of target, and stores；
382) preceding 5 records calculated using step 381), will record distribution weight every time, obtain the set of values of preceding 5 records
With corresponding weight sets；
383) using the set of values and corresponding weight sets of preceding 5 records recorded in the step 382) according to Kalman filtering
Formula algorithm calculates target RGB mean values next time, obtains prediction target rgb value during next 50 frame；
384) with the prediction target rgb value according to step 383), find with the prediction approximate target of target rgb value, determine that it is with
With target.
7. the method that vision follows movement is realized as described in claim 1, it is characterized in that：The further comprising the steps of reality of the method
Existing robot by the x in image coordinate, y is converted into the coordinate using picture centre as center of circle o, and converse first, second and third,
Four-quadrant：
41) after tracking target, according to the following formula using available points of the target's center point M as target in a coordinate system, and
Convert coordinate system；
xm0=round (xm-cols/2)
ym0=round (ym-cols/2)
Wherein (xm,ym) for conversion before image coordinate system target window central point M coordinate values；
Rows is total line number of image；
Cols is total columns of image；
(xm0,ym0) for conversion after point M' coordinate values；
And then position (the x by target point in a coordinate system 42)m0,ym0), converse motion planning and robot control order, driving electricity
Machine realizes tracking target movement by robot.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711476216.5A CN108171724A (en) | 2017-12-29 | 2017-12-29 | A kind of method realized vision and follow movement |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711476216.5A CN108171724A (en) | 2017-12-29 | 2017-12-29 | A kind of method realized vision and follow movement |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN108171724A true CN108171724A (en) | 2018-06-15 |
Family
ID=62520014
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201711476216.5A Withdrawn CN108171724A (en) | 2017-12-29 | 2017-12-29 | A kind of method realized vision and follow movement |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108171724A (en) |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR20030013092A (en) * | 2001-08-07 | 2003-02-14 | 삼성전자주식회사 | Device and method for auto tracking moving object |
| CN101022505A (en) * | 2007-03-23 | 2007-08-22 | 中国科学院光电技术研究所 | Mobile target in complex background automatic testing method and device |
-
2017
- 2017-12-29 CN CN201711476216.5A patent/CN108171724A/en not_active Withdrawn
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR20030013092A (en) * | 2001-08-07 | 2003-02-14 | 삼성전자주식회사 | Device and method for auto tracking moving object |
| CN101022505A (en) * | 2007-03-23 | 2007-08-22 | 中国科学院光电技术研究所 | Mobile target in complex background automatic testing method and device |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108133491A (en) | A kind of method for realizing dynamic target tracking | |
| CN106462976B (en) | Method for tracking shape in scene observed by asynchronous sensor | |
| Dong et al. | Infrared dim and small target detecting and tracking method inspired by human visual system | |
| US6400830B1 (en) | Technique for tracking objects through a series of images | |
| US6556708B1 (en) | Technique for classifying objects within an image | |
| US6141434A (en) | Technique for processing images | |
| US10075652B2 (en) | Image processing device for tracking a target object and image processing method, computer program product, and image processing system for same | |
| CN108961312A (en) | High-performance visual object tracking and system for embedded vision system | |
| CN107403175A (en) | Visual tracking method and Visual Tracking System under a kind of movement background | |
| CN107633226B (en) | Human body motion tracking feature processing method | |
| CN106951870B (en) | Intelligent detection and early warning method for active visual attention of significant events of surveillance video | |
| US20110026770A1 (en) | Person Following Using Histograms of Oriented Gradients | |
| CN104823444A (en) | Image stabilization techniques for video surveillance systems | |
| KR101912569B1 (en) | The object tracking system of video images | |
| US6421462B1 (en) | Technique for differencing an image | |
| CN112836640B (en) | Single-camera multi-target pedestrian tracking method | |
| WO2010030814A2 (en) | Pixel-level based micro-feature extraction | |
| CN113052873B (en) | Single-target tracking method for on-line self-supervision learning scene adaptation | |
| WO2007044037A1 (en) | Robust perceptual color identification | |
| CN111161313A (en) | Multi-target tracking method and device in video stream | |
| US6434271B1 (en) | Technique for locating objects within an image | |
| CN111161309A (en) | Searching and positioning method for vehicle-mounted video dynamic target | |
| CN111582349A (en) | Improved target tracking algorithm based on YOLOv3 and kernel correlation filtering | |
| CN111145211B (en) | Method for acquiring pixel height of head of upright pedestrian of monocular camera | |
| CN108122013A (en) | One kind, which follows, to be excluded non-to follow mesh calibration method in movement |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WW01 | Invention patent application withdrawn after publication | ||
| WW01 | Invention patent application withdrawn after publication |
Application publication date: 20180615 |