CN102999918B - Multi-target object tracking system of panorama video sequence image - Google Patents

Abstract

A multi-target object tracking system of a panorama video sequence image comprises a panorama camera shooting device and a microprocessor. The panorama camera shooting device is used for acquiring a scene in a large area, and the microprocessor is used for video analyzing and processing a panorama image shot by the panorama camera shooting device. During the processing process of moving target object extracting and simple multi-target object tracking, an MHoEI (motion history or energy images) algorithm which is low in computation complexity, simple in parameters and threshold value selection and convenient to realize in a system-on-chip is utilized. In order to effectively track multi-target objects in the scene in cases of shielding, separating and merging, the multi-target objects are sequentially tracked via motion characteristics, color characteristics and shape characteristics of the target objects according to matched results, and while tracking efficiency is improved, robustness for tracking the multi-target objects is improved.

Description

The multiple goal object-tracking systems of panorama video sequence image

Technical field

The application foundation technology of the technology such as computer vision, omni-directional visual that the present invention relates in panorama intelligent video analysis, relates in particular to the multiple goal object-tracking systems in panorama video sequence image.

Background technology

It is the analysis of destination object behavior and the prerequisite of identification that destination object is followed the tracks of, and it plays extremely important role in intelligent video analysis.It is exactly the dynamic image sequence photographed by analyzing video camera that destination object is followed the tracks of, and builds the destination object matching problem of the attributive character such as Shape-based interpolation, size, position, direction of motion, speed, color, texture between continuous print picture frame.

The destination object detection system that robustness is high needs to possess motion and detects, eliminates the graphics processing function such as shade and noise, and will realize above-mentioned functions at present also needs to adjust various parameter in advance.And existing intelligent video analysis system often all adopts fixing parameter value, make easily to cause video Detection results not good under different scene, in different application process.In addition, the parameter adjustment in destination object detection system also depends on expertise, becomes a bottleneck of extensive commercial application.Therefore, how to study a kind of efficiently, robustness high, operand is little, be convenient to hard-wired track algorithm problem urgently to be resolved hurrily in intelligent video analysis technology.

Destination object track algorithm not yet has an authoritative sorting technique, and different criteria for classifications has different classification results.Generally can be divided into following a few class: the tracking based on Model Matching, the tracking based on distorted pattern, the tracking based on Region Matching, the tracking of feature based coupling, the tracking based on kinetic characteristic and the tracking based on probability statistics.More above-mentioned track algorithms are all carry out following the tracks of according to certain attribute of object, wherein based on the tracking of kinetic characteristic be a kind of the most efficiently, closest to the tracker of human vision.

Frame difference method is based on a kind of main method in the track algorithm of kinetic characteristic, is the most straightforward procedure detecting Moving Targets in Sequent Images, have do not need background modeling, operand little, be easy to the advantages such as hardware implementing.Mainly can be divided into two frame differences and symmetric difference.

Two frame differences: the part that in difference image, gray-scale value is non-vanishing, the variation range namely caused in the picture due to target travel.But this variation range can only represent the relative position change of moving object in this two two field picture, and cannot obtain the concrete shape of moving object, and insensitive to the object slowly that moves, so there is certain limitation.

Symmetric difference: the symmetric difference technology of continuous three picture frames compensate for limitation existing for two frame difference methods.Adjacent two two field pictures carry out difference, and the significant difference between two two field pictures can detect the range of movement of target rapidly, and continuous three frame sequence images can detect the shape profile of intermediate frame motion target preferably by & operation.

Different conditions (motion or static) for the target entered in video scene takes different tracking strategy respectively, handle the application transfer problem of tracking strategy well, realizing real-time, lasting, the stable tracking of quiet target when moving when entering panoramic video scene multiple is the problem that the present invention endeavours to solve.

Summary of the invention

Follow the tracks of that operand is large, following range is limited to overcome in existing video sequence image destination object, tracking parameter be not easy to arrange, track algorithm robustness shortcoming, tracking strategy merge difficulty and be difficult to the problems such as Hardware, the invention provides a kind of efficiently, robustness high, operand is little, be convenient to hard-wired full-view video image destination object tracker.

The technical solution adopted for the present invention to solve the technical problems is:

A kind of multiple goal object-tracking systems of panorama video sequence image, comprise the omnibearing shooting device of the full-view video image capturing whole scene internal object object, described omnibearing shooting device ODVS represents, described ODVS is placed in above the middle part of monitoring scene, and described ODVS is connected with microprocessor by USB interface; Described microprocessor is connected with described PC by computer network; Described microprocessor comprises:

Video image reading unit, for being read the panoramic picture captured by ODVS by USB interface, and submits to video image expanding unit and video image storage unit by the panoramic picture of reading;

Video image expanding unit, for panoramic picture is carried out column expansion, target-object detecting unit submitted to by the panorama histogram picture after expansion;

Destination object detecting and tracking unit, for detecting the moving target object existed in panorama column unfolded image, and live destination object by rectangle circle, with a kind of Motion History or Energy Images algorithm, hereinafter referred to as MHoEI algorithm, extract and tracking target object with the motion history of destination object and energygram picture;

In described PC, carry out formalization process and the process of behavior semantization of full-view video image, described PC comprises: multiple goal subject tracking unit, block in scene for multiple goal object, be separated and combination situation time effectively follow the tracks of.

Further, in described destination object detecting and tracking unit, adopt MHoEI algorithm to extract and tracking target object the motion history of destination object and energygram picture, represent with formula (3):

$H_{\mathrm{\τ}}(x,y,t)=\left\{\begin{array}{cc}\mathrm{\τ}& \mathrm{ifD}(x,y,t)=1\\ \max (0,H_{\mathrm{\τ}}(x,y,t-1))& \mathrm{ifS}\≤\mathrm{\δ}\\ \max (0,H_{\mathrm{\τ}}(x,y,t-1)-1)& \mathrm{otherwise}\end{array}\right.---\left(3\right)$

In formula, S is the movement velocity of destination object, and τ is the duration, the binary picture sequence that D (x, y, t) is moving region, H _τthe binary picture sequence that (x, y, t-1) is non-moving areas, duration τ needs to carry out dynamic conditioning according to destination object movement velocity S.

Further again, in described destination object detecting and tracking unit, according to the feature that the colourity of pixel in shadow region is almost equal compared with background pixel colourity, the rgb color space of original image is changed into HSI color space, and then carry out frame difference method computing and just can eliminate shade, rgb color space changes into the computing method of HSI color space as shown in formula (7)

$H=\left\{\begin{array}{c}\mathrm{\&theta;}\&CenterDot;\&CenterDot;\&CenterDot;\&CenterDot;\&CenterDot;\&CenterDot;G\&GreaterEqual;B\\ 2\mathrm{\&pi;}-\mathrm{\&theta;}\&CenterDot;\&CenterDot;\&CenterDot;G<B\end{array}\right.$

$S=1-\frac{3}{(R+G+B)}\left[\min (R,G,B)\right]---\left(7\right)$

$\mathrm{\&theta;}=\arccos \left\{\frac{[(R-G)+(R-B)]/2}{{[{(R-G)}^2+(R-B)(G-B)]}^{1/2}}\right\}$

In formula, R is the red component in rgb color space, and G is the green component in rgb color space, and B is the blue component in rgb color space; H is the tone in HSI color space, represents by angle, reflects color closest to which type of spectral wavelength; S is the depth degree of the saturation degree in HSI color space, characterizing color; Tone H and saturation degree S claims colourity altogether;

For the destination object of the distant place of distance 0DVS, on H component and S component, carry out frame difference method process respectively, its computing formula as shown in (8),

${\mathrm{IP}}_{L,H}\mathrm{Image}(i,j)=\left\{\begin{array}{c}1\&CenterDot;\&CenterDot;\&CenterDot;\&CenterDot;\&CenterDot;\&CenterDot;\mathrm{if}|{\mathrm{Pix}}_{H,t}(i,j)-{\mathrm{Pix}}_{H,t-3}(i,j)|>\mathrm{Threshold}1\\ 0\&CenterDot;\&CenterDot;\&CenterDot;\&CenterDot;\&CenterDot;\&CenterDot;\mathrm{else}\end{array}\right.---\left(8\right)$

${\mathrm{IP}}_{L,S}\mathrm{Image}(i,j)=\left\{\begin{array}{c}1\&CenterDot;\&CenterDot;\&CenterDot;\&CenterDot;\&CenterDot;\&CenterDot;\mathrm{if}|{\mathrm{Pix}}_{S,t}(i,j)-{\mathrm{Pix}}_{S,t-3}(i,j)|>\mathrm{Threshold}1\\ 0\&CenterDot;\&CenterDot;\&CenterDot;\&CenterDot;\&CenterDot;\&CenterDot;\mathrm{else}\end{array}\right.$

In formula, IP _{l, H}image (i, j) for the coordinate of the top H color component in current input panoramic image frame be the testing result that (i, j) puts, IP _{l, S}image (i, j) is the testing result that (i, j) puts for the coordinate of the top S color component in current input panoramic image frame, represents by binary map, and 1 represents foreground moving object, and 0 represents background, Pix _{h, t}(i, j) and Pix _{h, t-3}(i, j) represents that top H color component coordinate in t and t-3 time chart picture frame of full-view video image is the pixel value that (i, j) puts respectively, Pi _{x, t}(i, j) and Pix _{s, t-3}(i, j) represents that top S color component coordinate in t and t-3 time chart picture frame of full-view video image is the pixel value that (i, j) puts respectively, and Threshold1 is corresponding judgment threshold, and value is 45 here;

For destination object at a distance in distance ODVS, on H component and S component, carry out frame difference method process respectively, its computing formula as shown in (9),

${\mathrm{IP}}_{M,H}\mathrm{Image}(i,j)=\left\{\begin{array}{c}1\&CenterDot;\&CenterDot;\&CenterDot;\&CenterDot;\&CenterDot;\&CenterDot;\mathrm{if}|{\mathrm{Pix}}_{H,t}(i,j)-{\mathrm{Pix}}_{H,t-2}(i,j)|>\mathrm{Threshold}2\\ 0\&CenterDot;\&CenterDot;\&CenterDot;\&CenterDot;\&CenterDot;\&CenterDot;\mathrm{else}\end{array}\right.---\left(9\right)$

${\mathrm{IP}}_{M,S}\mathrm{Image}(i,j)=\left\{\begin{array}{c}1\&CenterDot;\&CenterDot;\&CenterDot;\&CenterDot;\&CenterDot;\&CenterDot;\mathrm{if}|{\mathrm{Pix}}_{S,t}(i,j)-{\mathrm{Pix}}_{S,t-2}(i,j)|>\mathrm{Threshold}2\\ 0\&CenterDot;\&CenterDot;\&CenterDot;\&CenterDot;\&CenterDot;\&CenterDot;\mathrm{else}\end{array}\right.$

In formula, IP _{m, H}im age (i, j) for the coordinate of the middle part H color component in current input panoramic image frame be the testing result that (i, j) puts, IP _{m, S}im age (i, j) is the testing result that (i, j) puts for the coordinate of the middle part S color component in current input panoramic image frame, represents by binary map, and 1 represents foreground moving object, and 0 represents background, Pix _{h, t}(i, j) and Pix _{h, t-2}(i, j) represents that middle part H color component coordinate in t and t-2 time chart picture frame of full-view video image is the pixel value that (i, j) puts respectively, Pix _{s, t}(i, j) and Pix _{s, t-2}(i, j) represents that middle part S color component coordinate in t and t-2 time chart picture frame of full-view video image is the pixel value that (i, j) puts respectively, and Threshold2 is corresponding judgment threshold, and value is 45 here;

For the destination object nearby of distance ODVS, on H component and S component, carry out frame difference method process respectively, its computing formula as shown in (10),

${\mathrm{IP}}_{N,H}\mathrm{Image}(i,j)=\left\{\begin{array}{c}1\&CenterDot;\&CenterDot;\&CenterDot;\&CenterDot;\&CenterDot;\&CenterDot;\mathrm{if}|{\mathrm{Pix}}_{H,t}(i,j)-{\mathrm{Pix}}_{H,t-1}(i,j)|>\mathrm{Threshold}3\\ 0\&CenterDot;\&CenterDot;\&CenterDot;\&CenterDot;\&CenterDot;\&CenterDot;\mathrm{else}\end{array}\right.---\left(10\right)$

${\mathrm{IP}}_{N,S}\mathrm{Image}(i,j)=\left\{\begin{array}{c}1\&CenterDot;\&CenterDot;\&CenterDot;\&CenterDot;\&CenterDot;\&CenterDot;\mathrm{if}|{\mathrm{Pix}}_{S,t}(i,j)-{\mathrm{Pix}}_{S,t-1}(i,j)|>\mathrm{Threshold}3\\ 0\&CenterDot;\&CenterDot;\&CenterDot;\&CenterDot;\&CenterDot;\&CenterDot;\mathrm{else}\end{array}\right.$

In formula, IP _{n, H}im age (i, j) for the coordinate of the bottom H color component in current input panoramic image frame be the testing result that (i, j) puts, IP _{n, S}im age (i, j) is the testing result that (i, j) puts for the coordinate of the bottom S color component in current input panoramic image frame, represents by binary map, and 1 represents foreground moving object, and 0 represents background, Pix _{h, t}(i, j) and Pix _{h, t-1}(i, j) represents that middle part H color component coordinate in t and t-1 time chart picture frame of full-view video image is the pixel value that (i, j) puts respectively, Pix _{s, t}(i, j) and Pix _{s, t-1}(i, j) represents that middle part S color component coordinate in t and t-1 time chart picture frame of full-view video image is the pixel value that (i, j) puts respectively, and Threshold3 is corresponding judgment threshold, and value is 45 here;

Finally, process is arranged for the destination object split after segmentation; On the one hand, because colourity is formed primarily of tone H and saturation degree S two part, need here to carry out or calculation process; On the other hand, in P4, P5 and P6 processing procedure, view picture panoramic picture is divided into three, upper, middle and lower part, also needs here to carry out or calculation process; Obtain the segmentation image of the moving target object on view picture panoramic picture through such process, disposal route as shown in formula (11),

D(x，y，t)＝IP _L，HIm age(i，j)∨IP _L，SIm age(i，j)∨IP _M，HIm age(i，j)∨IP _M，SIm age(i，j)∨IP _N，HIm age(i，j)∨IP _N，SIm age(i，j) (11)

In formula, D (x, y, t) for coordinate in current input panoramic image frame be the testing result that (i, j) puts, IP _{n, H}image (i, j) for the coordinate of the bottom H color component in current input panoramic image frame be the testing result that (i, j) puts, IP _{n, S}im age (i, j) for the coordinate of the bottom S color component in current input panoramic image frame be the testing result that (i, j) puts, IP _{m, H}im age (i, j) for the coordinate of the middle part H color component in current input panoramic image frame be the testing result that (i, j) puts, IP _{m, S}im age (i, j) for the coordinate of the middle part S color component in current input panoramic image frame be the testing result that (i, j) puts, IP _{l, H}im age (i, j) for the coordinate of the top H color component in current input panoramic image frame be the testing result that (i, j) puts, IP _{l, S}image (i, j) for the coordinate of the top S color component in current input panoramic image frame be the testing result that (i, j) puts.

In described video image expanding unit, according to the centre coordinate of the panoramic picture calculated in initialization process and the inside and outside circle radius of image, by the initial point O of the centre coordinate of panoramic picture setting plane coordinate system ^*(0,0), X ^*axle, Y ^*the internal diameter of axle, panoramic picture is r, external diameter is R, and with radius of a circle in the middle of r1=(r+R)/2 setting, position angle is β=tan ^-1(y ^*/ x ^*); Panorama column unfolded image is with true origin O ^*(0,0), X ^*axle, Y ^*axle is plane coordinate system, is r and X by the internal diameter in panoramic picture ^*the intersection point (r, 0) of axle is as true origin O ^*(0,0), counterclockwise launches with azimuthal angle beta; Set up any point pixel coordinates P in panorama column unfolded image ^*(x ^*, y ^*) with panoramic picture in pixel coordinates Q ^*(x ^*, y ^*) corresponding relation, its calculating formula is:

x ^*＝y ^*/(tan(360x ^**/π(R+r))) (4)

y ^*＝(y ^**+r)coSβ (5)

In above formula, x ^*, y ^*for the pixel coordinates value of panorama column unfolded image, x ^*, y ^*for the pixel coordinates value of panoramic picture, R is the external diameter of circular panoramic picture, and r is the internal diameter of circular panoramic picture, and β is the position angle of circular panoramic picture coordinate.

In described destination object detecting and tracking unit, when adopting the extraction of MHoEI algorithm and tracking target object, obtaining the ROI of each moving target object, calculating i-th ROI _icenter-of-mass coordinate ROI _{i, m}(x, y, t), obtains i-th ROI in then utilizing one to circulate _icenter-of-mass coordinate ROI _{i, m}(x, y, t-1) calculates the speed of i-th moving target object, computing method as shown in formula (15),

$S_i\left(t\right)=\frac{|{\mathrm{ROI}}_{i,m}(x,y,t)-{\mathrm{ROI}}_{i,m}(x,y,t-1)|}{\mathrm{\&Delta;t}}---\left(15\right)$

In formula, ROI _{i, m}(x, y, t) is i-th ROI in frame under process _icenter-of-mass coordinate, ROI _{i, m}(x, y, t-1) is i-th ROI in a upper processed frame _icenter-of-mass coordinate, Δ t was two frame period times, S _it () is i-th ROI in frame under process _imovement velocity.

In described destination object detecting and tracking unit, the destination object movement velocity S calculated by formula (15) _it () is as calculating prolongeding time τ _{i, M}foundation; Computing method as shown in formula (12),

τ _i，M＝k/S _i(t) (12)

In formula, τ _{i, M}be the duration of i-th destination object, S _it () is the translational speed of i-th destination object, k is a certain constant;

For the far and near different destination object of distance ODVS, τ _{i, M}value also needs suitable adjustment, for the destination object of same movement velocity, the speed that the near destination object of distance ODVS reflects on panoramic expansion figure can be hurry up, the speed that the destination object that then distance ODVS is far away reflects on panoramic expansion figure can be slow, and what formula (12) was tried to achieve is from the τ in ODVS moderate distance situation _{i, M}value, carried out normalized to the duration here, specific practice is that the duration is nearby set to H _τ(x, y, t)=τ _{i, M}-α, the duration of middle distant place is set to H _τ(x, y, t)=τ _{i, M}, the duration is at a distance set to H _τ(x, y, t)=τ _{i, M}+ α; Wherein α=2 ~ 4.

In described destination object detecting and tracking unit, each circulation obtains i ROI area-of-interest, the center-of-mass coordinate value in region and the sizes values of regional frame; The extraction of these destination objects and tracking data also call when being supplied to video sequence image process on the middle and senior level with full-view video image in the mode of software interface.

In described multiple goal subject tracking unit, each destination object entering scene is created automatically to the object of a software, the describing method of object is as follows:

In described multiple goal subject tracking unit, when adopting color histogram to occur division situation as color characteristic model and multiple goal object, first mates foundation, and specific practice is before multiple object merging, is kept in object by the chrominance information of each object; The color histogram of the relevant target object be stored in object is read respectively when division situation appears in multiple goal object data; Color histogram is done to the ROI of relevant target object during division situation simultaneously using the tolerance of Bhattacharyya distance as two color histogram similaritys, the computing method of discrete Bart Charlie subbreed number as shown in formula (16),

$\mathrm{\&rho;}[P_i,Q_j]=\underset{\mathrm{\&mu;}=1}{\overset{m}{\mathrm{\&Sigma;}}}\sqrt{p_i^{\left(\mathrm{\&mu;}\right)}{q}_j^{\left(\mathrm{\&mu;}\right)}}---\left(16\right)$

In formula, ρ [P _i, Q _j] ∈ [0,1] is Bart Charlie subbreed number, Bhattacharyya distance formula (17) calculates,

$d=\min \left\{d_{i,j}\right\}=\min \left\{\sqrt{1-\mathrm{\&rho;}[P_i,Q_j]}\right\}---\left(17\right)$

In formula, d is Bhattacharyya distance, if the threshold value Threshold4 that this value is less than a certain regulation just represents that the match is successful, continues by the second coupling according to mating for not having the destination object that the match is successful.

Described multiple goal subject tracking unit, when adopting Hu bending moment does not occur division situation as destination object shape facility and multiple goal object, second mates foundation, specific practice is before multiple object merging, is kept in object by 7 vector information of the Hu invariant moment features of each object; The Hu invariant moment features vector information be stored in the relevant target object of object is read respectively when division situation appears in multiple goal object; The vector operation of Hu invariant moment features is done to relevant target object during division situation simultaneously, using the tolerance of Euclidean distance as two Hu invariant moment features similaritys, the computing method of Hu not bending moment 7 proper vectors as shown in formula (18),

φ ₁＝η ₂₀+η ₀₂

${\mathrm{\&phi;}}_2={({\mathrm{\&eta;}}_{20}-{\mathrm{\&eta;}}_{02})}^2+{4\mathrm{\&eta;}}_{11}^2$

φ ₃＝(θ ₃₀-3η ₁₂) ²+(3η ₂₁-η ₀₃) ²

φ ₄＝(η ₃₀+η ₁₂) ²+(η ₂₁+η ₀₃) ²

φ ₅＝(η ₃₀-3η ₁₂)(η ₃₀+η ₁₂)[(η ₃₀+η ₁₂) ²-3(η ₂₁+η ₀₃) ²]+ (18)

(3η ₂₁-η ₀₃)(η ₂₁-η ₀₃)[3(η ₃₀+η ₁₂) ²-(η ₂₁+η ₀₃) ²]

φ ₆＝(η ₂₀-η ₀₂)[(η ₃₀+η ₁₂) ²-3(η ₂₁+η ₀₃) ²]+4η ₁₁(η ₃₀+η ₁₂)(η ₂₁+η ₀₃)

φ ₇＝(3η ₂₁-η ₀₃)(η ₃₀+η ₁₂)[(η ₃₀+η ₁₂) ²-3(η ₂₁+η ₀₃) ²]-

(η ₃₀-3η ₁₂)(η ₂₁+η ₀₃)[3(η ₃₀+η ₁₂) ²-(η ₂₁+η ₀₃) ²]

In formula, η _pqfor normalization centre distance, computing method are provided by formula (19),

${\mathrm{\&eta;}}_{\mathrm{pq}}={\mathrm{\&mu;}}_{\mathrm{pq}}/{\mathrm{\&mu;}}_{\mathrm{pq}}^r\&CenterDot;\&CenterDot;\&CenterDot;\&CenterDot;\&CenterDot;\&CenterDot;r=(p+q)/2\&CenterDot;\&CenterDot;\&CenterDot;p+q=\mathrm{2,3},\&CenterDot;\&CenterDot;\&CenterDot;---\left(19\right)$

In formula, μ _pqcentered by distance, computing method are provided by formula (20),

${\mathrm{\&mu;}}_{\mathrm{pq}}=\underset{x=1}{\overset{M}{\mathrm{\&Sigma;}}}\underset{y=1}{\overset{N}{\mathrm{\&Sigma;}}}{(x-\stackrel{\&OverBar;}{x})}^p{(y-\stackrel{\&OverBar;}{y})}^{q}f(x,y)---\left(20\right)$

In formula, the binary map that f (x, y) is destination object, with for the barycentric coordinates of destination object, computing method are provided by formula (21),

$\stackrel{\&OverBar;}{x}=m_{10}/m_{11},\stackrel{\&OverBar;}{y}=m_{01}/m_{00}$ $\left(21\right)$

$m_{\mathrm{pq}}=\underset{x=1}{\overset{M}{\mathrm{\&Sigma;}}}\underset{y=1}{\overset{N}{\mathrm{\&Sigma;}}}{x}^p{y}^{q}f(x,y)$

In formula, the binary map that f (x, y) is destination object; Can pass through for each destination object 7 vectors that formula (18) ~ (21) calculate its Hu not bending moment, then use formula (22) Euclidean distance go to judge both similarity,

$d_{\mathrm{ms}}=\underset{i=1}{\overset{7}{\mathrm{\&Sigma;}}}{({\mathrm{\&phi;}}_{\mathrm{mi}}-{\mathrm{\&phi;}}_{\mathrm{si}})}^2\&CenterDot;\&CenterDot;\&CenterDot;s\&Element;S---\left(22\right)$

In formula, d _msfor being stored in the Hu invariant moment features vector value φ in object _siwith by the Hu invariant moment features vector value φ of destination object mated _mieuclidean distance value, S be multiple destination object merge time destination object number, if meet d _mj=min{d _msj, s ∈ S, so a jth object is just judged to be by the destination object mated.

Beneficial effect of the present invention is mainly manifested in: in the process extracting moving target object, 1, automatically eliminate the shade that moving target object produces; 2, algorithm simple, utilize recursive calculation method, only up-to-date information is needed to store, makes to calculate fast and efficient; 3, the common problem in the bottom visual processes of intelligent video analysis extracts foreground object effectively easily and effectively follows the tracks of foreground object, and wish to realize by SOC (system on a chip); 4, adopt software object describing mode that multiple target tracking real-time and robustness are greatly improved.

Accompanying drawing explanation

Fig. 1 is the processing flow chart of the multiple goal object-tracking systems of panorama video sequence image;

Fig. 2 is the hardware structure diagram of the multiple goal object-tracking systems of panorama video sequence image;

Fig. 3 is the structural drawing of a kind of ODVS;

Fig. 4 is the imaging panorama schematic diagram of ODVS;

Fig. 5 is a kind of ODVS imaging schematic diagram;

Fig. 6 is that the panoramic perspective of the panoramic vision imaging of single view launches key diagram;

Fig. 7 be full-view perspective far away, in, foreground object modeling in nearly piecemeal and block illustrates schematic diagram;

Fig. 8 is the multiple goal object extraction algorithm flow chart that panorama column launches video image;

Fig. 9 is the multiple goal object tracking algorithm process flow diagram that panorama column launches video image;

Figure 10 is that a kind of simple multiple goal Object tracking illustrates schematic diagram;

Figure 11 is that a kind of multiple goal Object tracking of complexity illustrates schematic diagram;

Figure 12 is a kind of foreground target object extraction process process flow diagram.

Embodiment

Below in conjunction with accompanying drawing, the invention will be further described.

With reference to Fig. 1 ~ 12, a kind of multiple goal object-tracking systems of panorama video sequence image, comprise the omnibearing shooting device for obtaining scene on a large scale, for carrying out the microprocessor of underlying concept process and the PC for carrying out formalization process and the process of behavior semantization to underlying concept result to the panoramic picture captured by omnibearing shooting device, as shown in Figure 1, its system hardware is formed as shown in Figure 2 panoramic vision treatment scheme; Described omnibearing shooting device, as shown in Figure 3, referred to as ODVS, above the middle part being placed in monitoring scene, makes ODVS can capture the full-view video image of whole scene internal object object, a captured panoramic picture circle as shown in Figure 4; Described omnibearing shooting device is connected with described microprocessor by USB interface; Described microprocessor is connected with described PC by computer network; Described microprocessor comprises hardware and software two parts, and hardware components adopts generally commercially available davinci platform; Application software part in davinci platform comprises: video image reading unit, for being read the panoramic picture captured by ODVS by USB interface, and the panoramic picture of reading is submitted to video image expanding unit and video image storage unit; Video image expanding unit is used for panoramic picture to carry out column expansion, and target-object detecting unit submitted to by the panorama histogram picture after expansion; Destination object detecting and tracking unit is for detecting the moving target object existed in panorama column unfolded image, and live destination object by rectangle circle, with a kind of Motion History orEnergy Images algorithm, hereinafter referred to as MHoEI algorithm, extract and tracking target object with the motion history of destination object and energygram picture; Described PC mainly carries out formalization process and the process of behavior semantization of full-view video image, mainly in the present invention realize complicated multiple goal Object tracking, multiple goal subject tracking unit be used for multiple goal object block in scene, be separated and the situation such as merging time effectively follow the tracks of;

Fig. 5 is the structural drawing of ODVS, ODVS comprises hyperboloid minute surface 2, upper cover 1, transparent housing 3, lower fixed seat 4, image unit holder 5, image unit 6, linkage unit 7, upper cover 8, described hyperboloid minute surface 2 is fixed on described upper cover 1, described lower fixed seat 4 and transparent housing 3 link into an integrated entity by described linkage unit 7, together with described transparent housing 3 is fixed by screws in described upper cover 1 and described upper cover 8, described image unit 6 is screwed on described image unit holder 5, described image unit 6 holder 5 is screwed on described lower fixed seat 4, the output port of described image unit 6 is USB interface,

The main treatment scheme realized as shown in Figure 8 in davinci platform, the flow chart below according to Fig. 8 illustrates and from panorama video sequence image, extracts destination object and the processing procedure of following the tracks of destination object;

P1 is that microprocessor reads full-view video image by USB interface from ODVS, and the panoramic picture of reading is submitted to P2;

P2 carries out column to full-view video image and launches process, according to the centre coordinate of the panoramic picture calculated in initialization process and the inside and outside circle radius of image, as shown in Figure 6, by the initial point O of the centre coordinate of panoramic picture setting plane coordinate system ^*(0,0), X ^*axle, Y ^*the internal diameter of axle, panoramic picture is r, external diameter is R, uses r _iradius of a circle in the middle of the setting of=(r+R)/2, position angle is β=tan ^-1(y ^*/ x ^*); Panorama column unfolded image is with true origin O ^*(0,0), X ^*axle, Y ^*axle is plane coordinate system, is r and X by the internal diameter in panoramic picture ^*the intersection point (r, 0) of axle is as true origin O ^*(0,0), counterclockwise launches with azimuthal angle beta; Set up any point pixel coordinates P in panorama column unfolded image ^*(x ^*, y ^*) with panoramic picture in pixel coordinates Q ^*(x ^*, y ^*) corresponding relation, its calculating formula is:

x ^*＝y ^*/(tan(360x ^**/π(R+r))) (4)

y ^*＝(y ^**+r)coSβ (5)

In above formula, x ^*, y ^*for the pixel coordinates value of panorama column unfolded image, x ^*, y ^*for the pixel coordinates value of panoramic picture, R is the external diameter of circular panoramic picture, and r is the internal diameter of circular panoramic picture, and β is the position angle of circular panoramic picture coordinate;

Because the scope of above-mentioned panorama column unfolded image is 0 ~ 360 °, the situation being judged as two destination objects is there will be when same tracking target object is in 0 ° or 360 ° of edges, for this reason, in the present invention, the scope of panorama column unfolded image is set to 0 ~ 380 °, namely the overlapping region of about 20 ° is had, as shown in Figure 7;

According to the image-forming principle of ODVS, as shown in Figure 5, destination object is in from the distant regional imaging of ODVS on the top of panorama column unfolded image, destination object is in from the middle part of the regional imaging away from ODVS moderate distance at panorama column unfolded image, and destination object is in regional imaging from ODVS close together in the bottom of panorama column unfolded image; For this reason, in the present invention, panorama column unfolded image scope is in vertical direction set to three regions, as shown in Figure 7, is respectively remote region, moderate distance region and nearby region;

P3 carries out the conversion process of gray-scale value conversion and HSI color space to panoramic expansion image, the object of gray-scale value conversion is to obtain moving target subject area when asking frame-to-frame differences, and the object of the conversion process of HSI color space is the shade in order to eliminate moving target object when asking frame-to-frame differences;

In order to do the shade eliminating moving target object in frame difference method calculating process; Frame difference method is a kind of based on seasonal effect in time series directly simple moving target detecting method, frame difference method computing method such as formula shown in (6),

$\mathrm{IP}\mathrm{Image}(i,j)=\left\{\begin{array}{c}1\&CenterDot;\&CenterDot;\&CenterDot;\&CenterDot;\&CenterDot;\&CenterDot;\mathrm{if}|{\mathrm{Pix}}_t(i,j)-{\mathrm{Pix}}_{t-n}(i,j)|>\mathrm{Threshold}\\ 0\&CenterDot;\&CenterDot;\&CenterDot;\&CenterDot;\&CenterDot;\&CenterDot;\mathrm{else}\end{array}\right.---\left(6\right)$

In formula, IP Im age (i, j) is the testing result that (i, j) puts for coordinate in current input image frame, represents by binary map, and 1 represents foreground moving object, and 0 represents background, Pix _t(i, j) and Pix _t-n(i, j) represents that video image coordinate in t and t-n time chart picture frame is the pixel value that (i, j) puts respectively, and Threshold is corresponding judgment threshold; To think when the pixel value difference of gained is greater than the threshold value that this sets that in video image coordinate in t frame belongs to a pixel in foreground moving set of regions as the pixel of (i, j), otherwise, be judged as the pixel that scene is concentrated;

Rgb color space changes into the computing method of HSI color space as shown in formula (7),

$H=\left\{\begin{array}{c}\mathrm{\&theta;}\&CenterDot;\&CenterDot;\&CenterDot;\&CenterDot;\&CenterDot;\&CenterDot;G\&GreaterEqual;B\\ 2\mathrm{\&pi;}-\mathrm{\&theta;}\&CenterDot;\&CenterDot;\&CenterDot;G<B\end{array}\right.$

$S=1-\frac{3}{(R+G+B)}\left[\min (R,G,B)\right]---\left(7\right)$

$\mathrm{\&theta;}=\arccos \left\{\frac{[(R-G)+(R-B)]/2}{{[{(R-G)}^2+(R-B)(G-B)]}^{1/2}}\right\}$

In formula, R is the red component in rgb color space, and G is the green component in rgb color space, and B is the blue component in rgb color space; H is the tone in HSI color space, represents by angle, reflects color closest to which type of spectral wavelength; S is the depth degree of the saturation degree in HSI color space, characterizing color; Tone H and saturation degree S claims colourity altogether;

In P3 process, the present invention carries out color space change respectively to panoramic expansion figure, obtains the panoramic expansion figure of H component and the panoramic expansion figure of S component, and the result after process is submitted to P4, P5 and P6 respectively according to the distance of distance ODVS and carries out frame difference method process;

P4 process be the destination object of the distant place of distance ODVS, be divided into H component and S component, its computing formula as shown in (8),

${\mathrm{IP}}_{L,H}\mathrm{Image}(i,j)=\left\{\begin{array}{c}1\&CenterDot;\&CenterDot;\&CenterDot;\&CenterDot;\&CenterDot;\&CenterDot;\mathrm{if}|{\mathrm{Pix}}_{H,t}(i,j)-{\mathrm{Pix}}_{H,t-3}(i,j)|>\mathrm{Threshold}1\\ 0\&CenterDot;\&CenterDot;\&CenterDot;\&CenterDot;\&CenterDot;\&CenterDot;\mathrm{else}\end{array}\right.---\left(8\right)$

${\mathrm{IP}}_{L,S}\mathrm{Image}(i,j)=\left\{\begin{array}{c}1\&CenterDot;\&CenterDot;\&CenterDot;\&CenterDot;\&CenterDot;\&CenterDot;\mathrm{if}|{\mathrm{Pix}}_{S,t}(i,j)-{\mathrm{Pix}}_{S,t-3}(i,j)|>\mathrm{Threshold}1\\ 0\&CenterDot;\&CenterDot;\&CenterDot;\&CenterDot;\&CenterDot;\&CenterDot;\mathrm{else}\end{array}\right.$

In formula, IP _{l, H}im age (i, j) for the coordinate of the top H color component in current input panoramic image frame be the testing result that (i, j) puts, IP _{l, S}im age (i, j) is the testing result that (i, j) puts for the coordinate of the top S color component in current input panoramic image frame, represents by binary map, and 1 represents foreground moving object, and 0 represents background, Pix _{h, t}(i, j) and Pix _{h, t-3}(i, j) represents that top H color component coordinate in t and t-3 time chart picture frame of full-view video image is the pixel value that (i, j) puts respectively, Pix _{s, t}(i, j) and Pix _{s, t-3}(i, j) represents that top S color component coordinate in t and t-3 time chart picture frame of full-view video image is the pixel value that (i, j) puts respectively, and Threshold1 is corresponding judgment threshold, and value is 45 here;

P5 process be the destination object of the middle distant place of distance ODVS, be divided into H component and S component, its computing formula as shown in (9),

${\mathrm{IP}}_{M,H}\mathrm{Image}(i,j)=\left\{\begin{array}{c}1\&CenterDot;\&CenterDot;\&CenterDot;\&CenterDot;\&CenterDot;\&CenterDot;\mathrm{if}|{\mathrm{Pix}}_{H,t}(i,j)-{\mathrm{Pix}}_{H,t-2}(i,j)|>\mathrm{Threshold}2\\ 0\&CenterDot;\&CenterDot;\&CenterDot;\&CenterDot;\&CenterDot;\&CenterDot;\mathrm{else}\end{array}\right.---\left(9\right)$

${\mathrm{IP}}_{M,S}\mathrm{Image}(i,j)=\left\{\begin{array}{c}1\&CenterDot;\&CenterDot;\&CenterDot;\&CenterDot;\&CenterDot;\&CenterDot;\mathrm{if}|{\mathrm{Pix}}_{S,t}(i,j)-{\mathrm{Pix}}_{S,t-2}(i,j)|>\mathrm{Threshold}2\\ 0\&CenterDot;\&CenterDot;\&CenterDot;\&CenterDot;\&CenterDot;\&CenterDot;\mathrm{else}\end{array}\right.$

In formula, IP _{m, H}im age (i, j) for the coordinate of the middle part H color component in current input panoramic image frame be the testing result that (i, j) puts, IP _{m, S}im age (i, j) is the testing result that (i, j) puts for the coordinate of the middle part S color component in current input panoramic image frame, represents by binary map, and 1 represents foreground moving object, and 0 represents background, Pix _{h, t}(i, j) and Pix _{h, t-2}(i, j) represents that middle part H color component coordinate in t and t-2 time chart picture frame of full-view video image is the pixel value that (i, j) puts respectively, Pix _{s, t}(i, j) and Pix _{s, t-2}(i, j) represents that middle part S color component coordinate in t and t-2 time chart picture frame of full-view video image is the pixel value that (i, j) puts respectively, and Threshold2 is corresponding judgment threshold, and value is 45 here;

P6 process be the destination object nearby of distance ODVS, be divided into H component and S component, its computing formula as shown in (10),

${\mathrm{IP}}_{N,H}\mathrm{Image}(i,j)=\left\{\begin{array}{c}1\&CenterDot;\&CenterDot;\&CenterDot;\&CenterDot;\&CenterDot;\&CenterDot;\mathrm{if}|{\mathrm{Pix}}_{H,t}(i,j)-{\mathrm{Pix}}_{H,t-1}(i,j)|>\mathrm{Threshold}3\\ 0\&CenterDot;\&CenterDot;\&CenterDot;\&CenterDot;\&CenterDot;\&CenterDot;\mathrm{else}\end{array}\right.---\left(10\right)$

${\mathrm{IP}}_{N,S}\mathrm{Image}(i,j)=\left\{\begin{array}{c}1\&CenterDot;\&CenterDot;\&CenterDot;\&CenterDot;\&CenterDot;\&CenterDot;\mathrm{if}|{\mathrm{Pix}}_{S,t}(i,j)-{\mathrm{Pix}}_{S,t-1}(i,j)|>\mathrm{Threshold}3\\ 0\&CenterDot;\&CenterDot;\&CenterDot;\&CenterDot;\&CenterDot;\&CenterDot;\mathrm{else}\end{array}\right.$

In formula, IP _{n, H}im age (i, j) for the coordinate of the bottom H color component in current input panoramic image frame be the testing result that (i, j) puts, IP _{n, S}im age (i, j) is the testing result that (i, j) puts for the coordinate of the bottom S color component in current input panoramic image frame, represents by binary map, and 1 represents foreground moving object, and 0 represents background, Pix _{h, t}(i, j) and Pix _{h, t-1}(i, j) represents that middle part H color component coordinate in t and t-1 time chart picture frame of full-view video image is the pixel value that (i, j) puts respectively, Pix _{s, t}(i, j) and Pix _{s, t-1}(i, j) represents that middle part S color component coordinate in t and t-1 time chart picture frame of full-view video image is the pixel value that (i, j) puts respectively, and Threshold3 is corresponding judgment threshold, and value is 45 here;

P8 mainly split after destination object split arrange process, on the one hand, because colourity is partly formed primarily of tone H and saturation degree S two, need here to carry out or calculation process; On the other hand, in P4, P5 and P6 processing procedure, view picture panoramic picture is divided into three, upper, middle and lower part, also needs here to carry out or calculation process; Obtain the segmentation image of the moving target object on view picture panoramic picture through such process, disposal route as shown in formula (11),

D(x，y，t)＝IP _L，HIm age(i，j)∨IP _L，SIm age(i，j)∨IP _M，HIm age(i，j)∨IP _M，SIm age(i，j)∨IP _N，HIm age(i，j)∨IP _N，SIm age(i，j) (11)

In formula, D (x, y, t) for coordinate in current input panoramic image frame be the testing result that (i, j) puts, IP _{n, H}im age (i, j) for the coordinate of the bottom H color component in current input panoramic image frame be the testing result that (i, j) puts, IP _{n, S}im age (i, j) for the coordinate of the bottom S color component in current input panoramic image frame be the testing result that (i, j) puts, IP _{m, H}im age (i, j) for the coordinate of the middle part H color component in current input panoramic image frame be the testing result that (i, j) puts, IP _{m, S}im age (i, j) for the coordinate of the middle part S color component in current input panoramic image frame be the testing result that (i, j) puts, IP _{l, H}im age (i, j) for the coordinate of the top H color component in current input panoramic image frame be the testing result that (i, j) puts, IP _{l, S}im age (i, j) for the coordinate of the top S color component in current input panoramic image frame be the testing result that (i, j) puts; The testing result of P8 submits to P9 process;

P9 mainly judges whether new motor image vegetarian refreshments, if there is new motor image vegetarian refreshments to be submitted to P11 process, otherwise is submitted to P10 process;

Whether the translational speed of P10 detected target object is greater than the threshold value δ of regulation, and the related pixel point satisfied condition submits to P16 process, otherwise submits to P15 process;

P11 selects the τ in formula (3) according to the speed of the translational speed of destination object, pixel for the fast destination object of translational speed adopts little τ value, the pixel of the destination object that translational speed is slow adopts large τ value, and computing method are as shown in formula (12)

τ _i，M＝k/S _i(t) (12)

In formula, τ _{i, M}be the duration of i-th destination object, S _it () is the translational speed of i-th destination object, k is a certain constant;

For the far and near different destination object of distance ODVS, τ _{i, M}value also needs suitable adjustment, for the destination object of same movement velocity, the speed that the near destination object of distance ODVS reflects on panoramic expansion figure can be hurry up, the speed that the destination object that then distance ODVS is far away reflects on panoramic expansion figure can be slow, and what formula (12) was tried to achieve is from the τ in ODVS moderate distance situation _{i, M}value, carried out normalized to the duration in the present invention, processed respectively in P12, P13 and P14, specific practice is that the duration is nearby set to H _τ(x, y, t)=τ _{i, M}-α, the duration of middle distant place is set to H _τ(x, y, t)=τ _{i, M}, the duration is at a distance set to H _τ(x, y, t)=τ _{i, M}+ α; Wherein α=2 ~ 4; At a distance, Fig. 7 is shown in middle distant place and division nearby;

P15 mainly for the grey scale pixel value that the destination object being in temporary stop motion state is formed carry out maintenance process, for following the tracks of and lock the destination object that those are in temporary stop motion state, computing method as shown in formula (13),

H _τ(x，y，t)＝max(0，H _τ(x，y，t-1)) (13)

P16 carries out subtracting 1 computing mainly for those grey scale pixel values that still moving destination object is formed, for progressively removing those motions pixel more of a specified duration, computing method as shown in formula (14),

H _τ(x，y，t)＝max(0，H _τ(x，y，t-1)-1) (14)

P17, according to the ROI obtaining each moving target object in the result of P12, P13, P14, P15 and P16, calculates i-th ROI _icenter-of-mass coordinate ROI _{i, m}(x, y, t), obtains i-th ROI in then utilizing one to circulate _icenter-of-mass coordinate ROI _{i, m}(x, y, t-1) calculates the speed of i-th moving target object, computing method as shown in formula (15),

$S_i\left(t\right)=\frac{|{\mathrm{ROI}}_{i,m}(x,y,t)-{\mathrm{ROI}}_{i,m}(x,y,t-1)|}{\mathrm{\&Delta;t}}---\left(15\right)$

In formula, ROI _{i, m}(x, y, t) is i-th ROI in frame under process _icenter-of-mass coordinate, ROI _{i, m}(x, y, t-1) is i-th ROI in a upper processed frame _icenter-of-mass coordinate, Δ t was two frame period times, S _it () is i-th ROI in frame under process _imovement velocity; According to this movement velocity S _it () conduct will detect Rule of judgment, as calculating prolongeding time τ in P11 in P10 _{i, M}foundation;

In the moving target object extraction shown in Fig. 8 and tracking processing procedure, because computation complexity is not high, parameter and threshold value choose fairly simple, be convenient to realize in SOC (system on a chip), substantially single goal Object tracking and simple multiple goal Object tracking problem can have been met, as shown in Figure 10, there is not situations such as blocking, merge and be separated in multiple goal object in this case; For Complex multi-target tracking problem, as shown in figure 11, in this case, there is merging and the situation such as to be separated in multiple goal object, need to process in another thread of the multiple-target system in the application software of PC shown in Fig. 9 to effectively distinguish different destination objects and recording each destination object movement locus in video sequence image; After P17 process terminates, obtain i ROI area-of-interest, the center-of-mass coordinate value in region and the sizes values of regional frame; The extraction of these destination objects and tracking data being submitted in the PC shown in Fig. 9 by network in the mode of software interface with full-view video image is processed;

Multiple goal subject tracking unit be used for multiple goal object block in scene, be separated and the situation such as merging time effectively follow the tracks of; Treatment scheme below according to Fig. 9 is described multiple goal Object tracking;

S1 step mainly reads the number i of present frame ROI and the center p of binary map _ithe sizes values s of (x, y) and ROI _i(Δ x, Δ y), uses p _i(x, y) and s _i(Δ x, Δ y) creates i ROI software object object _{rOI (i)};

S2 step mainly judges already present destination object object in the number i of present frame ROI and previous frame _{presence (j)}number j compare, two or more destination object is deposited in case, if the number i of ROI equals already present destination object object _{presence (j)}number j when showing in scene destination object number and do not change simultaneously also not occur merging or being separated, then proceed to S3 pairing;

To the ROI in present frame and the destination object object that also existed in S3 step _{presence (j)}adopt the very little criterion of spatial dimension change to match, match computing method as shown in formula (23),

d _t，j＝|ROIi(x，y，t)-object _presence(j)(x，y，t-1)| (23)

In formula, d _{t, j}for i-th ROI in present frame _i(x, y, t) and an already present jth object _{presence (j)}city distance between (x, y, t-1), if meet d _{t, j}the condition of≤D, shows both successful matchings;

For successful matching ROI _i(x, y, t) and object _{presence (j)}(x, y, t-1), first, by system time with HHMMSS form and object _{presence (j)}(x, y, t-1) adds object object to _{presence (j)}enumerations in; Then with ROI _i(x, y, t) substitutes object _{presence (j)}(x, y, t-1);

If the number i of ROI is greater than already present destination object object _{presence (j)}number j show in scene, have new destination object to enter or occurred situation about being separated; Processing procedure is as follows: first adopt S3 treatment step to carry out pairing and calculate, and adopt the very little criterion of spatial dimension change to match, pairing computing method are as shown in formula (23); Then S7 treatment step is proceeded to;

Calculate the ROI not having successful matching by formula (24) in S7 treatment step _i(x, y, t) and group _jthe city distance of (x, y),

In formula, ROI _i(x, y, t) be not for having the ROI software object of successful matching, group _j(x, y) group of objects for existing in scene, then proceeds to the judgement process of S8;

S8 judges that process judges by formula (25),

dR _t，j≤D _R(25)

In formula, dR _{t, j}the ROI not having successful matching _i(x, y, t) and group _jthe city distance of (x, y), D _rfor the threshold value arranged; When judging that formula (25) is set up, representing that group of objects has separation case to occur, proceeding to S9 treatment step; Otherwise proceed to S10 treatment step;

S9 treatment step the first matching condition is mated, and reads respectively and is stored in group of objects group _jeach relevant target object object in (x, y) _{presence (j)}the color histogram of (x, y, t-1) data; Simultaneously to the ROI not having successful matching _i(x, y, t) does color histogram using the tolerance of Bhattacharyya distance as two color histogram similaritys, the computing method of discrete Bart Charlie subbreed number as shown in formula (16),

$\mathrm{\&rho;}[P_i,Q_j]=\underset{\mathrm{\&mu;}=1}{\overset{m}{\mathrm{\&Sigma;}}}\sqrt{p_i^{\left(\mathrm{\&mu;}\right)}{q}_j^{\left(\mathrm{\&mu;}\right)}}---\left(16\right)$

In formula, ρ [P _i, Q _j] ∈ [0,1] is Bart Charlie subbreed number, Bhattacharyya distance formula (17) calculates,

$d=\min \left\{d_{i,j}\right\}=\min \left\{\sqrt{1-\mathrm{\&rho;}[P_i,Q_j]}\right\}---\left(17\right)$

In formula, d is Bhattacharyya distance, if the threshold value Threshold4 that this value is less than a certain regulation just represents that the match is successful; Proceed to step S12 for what the match is successful, otherwise proceed to step S11;

S11 treatment step does not have successful situation to carry out second time matching treatment to first time coupling, and what adopt in first time matching treatment is the color characteristic of destination object, and in second time coupling employing be Hu not bending moment; Specific practice is: read respectively and be stored in group of objects group _jeach relevant target object object in (x, y) _{presence (j)}the Hu invariant moment features vector information of (x, y, t-1); Simultaneously to the ROI not having successful matching _i(x, y, t) does the vector operation of Hu invariant moment features, using the tolerance of Euclidean distance as two Hu invariant moment features similaritys, the computing method of Hu not bending moment 7 proper vectors as shown in formula (18),

φ ₁＝η ₂₀+η ₀₂

${\mathrm{\&phi;}}_2={({\mathrm{\&eta;}}_{20}-{\mathrm{\&eta;}}_{02})}^2+{4\mathrm{\&eta;}}_{11}^2$

φ ₃＝(η ₃₀-3η ₁₂) ²+(3η ₂₁-η ₀₃) ²

φ ₄＝(η ₃₀+η ₁₂) ²+(η ₂₁+η ₀₃) ²

φ ₅＝(η ₃₀-3η ₁₂)(η ₃₀+η ₁₂)[(η ₃₀+η ₁₂) ²-3(η ₂₁+η ₀₃) ²]+

(3η ₂₁-η ₀₃)(η ₂₁-η ₀₃)[3(η ₃₀+η ₁₂) ²-(η ₂₁+η ₀₃) ²] (18)

φ ₆＝(η ₂₀-η ₀₂)[(η ₃₀+η ₁₂) ²-3(η ₂₁+η ₀₃) ²]+4η ₁₁(η ₃₀+η ₁₂)(η ₂₁+η ₀₃)

φ ₇＝(3η ₂₁-η ₀₃)(η ₃₀+η ₁₂)[(η ₃₀+η ₁₂) ²-3(η ₂₁+η ₀₃) ²]-

(η ₃₀-3η ₁₂)(η ₂₁+η ₀₃)[3(η ₃₀+η ₁₂) ²-(η ₂₁+η ₀₃) ²]

In formula, η _pqfor normalization centre distance, computing method are provided by formula (19),

${\mathrm{\&eta;}}_{\mathrm{pq}}={\mathrm{\&mu;}}_{\mathrm{pq}}/{\mathrm{\&mu;}}_{\mathrm{pq}}^r\&CenterDot;\&CenterDot;\&CenterDot;\&CenterDot;\&CenterDot;\&CenterDot;r=(p+q)/2\&CenterDot;\&CenterDot;\&CenterDot;p+q=\mathrm{2,3},\&CenterDot;\&CenterDot;\&CenterDot;---\left(19\right)$

In formula, μ _pqcentered by distance, computing method are provided by formula (20),

${\mathrm{\&mu;}}_{\mathrm{pq}}=\underset{x=1}{\overset{M}{\mathrm{\&Sigma;}}}\underset{y=1}{\overset{N}{\mathrm{\&Sigma;}}}{(x-\stackrel{\&OverBar;}{x})}^p{(y-\stackrel{\&OverBar;}{y})}^{q}f(x,y)---\left(20\right)$

In formula, f (x, y) is destination object ROI _ithe binary map of (x, y, t), with for destination object ROI _ithe barycentric coordinates of (x, y, t), computing method are provided by formula (21),

$\stackrel{\&OverBar;}{x}=m_{10}/m_{11},\stackrel{\&OverBar;}{y}=m_{01}/m_{00}$

$\left(21\right)$

$m_{\mathrm{pq}}=\underset{x=1}{\overset{M}{\mathrm{\&Sigma;}}}\underset{y=1}{\overset{N}{\mathrm{\&Sigma;}}}{x}^p{y}^{q}f(x,y)$

In formula, f (x, y) is destination object ROI _ithe binary map of (x, y, t); Can pass through for each destination object 7 vectors that formula (18) ~ (21) calculate its Hu not bending moment, then use formula (22) Euclidean distance go to judge both similarity,

$d_{\mathrm{ms}}=\underset{i=1}{\overset{7}{\mathrm{\&Sigma;}}}{({\mathrm{\&phi;}}_{\mathrm{mi}}-{\mathrm{\&phi;}}_{\mathrm{si}})}^2\&CenterDot;\&CenterDot;\&CenterDot;s\&Element;S---\left(22\right)$

In formula, d _msfor being stored in group of objects group _jeach relevant target object object in (x, y) _{presence (j)}the Hu invariant moment features vector value φ of (x, y, t-1) _siwith by the destination object ROI mated _ithe Hu invariant moment features vector value φ of (x, y, t) _mieuclidean distance value, S be multiple destination object merge time destination object number, if meet d _mj=min{d _msj, s ∈ S, so a jth object _{presence (j)}(x, y, t-1) is just judged to be by the destination object mated; Then S12 treatment step is proceeded to;

First group of objects group is detected in S12 treatment step _jeach relevant target object object in (x, y) _{presence (j)}the number N of (x, y, t-1), if N=2 is so by group _jthe object reference variable of (x, y) is Null, with the ROI that the match is successful _ithe center of (x, y, t) substitutes object _{presence (j)}the center of (x, y, t-1); If N > 2 is so by group _jthe object reference variable of (x, y) is Null, will remain in group of objects group _jeach relevant target object object in (x, y) _{presence (j)}(x, y, t-1) re-creates new group of objects group _j(x, y), with the ROI that the match is successful _ithe center of (x, y, t) substitutes object _{presence (j)}the center of (x, y, t-1);

S10 treatment step mainly for the process having new destination object to enter scenario, at this moment needs to use ROI _i(x, y, t) creates a new object _{presence (j)}(x, y, t-1) destination object, and by formulae discovery color histogram and the Hu not data such as bending moment, and be saved in object with its spatial positional information _{presence (j)}in (x, y, t-1) destination object;

If the number i of ROI is less than already present destination object object _{presence (j)}number j show that in scene existing destination object leaves or occurred situation about merging; Processing procedure is as follows: first adopt S3 treatment step to carry out pairing and calculate, and adopt the very little criterion of spatial dimension change to match, pairing computing method are as shown in formula (23); Then S4 treatment step is proceeded to;

According to the result in S3 treatment step in S4 treatment step, if there is two or more distance value d when matching computing formula (23) calculating _{t, j}be less than D, namely meet formula (26),

(d _t，n＜D)∧(d _t，m＜D) (26)

So just think the n-th destination object object _{presence (n)}with m destination object object _{presence (m)}there occurs merging, proceed to S5 process; Otherwise proceed to S6 process;

S5 treatment step according to the judged result of formula (26), by the n-th destination object object _{presence (n)}with m destination object object _{presence (m)}carry out merging treatment, create a new group of objects group _j(x, y);

S6 treatment step, according to the judged result of formula (26), shows the scene having a destination object to leave, to the object not having successful matching _{presence (j)}the object reference variable of (x, y, t-1) is Null, allows the garbage collector in Java language automatically collect the destination object having left scene.

Extraction and the tracking algorithm of the panorama sequence images expansion shown in Fig. 8, destination object realize primarily of C language, run on davinci platform; Formalization process and the behavior semantization process scheduling algorithm of the multiple goal Object tracking Processing Algorithm shown in Fig. 9 and image are write primarily of Java language, run on PC.

Claims (9)

1. the multiple goal object-tracking systems of a panorama video sequence image, it is characterized in that: the multiple goal object-tracking systems of described panorama video sequence image comprises the omnibearing shooting device of the full-view video image capturing whole scene internal object object, described omnibearing shooting device ODVS represents, described ODVS is placed in above the middle part of monitoring scene, and described ODVS is connected with microprocessor by USB interface; Described microprocessor is connected with PC by computer network; Described microprocessor comprises:

Video image reading unit, for being read the panoramic picture captured by ODVS by USB interface, and submits to video image expanding unit and video image storage unit by the panoramic picture of reading;

Video image expanding unit, for panoramic picture is carried out column expansion, target-object detecting unit submitted to by the panorama histogram picture after expansion;

Destination object detecting and tracking unit, for detecting the moving target object existed in panorama column unfolded image, and live destination object by rectangle circle, with a kind of Motion History or Energy Images algorithm, hereinafter referred to as MHoEI algorithm, extract and tracking target object with the motion history of destination object and energygram picture; Described PC mainly carries out formalization process and the process of behavior semantization of full-view video image, multiple goal subject tracking unit be used for multiple goal object block in scene, be separated and the situation such as merging time effectively follow the tracks of;

Described video image expanding unit, for panoramic picture is carried out column expansion, described destination object detecting and tracking unit submitted to by the panorama histogram picture after expansion;

Described destination object detecting and tracking unit, for the moving target object that detection and tracking exist in panorama column unfolded image, adopt MHoEI algorithm to extract and tracking target object the motion history of destination object and energygram picture, obtain region of interest ROI, the center-of-mass coordinate value in region and the sizes values of regional frame; The extraction of these destination objects and tracking data are also submitted to described multiple goal subject tracking unit in the mode of software interface by network with full-view video image and process;

Described multiple goal subject tracking unit, block in scene for multiple goal object, be separated and the situation such as merging time effectively follow the tracks of;

Described destination object detecting and tracking unit, in order to realize the shade that moving target object is effectively split and elimination moving target produces while segmentation, some feature utilizing shade and non-hatched area to distinguish in conducting frame difference method process is to eliminate shade, the feature that in Main Basis shadow region, the colourity of pixel is almost equal compared with background pixel colourity, the rgb color space of original image is changed into HSI color space, and then carry out frame difference method computing and just can eliminate shade, rgb color space changes into the computing method of HSI color space as shown in formula (7),

$H=\left\{\begin{array}{c}\mathrm{\&theta;}......G\&GreaterEqual;B\\ 2\mathrm{\&pi;}-\mathrm{\&theta;}...G<B\end{array}\right.$

$S=1-\frac{3}{(R+G+B)}\left[\min (R,G,B)\right]---\left(7\right)$

$\mathrm{\&theta;}=\arccos \left\{\frac{[(R-G)+(R-B)]/2}{[{(R-G)}^2+(R-B)(G-B){]}^{1/2}}\right\}$

In formula, R is the red component in rgb color space, and G is the green component in rgb color space, and B is the blue component in rgb color space; H is the tone in HSI color space, represents by angle, reflects color closest to which type of spectral wavelength; S is the depth degree of the saturation degree in HSI color space, characterizing color; Tone H and saturation degree S claims colourity altogether;

For the destination object of the distant place of distance ODVS, on H component and S component, carry out frame difference method process respectively, its computing formula as shown in (8),

${\mathrm{IP}}_{L,H}\mathrm{Im\; age}(i,j)=\left\{\begin{array}{c}1......\mathrm{if}|{\mathrm{Pix}}_{H,t}(i,j)-{\mathrm{Pix}}_{H,t-3}(i,j)|>\mathrm{Threshold}1\\ 0......\mathrm{else}\end{array}\right.---\left(8\right)$

${\mathrm{IP}}_{L,S}\mathrm{Im\; age}(i,j)=\left\{\begin{array}{c}1......\mathrm{if}|{\mathrm{pix}}_{S,t}(i,j)-{\mathrm{Pix}}_{S,t-3}(i,j)|>\mathrm{Threshold}1\\ 0......\mathrm{else}\end{array}\right.$

In formula, IP _l,Himage (i, j) for the coordinate of the top H color component in current input panoramic image frame be the testing result that (i, j) puts, IP _l,Simage (i, j) is the testing result that (i, j) puts for the coordinate of the top S color component in current input panoramic image frame, represents by binary map, and 1 represents foreground moving object, and 0 represents background, Pix _h,t(i, j) and Pix _{h, t-3}(i, j) represents that top H color component coordinate in t and t-3 time chart picture frame of full-view video image is the pixel value that (i, j) puts respectively, Pix _s,t(i, j) and Pix _{s, t-3}(i, j) represents that top S color component coordinate in t and t-3 time chart picture frame of full-view video image is the pixel value that (i, j) puts respectively, and Threshold1 is corresponding judgment threshold, and value is 45 here;

For destination object at a distance in distance ODVS, on H component and S component, carry out frame difference method process respectively, its computing formula as shown in (9),

${\mathrm{IP}}_{M,H}\mathrm{Im\; age}(i,j)=\left\{\begin{array}{c}1......\mathrm{if}|{\mathrm{Pix}}_{H,t}(i,j)-{\mathrm{Pix}}_{H,t-2}(i,j)|>\mathrm{Threshold}2\\ 0......\mathrm{else}\end{array}\right.---\left(9\right)$

${\mathrm{IP}}_{M,S}\mathrm{Im\; age}(i,j)=\left\{\begin{array}{c}1......\mathrm{if}|{\mathrm{Pix}}_{S,t}(i,j)-{\mathrm{Pix}}_{S,t-2}(i,j)|>\mathrm{Threshold}2\\ 0......\mathrm{else}\end{array}\right.$

In formula, IP _m,Himage (i, j) for the coordinate of the middle part H color component in current input panoramic image frame be the testing result that (i, j) puts, IP _m,Simage (i, j) is the testing result that (i, j) puts for the coordinate of the middle part S color component in current input panoramic image frame, represents by binary map, and 1 represents foreground moving object, and 0 represents background, Pix _h,t(i, j) and Pix _{h, t-2}(i, j) represents that middle part H color component coordinate in t and t-2 time chart picture frame of full-view video image is the pixel value that (i, j) puts respectively, Pix _s,t(i, j) and Pix _{s, t-2}(i, j) represents that middle part S color component coordinate in t and t-2 time chart picture frame of full-view video image is the pixel value that (i, j) puts respectively, and Threshold2 is corresponding judgment threshold, and value is 45 here;

For the destination object nearby of distance ODVS, on H component and S component, carry out frame difference method process respectively, its computing formula as shown in (10),

${\mathrm{IP}}_{N,H}\mathrm{Im\; age}(i,j)=\left\{\begin{array}{c}1......\mathrm{if}|{\mathrm{Pix}}_{H,t}(i,j)-{\mathrm{Pix}}_{H,t-1}(i,j)|>\mathrm{Threshold}3\\ 0......\mathrm{else}\end{array}\right.---\left(10\right)$

${\mathrm{IP}}_{M,S}\mathrm{Im\; age}(i,j)=\left\{\begin{array}{c}1......\mathrm{if}|{\mathrm{Pix}}_{S,t}(i,j)-{\mathrm{Pix}}_{S,t-1}(i,j)|>\mathrm{Threshold}3\\ 0......\mathrm{else}\end{array}\right.$

In formula, IP _n,Himage (i, j) for the coordinate of the bottom H color component in current input panoramic image frame be the testing result that (i, j) puts, IP _n,Simage (i, j) is the testing result that (i, j) puts for the coordinate of the bottom S color component in current input panoramic image frame, represents by binary map, and 1 represents foreground moving object, and 0 represents background, Pix _h,t(i, j) and Pix _{h, t-1}(i, j) represents that middle part H color component coordinate in t and t-1 time chart picture frame of full-view video image is the pixel value that (i, j) puts respectively, Pix _s,t(i, j) and Pix _{s, t-1}(i, j) represents that middle part S color component coordinate in t and t-1 time chart picture frame of full-view video image is the pixel value that (i, j) puts respectively, and Threshold3 is corresponding judgment threshold, and value is 45 here;

Finally, process is arranged for the destination object split after segmentation; On the one hand, because colourity is formed primarily of tone H and saturation degree S two part, need here to carry out or calculation process; On the other hand, in P4, P5 and P6 processing procedure, view picture panoramic picture is divided into three, upper, middle and lower part, also needs here to carry out or calculation process; Obtain the segmentation image of the moving target object on view picture panoramic picture through such process, disposal route as shown in formula (11),

D(x,y,t)＝IP _L,HImage(i,j)∨IP _L,SImage(i,j)∨IP _M,HImage(i,j)∨

IP _M,SImage(i,j)∨IP _N,HImage(i,j)∨IP _N,SImage(i,j) (11)

In formula, D (x, y, t) for coordinate in current input panoramic image frame be the testing result that (i, j) puts, IP _n,Himage (i, j) for the coordinate of the bottom H color component in current input panoramic image frame be the testing result that (i, j) puts, IP _n,Simage (i, j) for the coordinate of the bottom S color component in current input panoramic image frame be the testing result that (i, j) puts, IP _m,Himage (i, j) for the coordinate of the middle part H color component in current input panoramic image frame be the testing result that (i, j) puts, IP _m,Simage (i, j) for the coordinate of the middle part S color component in current input panoramic image frame be the testing result that (i, j) puts, IP _l,Himage (i, j) for the coordinate of the top H color component in current input panoramic image frame be the testing result that (i, j) puts, IP _l,Simage (i, j) for the coordinate of the top S color component in current input panoramic image frame be the testing result that (i, j) puts.

2. the multiple goal object-tracking systems of panorama video sequence image as claimed in claim 1, it is characterized in that: described destination object detecting and tracking unit, for the moving target object that detection and tracking exist in panorama column unfolded image, adopt MHoEI algorithm to extract and tracking target object the motion history of destination object and energygram picture, represent with formula (3):

$H_{\mathrm{\&tau;}}(x,y,t)=\left\{\begin{array}{cc}\mathrm{\&tau;}& \mathrm{if\; D}(x,y,t)=1\\ \max (0,H_{\mathrm{\&tau;}}(x,y,t-1))& \mathrm{if\; S}\&le;\mathrm{\&delta;}\\ \max (0,H_{\mathrm{\&tau;}}(x,y,t-1)-1)& \mathrm{otherwise}\end{array}\right.---\left(3\right)$

In formula, S is the movement velocity of destination object, and τ is the duration, the binary picture sequence that D (x, y, t) is moving region, H _τthe binary picture sequence that (x, y, t-1) is non-moving areas, duration τ needs to carry out dynamic conditioning according to destination object movement velocity S, and δ is threshold speed.

3. the multiple goal object-tracking systems of panorama video sequence image as claimed in claim 1, it is characterized in that: described video image expanding unit, for panoramic picture is carried out column expansion, according to the centre coordinate of the panoramic picture calculated in initialization process and the inside and outside circle radius of image, by the initial point O of the centre coordinate of panoramic picture setting plane coordinate system ^*(0,0), X ^*axle, Y ^*the internal diameter of axle, panoramic picture is r, external diameter is R, and with radius of a circle in the middle of r1=(r+R)/2 setting, position angle is β=tan ^-1(y ^*/ x ^*); Panorama column unfolded image is with true origin O ^*(0,0), X ^*axle, Y ^*axle is plane coordinate system, is r and X by the internal diameter in panoramic picture ^*the intersection point (r, 0) of axle is as true origin O ^*(0,0), counterclockwise launches with azimuthal angle beta; Set up any point pixel coordinates P in panorama column unfolded image ^*(x ^*, y ^*) with panoramic picture in pixel coordinates Q ^*(x ^*, y ^*) corresponding relation, its calculating formula is:

x ^*＝y ^*/(tan(360x ^**/π(R+r))) (4)

y ^*＝(y ^**+r)cosβ (5)

In above formula, x ^*, y ^*for the pixel coordinates value of panorama column unfolded image, x ^*, y ^*for the pixel coordinates value of panoramic picture, R is the external diameter of circular panoramic picture, and r is the internal diameter of circular panoramic picture, and β is the position angle of circular panoramic picture coordinate;

Because the scope of above-mentioned panorama column unfolded image is 0 ~ 360 °, the situation being judged as two destination objects is there will be when same tracking target object is in 0 ° or 360 ° of edges, the scope of panorama column unfolded image is set to 0 ~ 380 °, namely has the overlapping region of about 20 °.

4. the multiple goal object-tracking systems of panorama video sequence image as described in claim 1 or 2, it is characterized in that: described destination object detecting and tracking unit, when adopting the extraction of MHoEI algorithm and tracking target object, obtaining the ROI of each moving target object, calculating i-th ROI _icenter-of-mass coordinate ROI _i,m(x, y, t), obtains i-th ROI in then utilizing one to circulate _icenter-of-mass coordinate ROI _i,m(x, y, t-1) calculates the speed of i-th moving target object, computing method as shown in formula (15),

$S_i\left(t\right)=\frac{|{\mathrm{ROI}}_{i,m}(x,y,t)-{\mathrm{ROI}}_{i,m}(x,y,t-1)|}{\mathrm{\&Delta;t}}---\left(15\right)$

In formula, ROI _i,m(x, y, t) is i-th ROI in frame under process _icenter-of-mass coordinate, ROI _i,m(x, y, t-1) is i-th ROI in a upper processed frame _icenter-of-mass coordinate, Δ t was two frame period times, S _it () is i-th ROI in frame under process _imovement velocity.

5. the multiple goal object-tracking systems of panorama video sequence image as described in claim 1 or 2, is characterized in that: described destination object detecting and tracking unit, the destination object movement velocity S calculated by formula (15) _it () is as calculating prolongeding time τ _i,Mfoundation; Computing method as shown in formula (12),

τ _i,M＝k/S _i(t) (12)

In formula, τ _i,Mbe the duration of i-th destination object, S _it () is the translational speed of i-th destination object, k is a certain constant;

For the far and near different destination object of distance ODVS, τ _i,Mvalue also needs suitable adjustment, for the destination object of same movement velocity, the speed that the near destination object of distance ODVS reflects on panoramic expansion figure can be hurry up, the speed that the destination object that then distance ODVS is far away reflects on panoramic expansion figure can be slow, and what formula (12) was tried to achieve is from the τ in ODVS moderate distance situation _i,Mvalue, carried out normalized to the duration here, specific practice is that the duration is nearby set to H _τ(x, y, t)=τ _i,M-α, the duration of middle distant place is set to H _τ(x, y, t)=τ _i,M, the duration is at a distance set to H _τ(x, y, t)=τ _i,M+ α; Wherein α=2 ~ 4.

6. the multiple goal object-tracking systems of panorama video sequence image as described in claim 1 or 2, it is characterized in that: described destination object detecting and tracking unit, each circulation obtains i ROI area-of-interest, the center-of-mass coordinate value in region and the sizes values of regional frame; The extraction of these destination objects and tracking data also call when being supplied to video sequence image process on the middle and senior level with full-view video image in the mode of software interface.

7. the multiple goal object-tracking systems of panorama video sequence image as claimed in claim 1, it is characterized in that: described multiple goal subject tracking unit, block in scene for multiple goal object, be separated and the situation such as merging time effectively follow the tracks of; In order to the effective tracking realizing multiple goal object needs the various attribute datas in destination object to merge, mate, upgrade, each destination object entering scene is created automatically to the object of a software here, the describing method of object is as follows:

object{

// morphological state variable

Locus (distance centered by observer, orientation, system represents with Gauss coordinate)

(centered by observer, distance carries out sameization process to size dimension, with mm ²/ pixel represents)

Shape, attitude (with tight ness rating, solid degree, excentricity, degree of irregularity, minimum enclosed rectangle, figure's ratio, Hu not bending moment etc. represent, preserve with 7 vector value data of Hu not bending moment)

Colourity (with the H component in HSI color space and S representation in components, preserving with color histogram data)

// motion state variable

Movement velocity (representing with mm/s)

Direction of motion (orientation references centered by observer)

Attitude rate

The residence time (representing with S)

Die-out time (representing with S)

// motion history status data

Enumerations (with time, locus record)

}。

8. the multiple goal object-tracking systems of panorama video sequence image as claimed in claim 1, it is characterized in that: described multiple goal subject tracking unit, when adopting color histogram to occur division situation as color characteristic model and multiple goal object, first mates foundation, specific practice is before multiple object merging, is kept in object by the chrominance information of each object; The color histogram of the relevant target object be stored in object is read respectively when division situation appears in multiple goal object data; Color histogram is done to the ROI of relevant target object during division situation simultaneously using the tolerance of Bhattacharyya distance as two color histogram similaritys, the computing method of discrete Bart Charlie subbreed number as shown in formula (16),

$\mathrm{\&rho;}[P_i,Q_j]=\underset{\mathrm{\&mu;}=1}{\overset{m}{\mathrm{\&Sigma;}}}\sqrt{p_i^{\left(\mathrm{\&mu;}\right)}{q}_i^{\left(\mathrm{\&mu;}\right)}}---\left(16\right)$

In formula, ρ [P _i, Q _j] ∈ [0,1] is Bart Charlie subbreed number, Bhattacharyya distance formula (17) calculates,

$d=\min \left\{d_{i,j}\right\}=\min \{\sqrt{1-\mathrm{\&rho;}[P_i,Q_j]\}}---\left(17\right)$

In formula, d is Bhattacharyya distance, if the threshold value Threshold4 that this value is less than a certain regulation just represents that the match is successful, continues by the second coupling according to mating for not having the destination object that the match is successful.

9. the multiple goal object-tracking systems of panorama video sequence image as claimed in claim 1, it is characterized in that: described multiple goal subject tracking unit, when adopting Hu bending moment does not occur division situation as destination object shape facility and multiple goal object, second mates foundation, specific practice is before multiple object merging, is kept in object by 7 vector information of the Hu invariant moment features of each object; The Hu invariant moment features vector information be stored in the relevant target object of object is read respectively when division situation appears in multiple goal object; The vector operation of Hu invariant moment features is done to relevant target object during division situation simultaneously, using the tolerance of Euclidean distance as two Hu invariant moment features similaritys, the computing method of Hu not bending moment 7 proper vectors as shown in formula (18),

φ ₁＝η ₂₀+η ₀₂

${\mathrm{\&phi;}}_2={({\mathrm{\&eta;}}_{20}-{\mathrm{\&eta;}}_{02})}^2+4{\mathrm{\&eta;}}_{11}^2$

φ ₃＝(η ₃₀-3η ₁₂) ²+(3η ₂₁-η ₀₃) ²

φ ₄＝(η ₃₀+η ₁₂) ²+(η ₂₁+η ₀₃) ²

φ ₅＝(η ₃₀-3η ₁₂)(η ₃₀+η ₁₂)[(η ₃₀+η ₁₂) ²-3(η ₂₁+η ₀₃) ²]+ (18)

(3η ₂₁-η ₀₃)(η ₂₁-η ₀₃)[3(η ₃₀+η ₁₂) ²-(η ₂₁+η ₀₃) ²]

φ ₆＝(η ₂₀-η ₀₂)[(η ₃₀+η ₁₂) ²-3(η ₂₁+η ₀₃) ²]+4η ₁₁(η ₃₀+η ₁₂)(η ₂₁+η ₀₃)

φ ₇＝(3η ₂₁-η ₀₃)(η ₃₀+η ₁₂)[(η ₃₀+η ₁₂) ²-3(η ₂₁+η ₀₃) ²]-

(η ₃₀-3η ₁₂)(η ₂₁+η ₀₃)[3(η ₃₀+η ₁₂) ²-(η ₂₁+η ₀₃) ²]

In formula, η _pqfor normalization centre distance, computing method are provided by formula (19),

${\mathrm{\&eta;}}_{\mathrm{pq}}={\mathrm{\&mu;}}_{\mathrm{pq}}/{\mathrm{\&mu;}}_{\mathrm{pq}}^r......r=(p+q)/2...p+q=\mathrm{2,3},...---\left(19\right)$

In formula, μ _pqcentered by distance, computing method are provided by formula (20),

${\mathrm{\&mu;}}_{\mathrm{pq}}=\underset{x=1}{\overset{M}{\mathrm{\&Sigma;}}}\underset{y=1}{\overset{N}{\mathrm{\&Sigma;}}}{(x-\stackrel{\&OverBar;}{x})}^p{(y-\stackrel{\&OverBar;}{y})}^{q}f(x,y)---\left(20\right)$

In formula, the binary map that f (x, y) is destination object, with for the barycentric coordinates of destination object, computing method are provided by formula (21),

$\begin{array}{c}\stackrel{\&OverBar;}{x}=m_{10}/m_{11},\stackrel{\&OverBar;}{y}=m_{01}/m_{00}\\ m_{\mathrm{pq}}=\underset{x=1}{\overset{M}{\mathrm{\&Sigma;}}}\underset{y=1}{\overset{N}{\mathrm{\&Sigma;}}}{x}^p{y}^{q}f(x,y)\end{array}---\left(21\right)$

In formula, the binary map that f (x, y) is destination object; Can pass through for each destination object 7 vectors that formula (18) ~ (21) calculate its Hu not bending moment, then use formula (22) Euclidean distance go to judge both similarity,

$d_{\mathrm{ms}}=\underset{t=1}{\overset{7}{\mathrm{\&Sigma;}}}{({\mathrm{\&phi;}}_{\mathrm{mi}}-{\mathrm{\&phi;}}_{\mathrm{si}})}^2...s\&Element;S---\left(22\right)$

In formula, d _msfor being stored in the Hu invariant moment features vector value φ in object _siwith by the Hu invariant moment features vector value φ of destination object mated _mieuclidean distance value, S be multiple destination object merge time destination object number, if meet d _mj=min{d _msj, s ∈ S, so a jth object is just judged to be by the destination object mated.