CN103325124A - Target detecting and tracking system and method using background differencing method based on FPGA - Google Patents

Abstract

A target detecting and tracking system using a background differencing method based on FPGA comprises a FPGA device, a SD card, an LED indicator light, a DDR2SDRAM, two seven-segment digital tubes, an LTM display screen and a minimum video detecting and tracking system, wherein the minimum video detecting and tracking system is embedded in the FPGA and is integrated in a SOPCbuilder environment with an AVLON bus as a standard. The target detecting and tracking system using the background differencing method based on FPGA is mainly used for detecting and tracking moving objects in static scenes. A target detecting and tracking method using the background differencing method based on FPGA comprises the steps of a. reading AVI format video files with the SD card, b. converting the AVI format video files into multiframe images, c. detecting and tracking the videos based on the images, d. obtaining the moving objects which are to be tracked, and e. displaying tracking results on the LTM display screen. Due to the fact that an FPGA platform is adopted by the system, the system is high in parallelism performance, and computing speed is improved. The target detecting and tracking method is good in tracking effect and adaptability, particle filter algorithm which is suitable for non-linear non-Gaussian dynamic models, and the target detecting and tracking system is similar with real scenes and is good in adaptability and high in tracking accuracy.

Description

A kind of background subtraction point-score target detection tracker and method based on FPGA

Technical field

The invention belongs to field of embedded technology, be specifically related to a kind of background subtraction point-score target detection tracker and method based on FPGA.

Background technology

At present, the video tracking technology is widely used in fields such as military affairs, security protection, traffic, and the embedded video tracker also will become a research focus.Prior art mainly contains following several at the method for moving object detection and motion target tracking:

(1) adjacent frame difference method

Extract moving region in the image by carry out difference between several two field pictures of continuity, basic thought is that adjacent several two field pictures are done calculus of differences, and difference is judged to be moving target greater than the part of a certain threshold value, and other parts are background.This algorithm operation quantity is little, is easy to software and realizes.But the threshold value of this algorithm lacks adaptivity, can only extract frontier point, can not extract the pixel on the zone of object completely, when the motion object velocity is very slow or very fast, might detect less than.When velocity to moving target is very slow, moving target as being background, when velocity to moving target is very fast, is judged into moving target with the part background again.

(2) background subtraction point-score

The present main stream approach in the motion detection is to utilize the difference of present image and background image to detect a kind of technology of moving region, and it generally can provide target data completely.Basic ideas: at first, carry out statistical modeling with the background image sequence of storing in advance or obtain in real time for each pixel, obtain background model; Secondly, current each two field picture and background model are subtracted each other, what calculate that the restriction of certain threshold value occurs in the present image down departs from those bigger pixels of background model value, judge thus that then this pixel is for appearing on the moving target, the result who obtains after the threshold operation of subtracting each other directly provides position, size, shape of target etc., thereby obtains more complete target information.The background subtraction point-score is a kind of the most simple and efficient method.But in actual applications, background tends to change, as the illumination variation of indoor environment, and the change of scenery etc. in the variation of the variation of one day different shade of light constantly, Various Seasonal, the background in the outdoor environment.So Chinese scholars is carried out some improvement on the basis of background subtraction method mostly, make algorithm can detect moving target more accurately.

(3) based on the tracking of Model Matching

Take full advantage of the special nature of target based on the tracking of Model Matching, more can adapt to the target following in the complex environment, but this method generally block with disturbed condition under have stronger robustness, but the very big shortcoming of operand is also arranged.

(4) tracking of mating based on the zone

Be exactly the zone of degree of correlation maximum of target area in the search previous frame in present frame based on Region Matching Algorithm.Thinking be with target image on current image frame with the displacement of different off-set value, handle two overlapping images under each off-set value according to certain similarity measure criterion then, judge the position of target according to criterion and result.The advantage of this algorithm is that when target is not blocked tracking accuracy is very high, follows the tracks of highly stable, but require target distortion little and can not have too greatly and block, otherwise precise decreasing can cause losing of target, and the expressivity of this model is relatively poor, is subject to external interference.

(5) based on the tracking of characteristic matching

Choose the notable feature sequence that has high stability in the target, after the extraction feature characteristic attribute is carried out degree of correlation coupling, obtain the position of moving target.The advantage of this algorithm is to occur under the situation of partial occlusion, as long as there are some features still as seen, and still correct pursuit movement target.

(6) based on the tracking of kinetic characteristic

The algorithm that generally can be divided into predicting tracing algorithm and associated objects motion continuity.The motion prediction algorithm can be estimated the position that next moment target may occur according to the motion feature of target, and related continuity algorithm is used in the occasion that track algorithm merges more.

The means that realize image processing algorithm at present mainly contain following several at different application demands: application-specific integrated circuit ASIC, digital signal processor DSP, arm processor, present programmable gate array FPGA.Application-specific integrated circuit ASIC design cost height can not be revised; , DSP and ARM are because the restriction of its handling property can't be satisfied the processing of big data quantity

Field programmable gate array (FPGA) device is the current programmable logic device (PLD) very widely of using, and is also referred to as Programmable ASIC.Advantage with a lot of other schemes is good as: construction cycle weak point, flexible design, developing instrument advanced person, real-time, support several operation systems, support advanced language programming, network transmission technology maturation etc.

In the prior art, realize the Video Motion Detection system at FPGA, use dedicated video process chip and FPGA to realize Digital Video Processing at a high speed, select for use SRAM as the external memory storage of video data, the motion monitoring system after the improvement has satisfied the needs that motion detection is handled.Adopt the design of FPGA realization system to improve the processing speed of system, have good dirigibility and adaptability simultaneously.But carry out because the device Video processing mainly also is placed in the special-purpose video frequency processing chip, be not applied to the characteristic of fpga chip parallel processing fully, only use the total system of FPGA to control.

Summary of the invention

At the deficiencies in the prior art, the invention provides a kind of background subtraction point-score target detection tracker and method based on FPGA, mainly be detection and the tracking at the moving object in the static scene, read AVI format video file by the SD card, to read video and convert multiple image to, be the processing that the video detection and tracking are carried out on the basis with the image, obtain moving target to be tracked, by the LTM display screen result who follows the tracks of is shown.

A kind of background subtraction point-score target detection tracker based on FPGA, bus clock reaches the DE3 development board, comprise data input module, motion detection block, target tracking module, Nios II soft-core processor module, data cache module and LTM display module, the customization assisted instruction.System has carried out process of frequency multiplication by phaselocked loop to Nios II soft-core processor, makes NIOSII soft-core processor core processing frequency can reach 266MHz, has improved the processing power of system greatly; Under the NIOS II soft-core processor, the FAT32 file system is transplanted, designed and Implemented the bottom layer driving IP kernel of SD card, stored video data in SD, and with its SDI as video frequency following system; Integrated DDR2 SDRAM Memory Controller in system, and the DDR2 memory bar that itself and DE3 development board are provided is connected, and by IP kernel is made amendment, realizes that the 256MB space is as system's storage environment of system's operation among the DDR2; According to Avalon bus interface standard development and Design LTM IP (Intellectual Property) nuclear, show the result who handles by the LTM display; Software section based on the video tracking algorithm of particle filter algorithm is carried out design realized, carried out background modeling by time averaging method and obtain background image, utilized the background subtraction point-score to obtain the parameter information of moving target, realized motion detection block; After obtaining the parameter information of moving target, as tracking characteristics, utilize particle filter algorithm to realize the tracking of target with the color of target area.

A kind of background subtraction point-score target detection tracker based on FPGA of the present invention comprises hardware components and software section, and wherein, hardware components comprises FPGA, SD card, LTM display screen, LED light, DDR2 SDRAM and two seven segment digital tubes; Comprise that also embedding among the FPGA is that the integrated video of standard detects the minimum system of following the tracks of with the AVLON bus under SOPC builder environment;

The output pin of described SD card connects the input pin of FPGA, and the LTM display screen is connected with the output pin of FPGA, and DDR2 SDRAM input pin is connected with the output pin of FPGA, and two seven segment digital tubes are connected with the output pin of FPGA respectively;

A kind of background subtraction point-score target detection tracker based on FPGA of the present invention mainly is under the FPGA platform, is core with NIOS II processor, creates video and detects the minimum system of following the tracks of, and the platform of operation is provided for ensuing software development application.

Under SOPC builder environment, be that the integrated video of standard detects the minimum system of following the tracks of with the AVLON bus, comprise following module: data input module, motion detection block, target tracking module, Nios II soft-core processor module, data cache module and LTM controller module.

The Data Source of system is the SD card, and storage AVI format video data utilize NIOS II Integrated Development Environment to dispose the SD card controller in the SD card, and SD is sticked into capable initialization, and the image data of every frame is read out from the SD card, handles again.Data input module of the present invention is to utilize NIOS II Integrated Development Environment to transplant the FAT32 file system, designed and Implemented 1 SD card memory controller, GPIO port under the employing SOPC builder is as SD card controller control signal, output comprises three semaphores for the SD card controller, be clock signal sd_clk, command signal sd_cmd, data-signal sd_dat, wherein according to the difference of SD card control signal, also different to the direction of the deration of signal and signal; By 3 input/output ports, three control signals are connected to minimum system under NIOS II Integrated Development Environment, above-mentioned SD card control signal is 1 bit pattern, therefore, being configured to sd_clk is 1 output mode, and sd_cmd is 1 two-way input and output mode, sd_dat is 1 two-way input and output mode, according to the control sequential of SD card read-write, finish video data stored information in the SD card is read, and result's storage.

Data input module is to utilize NIOS II Integrated Development Environment to transplant the FAT32 file system, has designed and Implemented 1 SD card memory controller, adopts GPIO port under the SOPC builder as SD card controller control signal,

Motion detection block of the present invention is to adopt the background subtraction point-score to obtain the parameter information of moving target, before realizing background modeling, adopts median filtering algorithm that image is carried out filtering and handles, and chooses the window of 3*3 size; Adopt time averaging method to carry out background modeling, choose preceding 5 frame background images, 5 two field picture additions are averaged then, obtain background image.Image after subtracting each other is carried out binary conversion treatment, obtain bianry image, behind the bianry image that obtains comparatively desirable moving target, will determine size and the coordinate of moving object, so that the realization of target following.

Target tracking module of the present invention as tracking characteristics, utilizes particle filter algorithm to realize the tracking of target with the color of target area.At first pending image is carried out RGB to the color space conversion of HSV, then tracing area is carried out the particle initialization, all will carry out the transfer of particle in every two field picture after the particle initialization, the new state of particle calculates with second order autoregression dynamic model in the native system.Calculate the relevant range histogram, obtain following the tracks of particle.The weights that then carry out particle calculate and normalization, and the resampling of particle obtains the zone of particle representative, just the target area of Gen Zonging.

Nios II soft-core processor module of the present invention is mainly finished overall core control, the coordinative role of each module.Be responsible for reading the image data in the SD card, in software, view data carried out motion detection and target following processing then, call LTM and show IP kernel, the view data after handling is presented on the LTM liquid crystal display.

Data cache module of the present invention: realize that the 256MB space is as system's storage environment of system's operation among the DDR2;

LTM display module of the present invention: according to Avalon bus interface standard development and Design LTM display IP (Intellectual Property) nuclear, show the result who handles by the LTM display.

A kind of input, processing and three parts of demonstration of mainly carrying out video image based on the background subtraction point-score target detection tracker of FPGA of the present invention.The input of view data is view data to be transferred among the DDR2 SDRAM DIMM handle from the SD card, the data peak transfer rate of SD card is 25Mbits/s, and the size of each frame true color image is 320*240*3*8bit=1843200bit, and reading the required time of a frame image data so is 0.073728s.What adopted the processing section is the DE3 development board of altera corp, and requiring to handle every frame sign is the image of 320*240, and processing speed approximately can reach 0.05s, and namely per second is handled 20 frames.Target tracking algorism adopts on the basis of particle filter theory, is the track algorithm that feature realizes with the color of object.Under the bigger situation of moving target, can effectively follow the tracks of moving target, have good accuracy and robustness, the accuracy of tracking can reach more than 90% substantially.For Video processing, requirement can show timely that can not cause losing of data, display requirement is than higher after each frame picture was finished dealing with.

Adopt said system to carry out the method that target detection is followed the tracks of, specifically carry out as follows:

Step 1:SD card initialization;

After system powers on, before SD is sticked into capable read operation, need stick into capable initialization to SD, namely obtain the information such as type, capacity of SD card.

Step 1.1: the SD card that resets makes the SD card enter mode bus;

Step 1.2: send CMD55 order and ACMD41 order;

The effect of CMD55 is notice SD card, and next command is the ACMD41 order, and just program is specifically instructed, the SD card power on finish enter ready state after, just do not send this two orders;

After step 1.3:SD card is ready, send CMD2 and CMD3 order, request SD card returns confirmation and corresponding self address;

Step 1.4: send CMD9 and CMD10 order, read CID register and the CSD register of SD card;

Step 1.5: transmission CMD7 order arranges the SD card and enters programming state, prepares the transmission of data;

Step 1.6: send the CMD16 order, the size of data transmission module is set, finish the initialization of SD card.

Step 2: read SD card data;

Filename according to appointment reads corresponding file from the SD card, be through three steps: open file, file reads and close file: first reading matter reason sector 0, and obtaining boot sector is the offset address of logic sector 0; Read the content in the boot sector again, obtain the basic configuration information of file system, and calculate its real address and the size of FAT, FDT, aggregate of data thus; According to the filename that will read search FDT table, obtain starting cluster data number and the size of this document then, in the FAT table, search aggregate of data that all this document take and the sequencing of aggregate of data visit by the numbering of starting cluster data; Read the starting cluster content of this document and the content of following cluster according to this precedence table, finish until the data read of whole file; Read finish after direct close file;

CMD17 reads every cluster data by data monolithic reading order, and when this order was received in the SD clamping, the SD card was inquired about the data that will read with regard to the address of data block, carries out reading of data then;

Step 3: adopt the background subtraction point-score that the data that read are carried out motion target detection;

The present invention adopts the background subtraction point-score to realize that the moving target video detects: chose 3～5 two field pictures and carry out background modeling before moving target occurs, draw background image; After moving target occurs fully, subtract each other with background image then, obtain the information parameter of moving target, specifically carry out as follows:

Step 3.1: pre-service;

Adopt rule of three to convert coloured image to gray-scale map, conversion formula as the formula (1), wherein, what coloured image adopted is the rgb color pattern, R, G, B represent the red, green, blue color respectively, I represents half-tone information,

I＝0.114×B+0.587×G+0.299×R (1)

After the conversion, adopt median filtering algorithm to handle, medium filtering be intermediate value with window as result, the step of realization is as follows:

(1) size of selected window;

(2) gray values of pixel points in the window is sorted;

Substitute in the window other data with the intermediate value after the ordering;

Step 3.2: background modeling

Adopt the average background model in the background subtraction point-score to realize target detection, average modeling basic thought is that threshold value is cut apart, utilize threshold value to divide the pixel value of difference front and back image, system reads continuous several two field pictures in the video sequence, the mean value that calculates these several two field pictures is image as a setting, and subsequent frame and background frames are done the difference computing; If difference, just judges that this pixel belongs to the motion object greater than this threshold value, otherwise just judge that this pixel is the image that belongs to background;

Step 3.3: data are carried out binaryzation, obtain the bianry image of moving target;

Step 3.4: adopt the array localization method in bianry image, determine size and the coordinate of moving target;

Judge the information of width and horizontal ordinate, at first set the one-dimension array size, its array size is identical with the width value of image, seeks the value of the pixel of each row all over, when the pixel of a certain row is 0 number when adding 1, in the one-dimension array therewith the number of row correspondence add 1; Set a threshold value, when the value in the array during less than this threshold value, the value zero clearing in the array can reduce the interference of other information in the image like this; After one two field picture has been sought all over, determine the value of width and horizontal ordinate according to the non-vanishing pixel in the one-dimension array: this starting point is the value of horizontal ordinate to first of the value that size is non-vanishing of the N continuous in the judgement array as starting point, and N is the width of target.

The height of target and the value of ordinate are also obtained by above method: set the one-dimension array size, seek the value of the pixel of each row all over, value in the array is the increase of 0 number and increasing one by one along with the pixel in the corresponding line, after finishing less than several zero clearings of threshold value; After image has been sought all over, judge according to the value in the array that when the non-vanishing value of continuous Y, Y is the height of target, the position in the non-vanishing some place array of first among the Y is as the ordinate of target.

Step 4: after detecting moving target, the size and location of return movement target namely obtain tracing area;

Step 5: adopt particle filter algorithm that moving target is followed the tracks of;

The pending image of one frame is carried out the conversion of color space, then tracing area is carried out the particle initialization, calculate the relevant range histogram, obtain following the tracks of particle.Then carry out weight calculation and the normalization of particle, the resampling of particle obtains the zone of particle representative, just the target area of Gen Zonging.The process of whole tracking is exactly repeatedly the process of iteration particle filter, obtains the weights of particle collection each time, and weights are carried out normalization, and the tracking that obtains a maximum is estimated, thereby realization is to the tracing process of interesting target.

Step 5.1: color space conversion;

Since the rgb color model can not point-device expression pixel colouring information, when especially needing the color similarity degree in the different moment of calculating pixel point, therefore the present invention adopts linear color model HSV model to calculate, follow the tracks of as proper vector with the target object color, be easy to realize the tracking of moving target.The target video of system is the rgb color model, thereby the rgb color model conversion need be become the HSV colour model, changes according to following formula:

r＝R/255.0，g＝G/255.0，b＝B/255.0

$H=\left\{\begin{array}{c}(g-b)/[\max (r,g,b)-\min (r,g,b)];r=\max (r,g,b)\\ 2+(b-r)/[\max (r,g,b)-\min (r,g,b)];g=\max (r,g,b)\\ 4+(r-g)/[\max (r,g,b)-\min (r,g,b)];b=\max (r,g,b)\end{array}\right.---\left(2\right)$

When H＞0, H=H * 60,

When H＜0, H=H+360

$S=\frac{\max (r,g,b)-\min (r,g,b)}{\max (r,g,b)}---\left(3\right)$

V＝max(r，g，b) (4)

Wherein R, G, B represent red, green, blue respectively, and r, g, b represent R, G respectively, the B value is projected in the value in the 0.0-1.0 scope.H, S, V represent 3 parameters of HSV color space, and namely H represents form and aspect, and S represents saturation degree, and V represents brightness.

Step 5.2: initialization distribution of particles;

Particle is a point in the tracing area, and N particle constitutes a particle collection.The motion state of tracing area is described by this particle collection, the more big description of N value more accurate.Each particle is made up of attribute as shown in the table.

Property Name	Attribute type	Represent meaning
			x	int	The horizontal ordinate of current time particle
y	int	The ordinate of current time particle
			xp	int	The horizontal ordinate of previous moment particle
yp	int	The ordinate of previous moment particle
			x0	int	Initial particle horizontal ordinate
y0	int	Initial particle ordinate
			width	Int	The width of tracing area
height	int	The height of tracing area
			histo	Pointer	The tracing area histogram is described
w	float	Weights

The particle initialization is exactly to come the stochastic distribution particle in selected zone, and the present invention determines according to formula (5):

$\left\{\begin{array}{c}\mathrm{particle}\left[i\right].x0=\mathrm{particle}\left[i\right].\mathrm{xp}=\mathrm{particle}\left[i\right].x=\mathrm{regions}.x+\mathrm{regions}.\mathrm{width}/2\\ \mathrm{particle}\left[i\right].\mathrm{<figure-callout\; id="0"\; label="y"\; filenames="DEST\_PATH\_HDA00001674655600012.png,DEST\_PATH\_HDA00001674655600013.png"\; state="\{\{state\}\}">y</figure-callout>}0=\mathrm{particle}\left[i\right].\mathrm{yp}=\mathrm{particle}\left[i\right].y=\mathrm{regions}.y+\mathrm{regions}.\mathrm{height}/2\\ \mathrm{particle}\left[i\right].s=\mathrm{particle}\left[i\right].\mathrm{sp}=1\\ \mathrm{particle}\left[i\right].\mathrm{width}=\mathrm{regions}.\mathrm{width}\\ \mathrm{particle}\left[i\right].\mathrm{height}=\mathrm{regions}.\mathrm{height}\\ \mathrm{particle}\left[i\right].w=0\end{array}\right.---\left(5\right)$

In the formula, particle[i] description of i particle state of expression.Regions is for describing the structure of tracing area size and coordinate.How much relevant the tracking effect quality is with population, and in general, in certain scope, population is more many, and the effect of tracking is more good.The Monte Carlo method explanation is when sampling population approach infinity, and sample average is close to expectation.But under actual conditions, after population is increased to certain value, continues to increase number of particles and can not improve the accuracy of tracking, can increase operand because of increasing of population on the contrary.

Step 5.3: particle transfer;

Particle after adopting second order autoregression dynamic model to initialization carries out the calculating of state transitions.Particle position after obtaining shifting by metastasis model.

The particle p of the function input parameter of particle transfer for shifting, the width of image and height; Function returns the particle after the transfer.Shown in formula

$\left\{\begin{array}{c}{x}_p=x_{p-1}+v\_x_{p-1}\\ y_p=y_{p-1}+v\_y_{p-1}\\ {\mathrm{vx}}_p=V\&times;\mathrm{random}\_1\\ {\mathrm{vy}}_p=V\&times;\mathrm{random}\_2\end{array}\right.---\left(6\right)$

Wherein, V is the speed base value, speed component all be with V as radix, multiply by the random number random in (1,1) then.x _pThe horizontal ordinate of expression prediction back target, y _pThe ordinate of expression prediction back target, vx _pThe expression p moment is along the speed of horizontal ordinate, vy _pThe expression p moment is along the speed of ordinate.x _P-1The horizontal ordinate, the v_x that represent a node _P-1The horizontal ordinate direction speed, the y that represent a last node _P-1The ordinate, the v_y that represent a node _P-1Represent that a last node y direction, random_1 and random_2 are the number between 0 to 1 that system produces.

What this state transition equation was asked is exactly the particle predicted position coordinate of p this moment, this position equals particle p-1 coordinate constantly and adds p-1 speed constantly, this velocity is radix with the speed base value, needs to set the base value with the tracking target speeds match in actual applications.

Step 5.4: calculate the particle weights;

Adopt Pasteur's coefficient (BC, Bhattacharyya Coefficient) to weigh the similarity of the color probability distribution of reference template (being tracing area) and candidate template (being the estimation range), further calculate the weights size of particle.Centered by the particle position that the estimation range refers to predict, be the zone of size with the tracing area border,

In same field of definition X, the BC value of probability distribution p and q is calculated as follows:

$\mathrm{BC}(p,q)=\underset{x\&Element;X}{\mathrm{\&Sigma;}}\sqrt{p\left(x\right)q\left(x\right)}---\left(7\right)$

The scope of BC is [0,1], and more big both similaritys of expression of value are more little, and namely when BC=1, the color histogram in two zones distributes different fully, and when BC=0, the color histogram in two zones distributes identical.The calculating of the weights that this method utilization calculating Pasteur coefficient is realized is to describe histogrammic similarity.Hypothesis weights is W, and computing formula is:

W＝1-BC (8)

Carry out similarity relatively according to the color histogram of the big young pathbreaker estimation range of weights and the color histogram of tracing area.The span of weights W is [0,1], and weights W is more big, and the expression similarity is more big, can think that when W gets 1 the estimation range just is the zone of following the tracks of.The more little similarity of weights W is more little, thinks that namely the estimation range is not tracing area.

The similarity according to calculating tracing area prediction and actual tracing area of particle weights is given weights to particle.In follow the tracks of estimating, utilize the normalized weighting particle of weights collection to estimate this moment tracing area, think that the estimation range of weights maximum is exactly tracing area.

When calculating the particle weights after the particle transfer, centered by this particle, draw one and tracing area shape zone of the same size, obtain this regional hsv color histogram then, this color histogram is compared with original tracing area color histogram, obtain Pasteur's coefficient B C value, thereby obtain the weights of particle.

Step 5.5: particle weights normalizing;

The normalization of particle weights is to obtain the ratio that each particle of particle set accounts for whole particle collection accumulative total weights, and namely this particle represents the proportion of tracing area state in particle set, and formula is as follows:

$\mathrm{weight}\left[i\right]=\frac{w\left[i\right]}{\underset{1}{\overset{N}{\mathrm{\&Sigma;}}}w\left[i\right]}---\left(9\right)$

Wherein, w[i] be the weights of each particle,

Be whole particle collection accumulative total weights.The scope of particle i weights is [1, N].

After the normalization of all particles of whole particle set is calculated and finished, carry out size and sort, proportion more big with regard to other particle of representing the relative particle set of this particle more near the state position of real motion target, in tracking, play bigger effect.Then the particle unified integration of whole particle collection is got up to sort, choose the proportion maximum estimation of tracking target the most.

Step 5.6: resample and the particle renewal;

In order to solve the particle degenerate problem, system adopts threshold method, and some degeneration particle is resampled.The degeneration of particle is owing to carried out iterative computation repeatedly, and the validity that some particle is described tracing area worse and worse.Particle after the degeneration not only can not be correct the pursuit movement zone, can take a large amount of operation time on the contrary.

In the algorithm, the default threshold value is considered as the particle of degenerating to weights less than the particle of threshold value, replaces the degeneration particle with the particle of a weights maximum.That is to say that all values of information of degeneration particle all are assigned the information of the particle of weights maximum.

The position that every two field picture tracking obtains is as central point, and draw one and tracing area shape zone of the same size are with the field color model after the replacement of the hsv color histogram in this zone initialization, as new target area model.After each two field picture is handled, all to carry out the renewal of primary particle model, with the correctness that guarantees to follow the tracks of.

Step 6: return step 2, continue to read the next frame view data.

Beneficial effect: system of the present invention adopts the FPGA platform to design and realizes, for video image is handled, FPGA compares with the dsp system, the ARM microcontroller that rely on serial command to finish the respective image Processing Algorithm, has very strong concurrency, can further improve arithmetic speed; Simultaneously FPGA has inherited the hardware configuration of ASIC circuit in the image processing method case, shortcomings such as the hardware reconstruct that has overcome the ASIC design is poor, expensive, design cycle length, underaction.The video based on FPGA that the present invention proposes detects tracker, on platform selecting, have than PC and have littler power consumption and volume, has the processing power of better concurrency and Geng Gao than other embedded processing platforms, compare with the special circuit implementation, have bigger design flexibility in the design.The inventive method has better tracking effect and adaptability, and selected particle filter algorithm itself is applicable to non-linear non-Gauss's dynamic model, and more near real scene, adaptability is stronger, and the tracking accuracy rate is better.

Description of drawings

Fig. 1 is embodiment of the invention system architecture diagram;

Fig. 2 is the circuit catenation principle figure of embodiment of the invention SD card and FPGA;

Fig. 3 is embodiment of the invention GPIO and FPGA pin catenation principle figure;

Fig. 4 is the circuit catenation principle figure of embodiment of the invention LED light and FPGA;

Fig. 5 is the circuit catenation principle figure of embodiment of the invention DDR2 STRAM and FPGA;

Fig. 6 is the circuit catenation principle figure of the embodiment of the invention two seven segment digital tubes HEX0, HEX1 and FPGA;

Fig. 7 is that embodiment of the invention LTM controller IP master port streamline is read transmission time sequence figure;

Fig. 8 is that the embodiment of the invention is based on the LTM IP interface signal distribution of Avalon bus;

Fig. 9 is the system architecture diagram of embodiment of the invention LTM controller IP;

Figure 10 is embodiment of the invention SOPC system integration structural drawing;

Figure 11 is embodiment of the invention SD card initialization process flow diagram;

Figure 12 is embodiment of the invention SD card monolithic data read process flow diagram;

Figure 13 is embodiment of the invention moving target video detection algorithm process flow diagram;

Figure 14 is embodiment of the invention background subtraction point-score process flow diagram;

Figure 15 is size and the coordinate process flow diagram that the embodiment of the invention is obtained moving target;

Figure 16 is embodiment of the invention target tracking algorism process flow diagram.

Embodiment

Below in conjunction with accompanying drawing concrete enforcement of the present invention is elaborated.

The background subtraction point-score target detection tracker that the present invention is based on FPGA realizes at the DE3 development board of the FPGA of altera corp development board, the DE3 development board carries the Stratix III series EP3SL150F1152C2N chip of ALTERA company, and system of the present invention comprises hardware components and software section.Hardware components comprises FPGA (EP3SL150F1152C2N), SD card, LTM display screen, LED light, DDR2 SDRAM and two seven segment digital tubes, and structure as shown in Figure 1.

Total system is implemented under the hardware and software environment:

(1) PC platform: Intel (R) Core (TM) i5 CPU 2.8GHz dominant frequency, 4 nuclears, 4 threads, internal memory 2.93GHz;

(2) display: the TRDB-LTM of friendly brilliant company, the image that can show size is 800*480;

(3)Quartus：Quartus II 9.1(32-Bit)

(4)NIOS IDE：NIOS II 9.1 IDE；

(5) test file: self-control video test.avi.

The SD card is selected 256MB for use, it is connected as shown in Figure 2 with the pin of FPGA, and specifically: the SD_CMD pin links to each other with the R10 pin of FPGA, and the SD_CLK pin links to each other with the P8 pin of FPGA, the SD_DAT0 pin links to each other with the P7 pin of FPGA, and the SD_WPn pin links to each other with the N5 pin of FPGA.

The LTM display screen is connected with the GPIO draw-in groove by winding displacement, the GPIO draw-in groove is connected with the FPGA pin by circuit, use GPIO0 LTM interface directly to connect 0 group of GPIO, as shown in Figure 3, the concrete connection is: GPIO0_CLKINn0 links to each other with the PIN_AE32 pin, GPIO0_D[0]～GPIO0_D[31] respectively with PIN_AL33, PIN_AL32, PIN_AL34, PIN_AH31, PIN_AM34, PIN_AH30, PIN_AK34, PIN_AJ32, PIN_AK33, PIN_AJ31, PIN_AH34, PIN_AG32, PIN_AJ34, PIN_AG31, PIN_AF32, PIN_AF31, PIN_AP21, PIN_AG34, PIN_AN21, PIN_AH33, PIN_AL21, PIN_AP22, PIN_AK21, PIN_AN22, PIN_AP25, PIN_AP24, PIN_AN25, PIN_AN24, PIN_AM23, PIN_AG21, PIN_AL23, the PIN_AF21 pin links to each other, GPIO0_CLKINp0 links to each other with the PIN_AE31 pin, GPIO0_CLKOUTnO links to each other with the PIN_AE19 pin, and GPIO0_CLKOUTp0 links to each other with the PIN_AD19 pin.

LED light is connected as shown in Figure 4 with the pin of FPGA, specifically: LEDB[0]～LEDB[7] respectively with the PIN_AB5 of FPGA, PIN_AB6, PIN_AA6, PIN_AA7, PIN_Y7, PIN_Y8, PIN_Y9, the PIN_Y10 pin links to each other, LEDG[0]～LEDG[7] respectively with the PIN_AB1 of FPGA, PIN_AA1, PIN_Y1, PIN_Y2, PIN_Y3, PIN_W3, PIN_AA4, the PIN_Y4 pin links to each other, LEDR[0]～LEDR[7] respectively with the PIN_AD1 of FPGA, PIN_AC1, PIN_AC2, PIN_AB2, PIN_AC4, PIN_AB4, PIN_AA3, the PIN_AB3 pin links to each other.

DDR2STRAM selects transcend 51454-14583 for use, be connected as shown in Figure 5 with the pin of FPGA, specifically: DDR2_DQ[0]～DDR2_DQ[63] respectively with PIN_AM1, PIN_AM2, PIN_AL1, PIN_AL2, PIN_AE7, PIN_AE8, PIN_AC8, PIN_AC9, PIN_AG3, PIN_AG4, PIN_AJ15A, PIN_AH2, PIN_AE5, PIN_AE6, PIN_AD6, PIN_AD7, PIN_AF3 5A, PIN_AF4 5A, PIN_AF1 5A, PIN_AF2 5A, PIN_AC7 5A, PIN_AB8 5A, PIN_AC5 5A, PIN_AC6 5A, PIN_AJ15, PIN_AH15, PIN_AP12, PIN_AN12, PIN_AK15, PIN_AG15, PIN_AP13, PIN_AM12, PIN_AP10, PIN_AN10, PIN_AF14, PIN_AF13, PIN_AP11, PIN_AP9, PIN_AE14, PIN_AE13, PIN_AL12, PIN_AK12, PIN_AP7, PIN_AN7, PIN_AM11, PIN_AK10, PIN_AM7, PIN_AP6, PIN_AJ12, PIN_AH12, PIN_AL9, PIN_AK9, PIN_AJ13, PIN_AG12, PIN_AJ9, PIN_AL7, PIN_AP4, PIN_AN4, PIN_AM5, PIN_AL5, PIN_AP5, PIN_AP2, PIN_AM4, the PIN_AL4 pin links to each other, DDR2_DQS_n[0]～DDR2_DQS_n[7] respectively with PIN_AJ3, PIN_AK1, PIN_AH1, PIN_AJ14, PIN_AN9, PIN_AL11, PIN_AJ11, the PIN_AP3 pin links to each other, DDR2_DQS_p[0]～DDR2_DQS_p[7] respectively with PIN_AJ4, PIN_AJ2, PIN_AG1, PIN_AH14, PIN_AM9, PIN_AL10, PIN_AH11, the PIN_AN3 pin links to each other, DDR2_DM[0]～DDR2_DM[7] respectively with PIN_AF5, PIN_AB10, PIN_AB9, PIN_AN13, PIN_AF15, PIN_AP8, PIN_AL84A, the PIN_AN6 pin links to each other, DDR2_CLK_p[0], DDR2_CLK_p[1] respectively with PIN_AE4, the PIN_AE11 pin links to each other, DDR2_CLK_n[0], DDR2_CLK_n[1] respectively with PIN_AE35A, the PIN_AF11 pin links to each other, DDR2_CKE[0], DDR2_CKE[1] respectively with PIN_AF6, the PIN_AC11 pin links to each other, DDR2_A[0]～DDR2_A[15] respectively with PIN_AE10, PIN_AD12, PIN_AF10, PIN_AL14, PIN_AJ64A, PIN_AM14, PIN_AK6, PIN_AM6, PIN_AJ74A, PIN_AK7, PIN_AC12, PIN_AJ10, PIN_AM8, PIN_AL15, PIN_AE15, the PIN_AA10 pin links to each other, DDR2_BA[0], DDR2_BA[1] respectively with PIN_AM17, the PIN_AE12 pin links to each other, DDR2_RAS_n links to each other with the PIN_AL13 pin, DDR2_WE_n links to each other with the PIN_AL17 pin, DDR2_CS_n[0] link to each other with the PIN_AM16 pin, DDR2_CAS_n links to each other with the PIN_AD13 pin, DDR2_ODT[0] link to each other with the PIN_AM15 pin, DDR2_CS_n[1] link to each other with the PIN_AL16 pin, DDR2_ODT[1] link to each other with the PIN_AJ16 pin, DDR2_SDA links to each other with the PIN_T9 pin, DDR2_SC links to each other with the PIN_T8 pin, DDR2_SA[0] link to each other DDR2_SA[1 with the PIN_U6 pin] link to each other with the PIN_T7 pin.

Two seven segment digital tubes HEX0, HEX1 are connected as shown in Figure 6 with the pin of FPGA, and specifically: HEX0_D0～HEX0_D6 of HEX0 links to each other with Y11, W12, Y5, Y6, W7, W8, the W10 pin of FPGA respectively, and HEX0_DP links to each other with the V3 pin of FPGA; HEX1_D0～HEX1_D6 of HEX1 links to each other with P3, N4, N3, N1, M1, L1, the L2 pin of FPGA respectively, and HEX1_DP links to each other with the V4 pin of FPGA.

A kind of motion target tracking system based on FPGA of the present invention mainly is under the FPGA platform, is core with NIOS II processor, creates video and detects the minimum system of following the tracks of, and the platform of operation is provided for ensuing software development application.Under SOPC builder environment, the Hardware I P nuclear of the hardware platform of minimum system is connected, comprise in the compiler of self-defined LTM controller IP kernel and selection and carry IP kernel, and connected mode, IP kernel plot, interrupt levels arranged, generate SOPC (SOC (system on a chip)), under Quartus II environment, carry out instantiation then, generate the hardware platform configuration file, embed FPGA.

Video of the present invention detects the minimum system of following the tracks of, and comprises following module: data input module, motion detection block, target tracking module, Nios II soft-core processor module, data cache module and LTM controller module.

Data input module of the present invention is to utilize NIOS II Integrated Development Environment to transplant the FAT32 file system, designed and Implemented 4 SD card memory controller, GPIO port under the employing SOPC builder is as SD card controller control signal SD_cmd, SD_clk, SD_clk, according to the control sequential of SD card read-write, finish the reading of video data stored high speed information in the SD card of Kingston company, and result's storage.The Data Source of system is the SD card, and storage AVI format video data utilize NIOS II Integrated Development Environment to dispose the SD card controller in the SD card, and SD is sticked into capable initialization, and the image data of every frame is read out from the SD card, handles again.Comprise three semaphores for SD card controller output, i.e. clock signal sd_clk, command signal sd_cmd, data-signal sd_dat is wherein according to the difference of SD card control signal, also different to the direction of the deration of signal and signal; By 3 input/output ports, three control signals are connected to minimum system under NIOS II Integrated Development Environment, above-mentioned SD card control signal is 1 bit pattern, therefore, being configured to sd_clk is 1 output mode, sd_cmd is 1 two-way input and output mode, and sd_dat is 1 two-way input and output mode.

Motion detection block of the present invention is to adopt the background subtraction point-score to obtain the parameter information of moving target, before realizing background modeling, adopts median filtering algorithm that image is carried out filtering and handles, and chooses the window of 3*3 size; Adopt time averaging method to carry out background modeling, choose preceding 5 frame background images, 5 two field picture additions are averaged then, obtain background image.Image after subtracting each other is carried out binary conversion treatment, obtain bianry image, behind the bianry image that obtains comparatively desirable moving target, will determine size and the coordinate of moving object, so that the realization of target following.

Target tracking module of the present invention as tracking characteristics, utilizes particle filter algorithm to realize the tracking of target with the color of target area.At first pending image is carried out RGB to the color space conversion of HSV, then tracing area is carried out the particle initialization, all will carry out the transfer of particle in every two field picture after the particle initialization, the new state of particle calculates with second order autoregression dynamic model in the native system.Calculate the relevant range histogram, obtain following the tracks of particle.The weights that then carry out particle calculate and normalization, and the resampling of particle obtains the zone of particle representative, just the target area of Gen Zonging.

Nios II soft-core processor module of the present invention is mainly finished overall core control, the coordinative role of each module.Be responsible for reading the image data in the SD card, in software, view data carried out motion detection and target following processing then, call LTM and show IP kernel, the view data after handling is presented on the LTM liquid crystal display.

Data cache module of the present invention: realize that the 256MB space is as system's storage environment of system's operation among the DDR2;

LTM display module of the present invention: according to Avalon bus interface standard development and Design LTM IP (Intellectual Property) nuclear, show the result who handles by the LTM display.

A kind of input, processing and three parts of demonstration of mainly carrying out video image based on the background subtraction point-score target detection tracker of FPGA of the present invention.The input of view data is view data to be transferred among the DDR2 SDRAM DIMM handle from the SD card, the data peak transfer rate of SD card is 25Mbits/s, and the size of each frame true color image is 320*240*3*8bit=1843200bit, and reading the required time of a frame image data so is 0.073728s.What adopted the processing section is the DE3 development board of altera corp, and requiring to handle every frame sign is the image of 320*240, and processing speed approximately can reach 0.05s, and namely per second is handled 20 frames.Target tracking algorism adopts on the basis of particle filter theory, is the track algorithm that feature realizes with the color of object.Under the bigger situation of moving target, can effectively follow the tracks of moving target, have good accuracy and robustness, the accuracy of tracking can reach more than 90% substantially.For Video processing, requirement can show timely that can not cause losing of data, display requirement is than higher after each frame picture was finished dealing with.

What FPGA was embedded is that the hardware of minimum system of core is integrated, specific as follows with NIOS II soft-core processor:

1, LTM controller IP kernel

LTM controller IP kernel is configured, uses Avalon-MM streamline transmission mode that LTM controller IP kernel is mounted on the Avalon bus, LTM controller IP kernel is integrated among the FPGA.The signal that Avalon-MM streamline transmission mode uses comprises: clk, read, readdata, flush, waitrequest, readdatavalid.

Master port (Master Interface) is provided LTM controller IP kernel and from port (Slave Interface), Fig. 7 reads transmission time sequence for the master port streamline.Execution sequence is as follows:

(1) Master was open to the custom and sends address 1 signal and read signal and draw high expression Master and begin to read transmission.The waitrequest signal is drawn high, and the Master port is in waiting status, and read signal must be waited for the next clock period.

(2) the Avalon bus drags down and receives the read signal command with the waitrequest signal.

(3) the Avalon bus receives second read order.Simultaneously, bus is sent the read data useful signal, and expression Master can read the data 1 in the address 1.

(4) the Avalon bus receives the 3rd read order, at this moment, had data 2 and data 3 wait to be transmitted.

(5) read data is effective, and Master catches reading data signal, reads the data in the address 2.

(6) read data is invalid, and Master can not read data.Master draws high the flush signal, and notice bus read data is effective, and Master catches reading data signal, reads data 4.

According to Altera Avalon-MM (Avalon Memory Mapped Interface) interface specification design LTM controller IP kernel, the fundamental purpose of this IP kernel is for the control that allows LTM can break away from CPU, can receive and reading video data.Use DDR2 SDRAM as a FIFO, for the realization of algorithm provides bigger spatial cache.Fig. 8 is that the LTM IP interface signal of Avalon bus distributes.

The interface signal of LTM controller IP kernel can be divided into:

(1) clock input signal and reset signal, the 50MHz clock of directly being imported by the outside.

(2) port signal comprises master port and from the input and output signal of port.

(3) input and output signal of LTM is used for configuration LTM display.

This IP kernel is provided with master port and from port, system uses master port to be used for reading data from DDR2 SDRAM.

Fig. 9 is the system architecture diagram of LTM controller IP, and LTM controller IP kernel basis that the present invention provides in the brilliant company of friend has redesigned the steering logic in the IP kernel and added the doubleclocking fifo module.LCD sequence generation module and I2C configuration module still adopt existing routine techniques means in the IP kernel, and being achieved as follows of Avalon Bus Master steering logic and doubleclocking FIFO:

Avalon Bus Master is responsible for the instruction according to control module, reads the data among the SDRAM, and is written among the FIFO, and its core is address accumulator.As FIFO when being empty, it is that unit begins to add up and reads the address of SDRAM for generation that address accumulator begins under the master port bus clock with 4.After running through the data of a frame, automatically reset to first address, continue to add up.The main equipment that host device interface adopts band to postpone is read transmission mode, and under this transmission mode, even without receiving last valid data, main equipment also can be initiated read command next time.When waitrequest invalidating signal (low level), the initiation read command that main equipment can be continuous, when waitrequest signal effectively when (high level), main equipment begins to wait for, becomes low level up to it.When readdatavalid signal effectively when (high level), the expression read data is effective, the valid data of main equipment this moment on can the latch data mouth.Here do not use the flush signal, the flush signal can be removed all uncompleted read commands of front.The Avalon bus guarantees the output order of data and the sequence consensus of main equipment requirement (namely with main equipment address output sequence consensus).The readdatavalid signal can directly be written to the data that read out among the FIFO like this as the wrreq signal of FIFO.When the current address equaled the tail address, the totalizer that then resets made it to restart to add up from first address.

FIFO writes data by Avalon Bus Master, and writing clock is the master port bus clock; By the sequential generation module sense data of lcd controller, readout clock is 33MHz.Independently writing clock and reading under the clock effect, FIFO can provide rdusedw[11:0] signal, be used to indicate and made the capacity of using up among the FIFO.System can arrange a upper limit and a lower limit, and the data volume in FIFO is higher than the upper limit or is lower than down in limited time, suspends or the reading of the internal memory of beginning Avalon Bus Master, in order to guarantee system performance.

The FIFO macroblock that the FIFO design adopts Quartus II to carry generates needed module automatically, for calling.Parameter arranges as follows:

parameter byteenablewidth＝4，

parameter length＝800*480，

parameter fifo_numberwords＝8192，

parameter fifo_depth＝13，

parameter fifo_value＝195。

2, the selection of IP kernel and setting in the compiler, specific as follows:

System uses the integrated IP kernel of translation and compiling environment to comprise: NIOS II soft-core processor, the DDR2 controller, the Onchip_mem controller, the Jtag_uart controller, the Sys_id controller, Seg7 seven segment numerical tube controller, Pio_led controller, Performance_counter and Clock crossing bridge controller.

According to demand, needs are revised the IP kernel of default setting and are reset, comprise as the lower part:

(1) NIOS II soft-core processor: option is set at Parameter Settings, Core Nios II arranges in the option, select fastest NIOS II/f to handle soft nuclear, has the support hardware multiplication, hardware division, branch prediction, the dynamic branch predictor performance has the processing speed up to 302DMIPS.Selecting address among the altmemddr at Reset Vetor is that the space of 0x08000000 is as the reseting vector start address.The address is the free air anomaly vector start address of 0x08000020 among the selection altmemddr in Exception Vetor drop-down menu.

In Caches and Memory Interfaces option, with data cache and command cache, be set to 64KB, by increasing the size of cache, increase the probability that data and instruction are hit in buffer memory.

(2) Onchip_Mem controller: memory module on the sheet, because memory module is except register on the sheet, handle the fastest storer, native system has all used memory module on the sheet, so with its size configure 240KB, in order to buffer memory on the sheet to be provided.In order to improve processing speed, its read latch is set to minimum, i.e. 1 clock delay.

(3) DDR2 SDRAM High Performance Controller (DDR2 controller): by Modify parameters option, DDR2 controller IP kernel is arranged, comprise dominant frequency, interface width, type of memory, largest access frequency, the row address line width, column address conductor width and CAS time-delay are after parameter configuration, in Memory Settings tab, can read parameter configuration, data access speed is configured to pattern at full speed.Finally, the DDR2 IP kernel is configured to have the clock of 266.667MHZ, the 256M size, the 64bit width, the storer of Unbuffered DIMM type.

Phase-locked loop pll: the PLL by DDR2 controller IP kernel is configured, and configures 3 clock frequencies, is respectively sysclk 266.666MHz, auxfull 266.666MHz, auxhalf 266.667MHz, pll_c0100MHz.

(4) Seg7 seven segment numerical tube controller: the platform of employing has 2 seven segment numerical display modules, thus native system to its configuration in, charactron quantity is 2, the address width of each is 3, is defaulted as low-voltage and enables.

(5) Performance_counter: Number of simultaneously-measured sections is set to maximum 7, once can test the runnability of 7 different code sections, performance comprises execution number of times, execution cycle number, execution time, whether this code segment is carried out.

(6) Pio_led controller: the development board that system uses has 3 groups, and 8 every group, amount to 24 led lamps, it is arranged to output mode, width is 24.

(7) Jtag_uart: be used for hardware system file * .sof, software program is downloaded, and the on-line debugging of NIOSII translation and compiling environment.

3, the SOPC system integration

Configure under SOPC builder 9.1 environment IP kernel and according to demand behind the self-defining IP kernel, under SOPC builder9.1 environment, carry out integrated to hardware platform, comprise the interpolation of IP kernel, data-interface, the link of instruction interface, the distribution of IP kernel base address, the selection of clock.

Framework after hardware platform is integrated, as shown in figure 10, the specific implementation that connects based on the signal of AVALON bus is:

(1) be that the module that Instruction_master is connected has with the instruction interface of NIOSII soft-core processor: for the treatment of the Jtag_uart of device debugging, be used for the DDR2 controller module of CPU processing procedure buffer memory;

(2) be that the module that Data_master is connected has with the data-interface of NIOSII soft-core processor: the Onchip_mem that is used for buffer memory, be used for software and download the Jtag_uart of debugging, be used for providing clock period and Timer module at interval, be used for showing the Seg7 seven segment numerical tube controller of processing procedure, the Sy_sid module that is used for Mk system information, the Performance_counter module of performance test when being used for running software is used for the Sd_clk that the SD card drives, Sd_cmd, the Sd_dat semaphore.

The clock signal allocative decision based on the AVALON bus of system is as shown in table 1: adopt to divide a clock zone scheme, to relating to the routine processes module, the dominant frequency that is 266.667MHz of employing, and assistant adjustment module, employing be the dominant frequency of 50MHz.For SD card control signal, taked speed 100MHz faster.

Table 1 system clock allocation table

The base address of each IP kernel module of system is distributed, and takes automatic allocative decision, concrete base address as shown in Figure 11, each base address represents the entry address of IP kernel.Because adopt integrated distribution instruction among the SOPC builder9.1, so address distribution method is not described here.

Adopt said system to carry out the method that moving object detection is followed the tracks of, specifically carry out as follows:

Step 1:SD card initialization;

As shown in figure 11, after system powers on, before SD is sticked into capable read operation, need stick into capable initialization to SD, namely obtain the information such as type, capacity of SD card, step is as follows:

Step 1.1: the SD card that resets makes the SD card enter mode bus;

Step 1.2: send CMD55 order and ACMD41 order;

The effect of CMD55 is notice SD card, and next command is the ACMD41 order, and just program is specifically instructed, the SD card power on finish enter ready state after, just do not send this two orders;

After step 1.3:SD card is ready, send CMD2 and CMD3 order, request SD card returns confirmation and corresponding self address;

Step 1.4: send CMD9 and CMD10 order, read CID register and the CSD register of SD card;

Step 1.5: transmission CMD7 order arranges the SD card and enters programming state, prepares the transmission of data;

Step 1.6: send the CMD16 order, the size of data transmission module is set, finish the initialization of SD card.

Step 2: read SD card data;

Filename according to appointment reads corresponding file from the SD card, through three steps: open file, file reads and close file: first reading matter reason sector 0, obtaining boot sector is the offset address of logic sector 0; Read the content in the boot sector again, obtain the basic configuration information of file system, and calculate its real address and the size of FAT, FDT, aggregate of data thus; According to the filename that will read search FDT table, obtain starting cluster data number and the size of this document then, in the FAT table, search aggregate of data that all this document take and the sequencing of aggregate of data visit by the numbering of starting cluster data; Read the starting cluster content of this document and the content of following cluster according to this precedence table, finish until the data read of whole file; Read finish after direct close file.

As shown in figure 12, CMD17 reads every cluster data by data monolithic reading order, and when this order was received in the SD clamping, the SD card was inquired about the data that will read with regard to the address of data block, carries out reading of data then, and the size of data block is 512 bytes.

Step 3: the data that read are carried out moving target video detection algorithm, realize that the video of moving target detects, flow process as shown in figure 13;

The present invention adopts the background subtraction point-score to realize that the moving target video detects: chose 3～5 two field pictures and carry out background modeling before moving target occurs, draw background image; After moving target occurs fully, carry out background and think subtraction then, obtain the information parameter of moving target.Specifically carry out as follows:

Step 3.1: pre-service;

Adopt rule of three to convert coloured image to gray-scale map, conversion formula as the formula (1), wherein, what coloured image adopted is the rgb color pattern, R, G, B represent the red, green, blue color respectively, I represents half-tone information,

I＝0.114×B+0.587×G+0.299×R (1)

After the conversion, adopt median filtering algorithm to handle, medium filtering be intermediate value with window as result, the step of realization is as follows:

(3) choose the window of 3 * 3 sizes;

(4) gray values of pixel points in the window is sorted;

Substitute in the window other data with the intermediate value after the ordering;

Step 3.2: background modeling

The present invention selects for use the average background model in the background subtraction point-score to realize target detection, as shown in figure 14, average modeling basic thought is that threshold value is cut apart, utilize threshold value to divide the pixel value of difference front and back image, system reads continuous several two field pictures in the video sequence, the mean value that calculates these several two field pictures is image as a setting, and subsequent frame and background frames are done the difference computing; If difference, just judges that this pixel belongs to the motion object greater than this threshold value, otherwise just judge that this pixel is the image that belongs to background.

Step 3.3: data are carried out binaryzation, obtain the bianry image of moving target;

Step 3.4: adopt the array localization method in bianry image, determine size and the coordinate of moving target, as shown in figure 15;

Judge the information of width and horizontal ordinate, at first the setting size is 320 one-dimension array, and its array size is identical with the width value of image, seeks the value of the pixel of each row all over, when the pixel of a certain row is 0 number when adding 1, the number that row are corresponding therewith in the one-dimension array adds 1; Set a threshold value, when the value in the array during less than this threshold value, the value zero clearing in the array can reduce the interference of other information in the image like this; After one two field picture has been sought all over, determine the value of width and horizontal ordinate according to the non-vanishing pixel in the one-dimension array: this starting point is the value of horizontal ordinate to first of the value that size is non-vanishing of the N continuous in the judgement array as starting point, and N is the width of target.

The height of target and the value of ordinate are also obtained by above method: set a size and be 240 one-dimension array, seek the value of the pixel of each row all over, value in the array is the increase of 0 number and increasing one by one along with the pixel in the corresponding line, after finishing less than several zero clearings of threshold value; After image has been sought all over, judge according to the value in the array that when the non-vanishing value of continuous Y, Y is the height of target, the position in the non-vanishing some place array of first among the Y is as the ordinate of target.

Step 4: after detecting moving target, the size and location of return movement target namely obtain tracing area;

Step 5: adopt particle filter algorithm that moving target is followed the tracks of, flow process as shown in figure 16;

The pending image of one frame is carried out the conversion of color space, then tracing area is carried out the particle initialization, calculate the relevant range histogram, obtain following the tracks of particle.Then carry out weight calculation and the normalization of particle, the resampling of particle obtains the zone of particle representative, just the target area of Gen Zonging.The process of whole tracking is exactly repeatedly the process of iteration particle filter, obtains the weights of particle collection each time, and weights are carried out normalization, and the tracking that obtains a maximum is estimated, thereby realization is to the tracing process of interesting target.

Step 5.1: color space conversion;

Adopt linear color model HSV model to calculate, follow the tracks of as proper vector with the target object color, be easy to realize the tracking of moving target.The target video of system is the rgb color model, thereby the rgb color model conversion need be become the HSV colour model, changes according to following formula:

r＝R/255.0，g＝G/255.0，b＝B/255.0

$H=\left\{\begin{array}{c}(g-b)/[\max (r,g,b)-\min (r,g,b)];r=\max (r,g,b)\\ 2+(b-r)/[\max (r,g,b)-\min (r,g,b)];g=\max (r,g,b)\\ 4+(r-g)/[\max (r,g,b)-\min (r,g,b)];b=\max (r,g,b)\end{array}\right.---\left(2\right)$

When H＞0, H=H * 60,

When H＜0, H=H+360

$S=\frac{\max (r,g,b)-\min (r,g,b)}{\max (r,g,b)}---\left(3\right)$

V＝max(r，g，b) (4)

Wherein R, G, B represent red, green, blue respectively, and r, g, b represent R, G respectively, the B value is projected in the value in the 0.0-1.0 scope.H, S, V represent 3 parameters of HSV color space, and namely H represents form and aspect, and S represents saturation degree, and V represents brightness.

Step 5.2: initialization distribution of particles;

Particle is a point in the tracing area, and N particle constitutes a particle collection.The motion state of tracing area is described by this particle collection, the more big description of N value more accurate.Each particle is made up of attribute as shown in the table.

Property Name	Attribute type	Represent meaning
			x	int	The horizontal ordinate of current time particle
y	int	The ordinate of current time particle
			xp	int	The horizontal ordinate of previous moment particle
yp	int	The ordinate of previous moment particle
			x0	int	Initial particle horizontal ordinate
y0	int	Initial particle ordinate
			width	Int	The width of tracing area
height	int	The height of tracing area
			histo	Pointer	The tracing area histogram is described
w	float	Weights

The particle initialization is exactly to come the stochastic distribution particle in selected zone, and the present invention determines according to formula (5):

$\left\{\begin{array}{c}\mathrm{particle}\left[i\right].x0=\mathrm{particle}\left[i\right].\mathrm{xp}=\mathrm{particle}\left[i\right].x=\mathrm{regions}.x+\mathrm{regions}.\mathrm{width}/2\\ \mathrm{particle}\left[i\right].\mathrm{<figure-callout\; id="0"\; label="y"\; filenames="DEST\_PATH\_HDA00001674655600012.png,DEST\_PATH\_HDA00001674655600013.png"\; state="\{\{state\}\}">y</figure-callout>}0=\mathrm{particle}\left[i\right].\mathrm{yp}=\mathrm{particle}\left[i\right].y=\mathrm{regions}.y+\mathrm{regions}.\mathrm{height}/2\\ \mathrm{particle}\left[i\right].s=\mathrm{particle}\left[i\right].\mathrm{sp}=1\\ \mathrm{particle}\left[i\right].\mathrm{width}=\mathrm{regions}.\mathrm{width}\\ \mathrm{particle}\left[i\right].\mathrm{height}=\mathrm{regions}.\mathrm{height}\\ \mathrm{particle}\left[i\right].w=0\end{array}\right.---\left(5\right)$

In the formula, particle[i] description of i particle state of expression.Regions is for describing the structure of tracing area size and coordinate.How much relevant the tracking effect quality is with population, and in general, in certain scope, population is more many, and the effect of tracking is more good.The Monte Carlo method explanation is when sampling population approach infinity, and sample average is close to expectation.But under actual conditions, after population is increased to certain value, continues to increase number of particles and can not improve the accuracy of tracking, can increase operand because of increasing of population on the contrary.

Step 5.3: particle transfer;

Particle after adopting second order autoregression dynamic model to initialization carries out the calculating of state transitions.Particle position after obtaining shifting by metastasis model.

The particle p of the function input parameter of particle transfer for shifting, the width of image and height; Function returns the particle after the transfer.Shown in formula

$\left\{\begin{array}{c}{x}_p=x_{p-1}+v\_x_{p-1}\\ y_p=y_{p-1}+v\_y_{p-1}\\ {\mathrm{vx}}_p=V\&times;\mathrm{random}\_1\\ {\mathrm{vy}}_p=V\&times;\mathrm{random}\_2\end{array}\right.---\left(6\right)$

Wherein, V is the speed base value, speed component all be with V as radix, multiply by the random number random in (1,1) then.x _pThe horizontal ordinate of expression prediction back target, y _pThe ordinate of expression prediction back target, vx _pThe expression p moment is along the speed of horizontal ordinate, vy _pThe expression p moment is along the speed of ordinate.x _P-1The horizontal ordinate, the v_x that represent a node _P-1The horizontal ordinate direction speed, the y that represent a last node _P-1The ordinate, the v_y that represent a node _P-1Represent that a last node y direction, random_1 and random_2 are the number between 0 to 1 that system produces

What this state transition equation was asked is exactly the particle predicted position coordinate of p this moment, this position equals particle p-1 coordinate constantly and adds p-1 speed constantly, this velocity is radix with the speed base value, needs to set the base value with the tracking target speeds match in actual applications.

Step 5.4: calculate the particle weights;

Adopt Pasteur's coefficient (BC, Bhattacharyya Coefficient) to weigh the similarity of the color probability distribution of reference template (being tracing area) and candidate template (being the estimation range), further calculate the weights size of particle.Centered by the particle position that the estimation range refers to predict, be the zone of size with the tracing area border,

In same field of definition X, the BC value of probability distribution p and q is calculated as follows:

$\mathrm{BC}(p,q)=\underset{x\&Element;X}{\mathrm{\&Sigma;}}\sqrt{p\left(x\right)q\left(x\right)}---\left(7\right)$

The scope of BC is [0,1], and more big both similaritys of expression of value are more little, and namely when BC=1, the color histogram in two zones distributes different fully, and when BC=0, the color histogram in two zones distributes identical.The calculating of the weights that this method utilization calculating Pasteur coefficient is realized is to describe histogrammic similarity.Hypothesis weights is W, and computing formula is:

W＝1-BC (8)

Carry out similarity relatively according to the color histogram of the big young pathbreaker estimation range of weights and the color histogram of tracing area.The span of weights W is [0,1], and weights W is more big, and the expression similarity is more big, and the estimation range is the zone of tracking when W gets 1.The more little similarity of weights W is more little, and namely the estimation range is not tracing area.

The similarity according to calculating tracing area prediction and actual tracing area of particle weights is given weights to particle.Utilize the normalized weighting particle of weights collection to estimate this moment tracing area in follow the tracks of estimating, namely the estimation range of weights maximum is exactly tracing area.

When calculating the particle weights after the particle transfer, centered by this particle, draw one and tracing area shape zone of the same size, obtain this regional hsv color histogram then, this color histogram is compared with original tracing area color histogram, obtain Pasteur's coefficient B C value, thereby obtain the weights of particle.

Step 5.5: particle weights normalizing;

The normalization of particle weights is to obtain the ratio that each particle of particle set accounts for whole particle collection accumulative total weights, and namely this particle represents the proportion of tracing area state in particle set, and formula is as follows:

$\mathrm{weight}\left[i\right]=\frac{w\left[i\right]}{\underset{1}{\overset{N}{\mathrm{\&Sigma;}}}w\left[i\right]}---\left(9\right)$

Wherein, w[i] be the weights of each particle, Be whole particle collection accumulative total weights.The weights scope of particle i is [1, N].

After the normalization of all particles of whole particle set is calculated and finished, carry out size and sort, proportion more big with regard to other particle of representing the relative particle set of this particle more near the state position of real motion target, in tracking, play bigger effect.Then the particle unified integration of whole particle collection is got up to sort, choose the proportion maximum estimation of tracking target the most.

Step 5.6: resample and the particle renewal;

Adopt threshold method, some degeneration particle is resampled, setting threshold is 0.6, and weights are considered as the particle of degenerating less than the particle of threshold value, replaces the degeneration particle with the particle of a weights maximum.The all values of information of particle of namely degenerating all are assigned the information of the particle of weights maximum.

The position that every two field picture tracking obtains is as central point, and draw one and tracing area shape zone of the same size are with the field color model after the replacement of the hsv color histogram in this zone initialization, as new target area model.After each two field picture is handled, all to carry out the renewal of primary particle model, with the correctness that guarantees to follow the tracks of.

Step 6: return step 2, continue to read the next frame view data.

Claims (6)

1. background subtraction point-score target detection tracker based on FPGA, comprise FPGA, SD card, LED light, DDR2 SDRAM, two seven segment digital tubes, the LTM display screen is characterized in that: comprise that also embedding among the FPGA is that the integrated video of standard detects the minimum system of following the tracks of with the AVLON bus under SOPC builder environment;

Described SD card storage AVI format video data, the output pin of SD card connects the input pin of FPGA, the LTM display screen is connected with the output pin of FPGA, be used for the display video tracking results, DDR2 SDRAM input pin is connected with the output pin of FPGA, and two seven segment digital tubes are connected with the output pin of FPGA respectively;

Described LED light and seven segment digital tubes all show for program debug and state;

Described is that the integrated video of standard detects the minimum system of following the tracks of and comprises data input module, motion detection block, target tracking module, Nios II soft-core processor module, data cache module and LTM controller module with the AVLON bus under SOPC builder environment;

Described data input module reads video data stored information in the SD card, delivering to motion detection block adopts the background subtraction point-score to obtain the parameter information of moving target, determine size and the coordinate of moving object, draw the target area, target tracking module with the color of target area as tracking characteristics, utilize particle filter algorithm to realize the tracking of target, Nios II soft-core processor module controls and each module of coordination, video data is carried out motion detection and target following processing, call LTM controller module IP kernel, view data after handling is presented on the LTM liquid crystal display, and data cache module realizes that the 256MB space is as system's storage environment of system's operation among the DDR2; The LTM controller module is according to Avalon bus interface standard development and Design LTM display IP kernel, shows the result who handles by the LTM display.

2. the background subtraction point-score target detection tracker based on FPGA according to claim 1, it is characterized in that: described data input module is to utilize NIOS II Integrated Development Environment to transplant the FAT32 file system, designed and Implemented 1 SD card memory controller, GPIO port under the employing SOPC builder is as SD card controller control signal, output comprises three semaphores for the SD card controller, be clock signal sd_clk, command signal sd_cmd, data-signal sd_dat, wherein according to the difference of SD card control signal, also different to the direction of the deration of signal and signal; By 3 input/output ports, three control signals are connected to minimum system under NIOS II Integrated Development Environment, above-mentioned SD card control signal is 1 bit pattern, therefore, being configured to sd_clk is 1 output mode, and sd_cmd is 1 two-way input and output mode, sd_dat is 1 two-way input and output mode, according to the control sequential of SD card read-write, finish video data stored information in the SD card is read, and result's storage.

3. the background subtraction point-score target detection tracker based on FPGA according to claim 1 is characterized in that: the IP kernel of described LTM controller module is according to the design of Avalon-MM interface specification, and the interface signal of this IP kernel comprises:

(1) clock input signal and reset signal;

(2) port signal comprises master port and from the input and output signal of port;

(3) input and output signal of LTM;

Design realizes steering logic module and doubleclocking fifo module in the IP kernel of LTM controller module;

Use Avalon-MM streamline transmission mode that LTM controller IP kernel is mounted on the Avalon bus, LTM controller module IP kernel is integrated among the FPGA.

4. adopt the described background subtraction point-score target detection tracker based on FPGA of claim 1 to carry out the method that target detection is followed the tracks of, it is characterized in that: carry out as follows:

Step 1:SD card initialization;

Step 2: read SD card data;

Step 3: adopt the background subtraction point-score that the data that read are carried out motion target detection;

Step 4: after detecting moving target, the size and location of return movement target namely obtain tracing area;

Step 5: adopt particle filter algorithm that moving target is followed the tracks of;

Step 6: return step 2, continue to read the next frame view data.

5. the background subtraction point-score target detection tracking method based on FPGA according to claim 4, it is characterized in that: the described employing background subtraction of step 3 point-score carries out motion target detection to the data that read, and specifically carries out as follows:

Step 3.1: pre-service;

Adopt rule of three to convert coloured image to gray-scale map, conversion formula as the formula (1), wherein, what coloured image adopted is the rgb color pattern, R, G, B represent the red, green, blue color respectively, I represents half-tone information,

I＝0.114×B+0.587×G+0.299×R (1)

After the conversion, adopt median filtering algorithm to handle, medium filtering be intermediate value with window as result, the step of realization is as follows:

(1) size of selected window;

(2) gray values of pixel points in the window is sorted;

Substitute in the window other data with the intermediate value after the ordering;

Step 3.2: background modeling

Adopt the average background model in the background subtraction point-score to realize target detection, average modeling basic thought is that threshold value is cut apart, utilize threshold value to divide the pixel value of difference front and back image, system reads continuous several two field pictures in the video sequence, the mean value that calculates these several two field pictures is image as a setting, and subsequent frame and background frames are done the difference computing; If difference, just judges that this pixel belongs to the motion object greater than this threshold value, otherwise just judge that this pixel is the image that belongs to background;

Step 3.3: data are carried out binaryzation, obtain the bianry image of moving target;

Step 3.4: adopt the array localization method in bianry image, determine size and the coordinate of moving target;

Judge the information of width and horizontal ordinate, at first set the one-dimension array size, its array size is identical with the width value of image, seeks the value of the pixel of each row all over, when the pixel of a certain row is 0 number when adding 1, in the one-dimension array therewith the number of row correspondence add 1; Set a threshold value, when the value in the array during less than this threshold value, the value zero clearing in the array can reduce the interference of other information in the image like this; After one two field picture has been sought all over, determine the value of width and horizontal ordinate according to the non-vanishing pixel in the one-dimension array: this starting point is the value of horizontal ordinate to first of the value that size is non-vanishing of the N continuous in the judgement array as starting point, and N is the width of target;

The height of target and the value of ordinate are also obtained by above method: set the one-dimension array size, seek the value of the pixel of each row all over, value in the array is the increase of 0 number and increasing one by one along with the pixel in the corresponding line, after finishing less than several zero clearings of threshold value; After image has been sought all over, judge according to the value in the array that when the non-vanishing value of continuous Y, Y is the height of target, the position in the non-vanishing some place array of first among the Y is as the ordinate of target.

6. the background subtraction point-score target detection tracking method based on FPGA according to claim 4, it is characterized in that: the described employing particle filter algorithm of step 5 is followed the tracks of moving target, specifically carries out as follows:

Step 5.1: color space conversion;

Adopt linear color model HSV model to calculate, follow the tracks of as proper vector with the target object color, following formula becomes the HSV colour model with the rgb color model conversion:

r＝R/255.0，g＝G/255.0，b＝B/255.0

$H=\left\{\begin{array}{c}(g-b)/[\max (r,g,b)-\min (r,g,b)];r=\max (r,g,b)\\ 2+(b-r)/[\max (r,g,b)-\min (r,g,b)];g=\max (r,g,b)\\ 4+(r-g)/[\max (r,g,b)-\min (r,g,b)];b=\max (r,g,b)\end{array}\right.---\left(2\right)$

When H＞0, H=H * 60,

When H＜0, H=H+360

$S=\frac{\max (r,g,b)-\min (r,g,b)}{\max (r,g,b)}---\left(3\right)$

V＝max(r，g，b) (4)

Wherein R, G, B represent red, green, blue respectively, and r, g, b represent R, G respectively, the B value is projected in the value in the 0.0-1.0 scope, and H, S, V represent 3 parameters of HSV color space, and namely H represents form and aspect, and S represents saturation degree, and V represents brightness;

Step 5.2: initialization distribution of particles;

Particle is a point in the tracing area, and N particle constitutes a particle collection, and the motion state of tracing area is described by this particle collection, the more big description of N value more accurate, and the particle initialization is exactly to come the stochastic distribution particle in the zone of selecting;

Step 5.3: particle transfer;

Particle after adopting second order autoregression dynamic model to initialization carries out the calculating of state transitions, the particle position after obtaining shifting by metastasis model;

The particle p of the function input parameter of particle transfer for shifting, the width of image and height; Function returns the particle after the transfer, shown in formula

$\left\{\begin{array}{c}{x}_p=x_{p-1}+v\_x_{p-1}\\ y_p=y_{p-1}+v\_y_{p-1}\\ {\mathrm{vx}}_p=V\&times;\mathrm{random}\_1\\ {\mathrm{vy}}_p=V\&times;\mathrm{random}\_2\end{array}\right.---\left(6\right)$

Wherein, V is the speed base value, speed component all be with V as radix, multiply by the random number random in (1,1) then; x _pThe horizontal ordinate of expression prediction back target, y _pThe ordinate of expression prediction back target, vx _pThe expression p moment is along the speed of horizontal ordinate, vy _pThe expression p moment is along the speed of ordinate; x _P-1The horizontal ordinate, the v_x that represent a node _P-1The horizontal ordinate direction speed, the y that represent a last node _P-1The ordinate, the v_y that represent a node _P-1Represent that a last node y direction, random_1 and random_2 are the number between 0 to 1 that system produces;

Step 5.4: calculate the particle weights;

Adopt Pasteur's coefficient to weigh the similarity of the color probability distribution of reference template and candidate template, further calculate the weights size of particle; Centered by the particle position that the estimation range refers to predict, be the zone of size with the tracing area border, in same field of definition X, the BC value of probability distribution p and q is calculated as follows:

$\mathrm{BC}(p,q)=\underset{x\&Element;X}{\mathrm{\&Sigma;}}\sqrt{p\left(x\right)q\left(x\right)}---\left(7\right)$

The scope of BC is [0,1], and more big both similaritys of expression of value are more little, and namely when BC=1, the color histogram in two zones distributes different fully, and when BC=0, the color histogram in two zones distributes identical; The calculating of the weights that utilization calculating Pasteur coefficient is realized, to describe histogrammic similarity, hypothesis weights is W, computing formula is:

W＝1-BC (8)

Carry out similarity relatively according to the color histogram of the big young pathbreaker estimation range of weights and the color histogram of tracing area, the span of weights W is [0,1], weights W is more big, and the expression similarity is more big, the estimation range is tracing area when W gets 1, the more little similarity of weights W is more little, and namely the estimation range is not tracing area;

The calculating of particle weights is to give weights according to tracing area prediction and the similarity of actual tracing area to particle, utilize the normalized weighting particle of weights collection to estimate this moment tracing area in follow the tracks of estimating, namely the estimation range of weights maximum is exactly tracing area;

When calculating the particle weights after the particle transfer, centered by this particle, draw one and tracing area shape zone of the same size, obtain this regional hsv color histogram then, this color histogram is compared with original tracing area color histogram, obtain Pasteur's coefficient B C value, thereby obtain the weights of particle;

Step 5.5: particle weights normalizing;

The normalization of particle weights is to obtain the ratio that each particle of particle set accounts for whole particle collection accumulative total weights, and namely this particle represents the proportion of tracing area state in particle set, and formula is as follows:

$\mathrm{weight}\left[i\right]=\frac{w\left[i\right]}{\underset{1}{\overset{N}{\mathrm{\&Sigma;}}}w\left[i\right]}---\left(9\right)$

Wherein, w[i] be the weights of each particle,

Be whole particle collection accumulative total weights, the scope of particle i weights is [1, N];

After the normalization of all particles of whole particle set is calculated and finished, carry out size and sort, proportion more big with regard to other particle of representing the relative particle set of this particle more near the state position of real motion target; Then the particle unified integration of whole particle collection is got up to sort, choose the proportion maximum estimation of tracking target the most;

Step 5.6: resample and the particle renewal;

Adopt threshold method, the degeneration particle is resampled: setting threshold is considered as the particle of degenerating to weights less than the particle of threshold value, replace the degeneration particle with the particle of a weights maximum, that is to say that all values of information of degeneration particle all are assigned the information of the particle of weights maximum;

The position that every two field picture tracking obtains is as central point, and draw one and tracing area shape zone of the same size are with the field color model after the replacement of the hsv color histogram in this zone initialization, as new target area model; After each two field picture is handled, all to carry out the renewal of primary particle model.

Images