CN109685827B - Target detection and tracking method based on DSP - Google Patents

Abstract

The invention discloses a DSP-based target detection and tracking method, comprising the following steps: firstly initialize the size of the wave gate and its surrounding background, and set the position of the wave gate in the image to include the target to be tracked; The inner image is binarized and filtered; then the connected domain is marked and the target template is determined; then the position of the target in the next frame of image is obtained according to the position of the target in two consecutive frames of images; finally, the target and the target in the next frame of image are obtained. Correlation coefficient between templates, and determine whether to continue tracking or stop tracking. The target detection and tracking method of the invention has low computational complexity, enlarges the correlation between the target template and the target in the image, ensures the stability of the image tracking, has high tracking accuracy and good real-time performance, and can satisfy the tracking method of the image tracking system. requirements.

Description

A Target Detection and Tracking Method Based on DSP

technical field

The invention belongs to the real-time image processing and tracking control technology, in particular to a DSP-based target detection and tracking method.

Background technique

Object detection and tracking is a fundamental and important task in various applications in the fields of digital video processing and computer vision. In some promising fields, robot control, motion-based recognition, vision-based control, augmented reality, video scene monitoring, and navigation and guidance all require target detection and tracking technology. In the field of computer vision, target detection and tracking is still a hot research field. The focus of the research is also from the simple pursuit of high-precision and high-stability simulation analysis, to the direction of industrial applications with small real-time computation. How to reduce the hardware requirements while meeting as many performance indicators as possible is the actual target detection and Difficulties in tracking research directions. In order to reduce the requirements for hardware, Chinese patent CN201010121006.6 proposes a DSP-based target detection and tracking method and a digital image processing system, but this method has a large amount of computation, poor generality and real-time performance, and is now mainly based on multi-core DSP. It is difficult to get a better application in the video image processing system of the hardware platform. It is of great significance to study a target detection and tracking method with high precision, low computational complexity and good practicability.

SUMMARY OF THE INVENTION

The technical problem solved by the present invention is to provide a target detection and tracking method with low computational complexity, easy implementation and high real-time system requirements.

The technical solution for realizing the object of the present invention is: a DSP-based target detection and tracking method, comprising the following steps:

Step 1. Collect the first frame image in the video;

Step 2. Initialize the size of the wave gate according to the first frame image collected, then initialize the size of the background around the wave gate according to the size of the wave gate, and set the position of the wave gate in the first frame image to include the target to be tracked. The grayscale histogram of the wave gate and the background around the wave gate;

Step 3. Determine a binarization threshold according to the grayscale histogram, and perform binarization processing on the image in the wave gate according to the threshold;

Step 4, filtering the image in the gate after binarization;

Step 5, use the neighborhood labeling algorithm to mark each area of the filtered wave gate image, and obtain several connected areas;

Step 6, count the size of all connected regions in step 5, and arrange the size of the connected regions in descending order, obtain the minimum circumscribed rectangle of the largest connected region among them, and use it as the target template;

Step 7, obtaining the center of the maximum connected area, that is, the location of the target in the first frame image, then obtaining the location of the target in the second frame image according to the center, and updating the target template;

Step 8, obtain the target position in the next frame image according to the target position in two consecutive frames of images;

Step 9, obtain the correlation coefficient between the target and the target template in the next frame of image, and determine whether to repeat step 8 to continue tracking or stop tracking according to the correlation coefficient; if the correlation coefficient Q is greater than or equal to the preset threshold p, then repeat Step 8 Continue tracking; otherwise, increase the number of failures n by 1, and judge the relationship between n and the preset threshold q of the number of failures, if n≥q, stop tracking, otherwise expand the size of the wave gate in pixel-level units, repeat Step 8 continue following.

Compared with the prior art, the present invention has significant advantages: 1) the present invention improves the matching efficiency by adopting the diamond search algorithm, and can reduce the amount of computation on the premise of ensuring the search effect; 2) the right to update the target template in the present invention The value of the value is associated with the correlation coefficient, which avoids updating the error information to the target template when the tracking effect is not good, and ensures the stability of subsequent image tracking; 3) In the present invention, the correlation coefficient is obtained by the de-average method, and in the While reducing the amount of computation, enlarge the correlation between the target template and the target in the image.

The present invention will be described in further detail below with reference to the accompanying drawings.

Description of drawings

FIG. 1 is a flow chart of the DSP-based target detection and tracking method of the present invention.

FIG. 2 is a schematic diagram of a tracking target to be detected in a certain frame of image according to an embodiment of the present invention.

3 is a schematic diagram of a search template used by a diamond search algorithm in an embodiment of the present invention, wherein FIG. 3( a ) is a 5×5 diamond search template, and FIG. 3( b ) is a 3×3 diamond search template.

FIG. 4 is a schematic diagram of a target tracking result in a certain frame of images in an embodiment of the present invention.

Detailed ways

1, a DSP-based target detection and tracking method of the present invention includes the following steps:

Step 1. Collect the first frame image in the video.

Step 2. Initialize the size of the wave gate according to the first frame image collected, then initialize the size of the background around the wave gate according to the size of the wave gate, and set the position of the wave gate in the first frame image to include the target to be tracked. Grayscale histogram of the wave gate and the background around the wave gate.

Further, initialize the size of the wave gate according to the first frame image collected, specifically: initialize the size of the wave gate according to the size of the first frame image and the size of the target to be tracked in the image, assuming that the size of the collected image is w×h , the size of the minimum circumscribed rectangle of the target to be tracked is x×y, and the size of the wave gate is w'×h', where x<w'≤w, y<h'≤h;

Further, the size of the background around the wave gate is determined according to the size of the wave gate, specifically: extending the boundary of the wave gate by several pixels.

Step 3: Determine a binarization threshold according to the grayscale histogram, and perform binarization processing on the image in the wave gate according to the threshold.

Further, the threshold value of binarization is determined according to the grayscale histogram, specifically: the grayscale value corresponding to the valley between the two peaks representing the target to be tracked and its background in the grayscale histogram is used as the threshold value of binarization. .

Step 4: Perform filtering processing on the binarized inner image of the wave gate.

Further, the filtering process specifically adopts the open operation process.

Step 5. Use the neighborhood labeling algorithm to label each area of the filtered wave gate image to obtain several connected areas.

Further, each area of the image in the filtered wave gate is marked with a neighborhood labeling algorithm, wherein the neighborhood labeling algorithm is specifically:

Scan the filtered image in the wave gate pixel by pixel from left to right and from top to bottom until all the pixels in the image in the wave gate are scanned, if the current scanned pixel value is 0, move directly to the bottom One pixel; if the current scanned pixel value is 1, it is marked according to the left pixel and the upper pixel of the current pixel, and the size of the marked value increases with the appearance of a new connected domain, including the following four situations:

(1) The pixel values on the left side and the upper side are both 0, indicating that the currently scanned pixel is the boundary of a new connected domain, and then a new label value is given to the currently scanned pixel;

(2) There is only one pixel value of 1 in the pixel values of the left side and the upper side, then the currently scanned pixel is given the same mark value as the pixel mark value of the pixel value of 1;

(3) The pixel values on the left side and the upper side are both 1 and the mark value is the same, then the currently scanned pixel is given the same mark value as the mark value;

(4) The pixel values on the left side and the upper side are both 1 but the flag values are different, and the flag value assigned to the currently scanned pixel is the smallest flag value among the flag values.

Step 6: Count the sizes of all connected regions in step 5, and sort the connected regions in descending order to obtain the smallest circumscribed rectangle of the largest connected region, and use it as the target template.

Step 7: Obtain the center of the largest connected area, that is, the position of the target in the first frame of image, then obtain the position of the target in the second frame of image according to the center, and update the target template.

Further, the position of the target in the second frame of image is obtained according to the center of the largest connected area, specifically:

Step 7-1. Move the center of the wave gate to the center of the maximum connected area;

Step 7-2: Match the image in the wave gate with the target template through a template matching algorithm to obtain the position of the target in the second frame of image.

Further, the template matching algorithm in step 7-2 specifically adopts a diamond search algorithm.

Further, update the target template, specifically:

Suppose T _k is the target template used for template matching of the k-th frame image, and T _k+1 is the updated target template used for template matching of the k+1-th frame image. The formula for updating the target template is:

In the formula, M _k is the best matching of the target template T _k obtained by the template matching algorithm in the kth frame image with the target position as the center and the template size as the coverage, α is the update weight, c _max is the template matching The output correlation coefficient, τ _t is the threshold for whether the set target template is updated. It can be seen from the formula that when the correlation coefficient c _max is greater than the threshold τ _t , the target template is updated, otherwise it remains unchanged.

Step 8: Acquire the position of the target in the next frame of images according to the position of the target in the two consecutive frames of images.

Further, obtain the location of the target in the next frame of images according to the location of the target in the two consecutive frames of images, specifically:

Step 8-1. Use the least squares method to process the location of the target in two consecutive frames of images, thereby obtaining the area where the target is located in the next frame of image;

Step 8-2, move the wave gate to the center of the area, perform template matching between the image in the wave gate and the updated target template, and obtain the position of the target in the current frame image.

Step 9: Obtain the correlation coefficient between the target in the next frame of image and the target template, and determine whether to repeat Step 8 to continue tracking or stop tracking according to the correlation coefficient.

Further, step 9 obtains the correlation coefficient between the target in the next frame of image and the target template, and judges whether to repeat step 8 to continue tracking or stop tracking according to the correlation coefficient, specifically:

Step 9-1, taking the position of the target in the next frame image in step 8 as the center, intercept the area of the same size as the target template as the target in the next frame image;

Step 9-2, to obtain the correlation coefficient Q between the target and the target template in the next frame of image, the formula used is:

为目标模板的像素灰度均值，y_i为图像中的目标中第i个像素的灰度值，

为图像中的目标的像素灰度均值；In the formula, x _i is the gray value of the ith pixel in the target template,

is the pixel gray mean value of the target template, y _i is the gray value of the ith pixel in the target in the image,

is the pixel gray mean value of the target in the image;

Step 9-3, determine the relationship between the correlation coefficient Q and the preset threshold p, if Q≥p, repeat step 8 to continue tracking; otherwise, increase the number of failures n by 1, and perform step 9-4; The initial value is 0;

Step 9-4: Determine the relationship between n and the preset threshold q for the number of failures. If n ≥ q, stop tracking, otherwise expand the size of the wave gate in pixel-level units, and repeat step 8 to continue tracking.

Example

1, the DSP-based target detection and tracking method of the present invention includes the following contents:

1. Initialize the wave gate size and the background size around the wave gate

The size of the wave gate is determined by the size of the collected image and the size of the target. The main requirement is that the wave gate can completely mark the position of the target. A certain frame of image collected in this embodiment is shown in Figure 2, the size is 1600 pixels * 900 pixels, the target size in the image is 80 pixels * 60 pixels, the size of the wave gate is set to 120 pixels * 120 pixels, and the surrounding area of the wave gate is 120 pixels * 120 pixels. The background size is set to 120px*30px.

2. Determine the binarization threshold

The grayscale histogram of the wave gate and the background surrounding the wave gate is counted to obtain a histogram with a bimodal distribution, and the gray value 20 corresponding to the trough between the wave gates is selected as the binarization threshold.

3. Binarize the inner image of the wave gate

According to the determined binarization threshold, the following binarization processing is performed on the image in the wave gate:

Among them, f(x, y) is the original gray value of the pixel in the wave gate, and f _T (x, y) is the gray value of the pixel in the wave gate after processing.

4. Filter the binarized wave gate image

The open operation is used to process, specifically, first scan each pixel in the image with a structuring element with a size of 3 × 3, and use each pixel in the structuring element to "AND" the pixels it covers. If both are 0, then the pixel is 0, otherwise 1. Then use a structuring element with a size of 3 × 3 to scan each pixel in the image, and use each pixel in the structuring element to "AND" the pixels it covers. If both are 1, the pixel is 1, otherwise it is 0.

5. Use the neighborhood labeling algorithm to mark each area of the image in the filtered wave gate, and obtain several connected areas;

6. Count the sizes of all connected regions marked, and sort the connected regions in descending order to obtain the circumscribed rectangle of the largest connected region, and use it as the target template;

7. Obtain the center of the largest connected area, that is, the position of the target in the first frame of image, and use the diamond search algorithm to obtain the position of the target in the second frame of image according to the center. The specific operations are:

Taking the template image and the target image as the output, first use the large diamond search algorithm to find the best area where the target is located, use this position as the center position of the small diamond search, and then use the small diamond search algorithm to find the exact position of the target. The search template used by the large diamond search algorithm is a 5 × 5 template, and the search template used by the small diamond search algorithm is a 3 × 3 template. The large diamond search template and the small diamond search template are shown in Figure 3. The diamond search algorithm repeatedly uses the diamond search template to search until the point with the smallest search error appears in the center of the template, and the obtained point with the smallest search error is the best matching point.

8. Target template update method

where T _k is the target template used for template matching of the kth frame image, T _k+1 is the updated target template used for template matching of the k+1 frame image, M _k is the template matching in the kth frame image The _algorithm obtains the best match of the target template Tk centered on the target position and covering the template size. α is the update weight, c _max is the correlation coefficient of template matching output, in this embodiment α=0.1c _max , τ _t is the threshold for whether the set target template is updated, in this embodiment τ _t =0.75, when the correlation When the coefficient c _max is greater than the threshold τ _t , the target template is updated, otherwise it remains unchanged.

9. Obtain the location of the target in the next frame of images according to the location of the target in two consecutive frames of images;

Using the position of the target in the first two frames of images, the least squares method is used to calculate the area of the target in the next frame of image. The least squares formula is as follows:

y=ax+b

为前两帧目标所在水平方向上的像素坐标均值，

为目标所在垂直方向上的像素坐标均值。where a is the slope of the fitted curve, b is the intercept of the fitted curve, x is the pixel coordinate in the horizontal direction where the target is located, y is the pixel coordinate in the vertical direction where the target is located,

is the mean of pixel coordinates in the horizontal direction of the target in the first two frames,

is the mean of pixel coordinates in the vertical direction of the target.

10. Move the wave gate to the center of the area where the target is located, perform template matching between the image in the wave gate and the target template updated in the previous frame, and obtain the position of the target in the current frame image.

11. Calculate the correlation coefficient between the target in the image and the target template. The formula used is:

为目标模板的像素灰度均值，y_i为图像中的目标中第i个像素的灰度值，

为图像中的目标的像素灰度均值；相关系数Q∈[-1,1]，且Q越大，相关性越高。In the formula, x _i is the gray value of the ith pixel in the target template,

is the pixel gray mean value of the target template, y _i is the gray value of the ith pixel in the target in the image,

is the pixel gray mean value of the target in the image; the correlation coefficient Q∈[-1,1], and the larger the Q, the higher the correlation.

12. Determine the relationship between the correlation coefficient and the set threshold

In this embodiment, the preset threshold p of the correlation coefficient is set to 0.75, the preset threshold q of the number of failures is set to 50, and the relationship between the correlation coefficient Q and the preset threshold p is determined. If Q≥p, the failure count is cleared and the tracking is continued. , the target tracking result is shown in Figure 4; if Q<p, increment the failure count n by 1, and judge the relationship between n and the preset threshold q for the number of failures, if n≥q, stop tracking, if n<q, then Expand the wave gate to 180 pixels * 180 pixels and continue tracking.

The DSP-based target detection and tracking method of the present invention has low computational complexity, amplifies the correlation between the target template and the target in the image, ensures the stability of image tracking, has high tracking accuracy and good real-time performance, and can meet the requirements of image tracking. System requirements for tracking methods.

Claims (10)

1. a target detection and tracking method based on DSP, is characterized in that, comprises the following steps: Step 1. Collect the first frame image in the video; Step 2. Initialize the size of the wave gate according to the first frame image collected, then initialize the size of the background around the wave gate according to the size of the wave gate, and set the position of the wave gate in the first frame image to include the target to be tracked. The grayscale histogram of the wave gate and the background around the wave gate; Step 3. Determine a binarization threshold according to the grayscale histogram, and perform binarization processing on the image in the wave gate according to the threshold; Step 4, filtering the image in the gate after binarization; Step 5, use the neighborhood labeling algorithm to mark each area of the filtered wave gate image, and obtain several connected areas; Step 6, count the size of all connected regions in step 5, and arrange the size of the connected regions in descending order, obtain the minimum circumscribed rectangle of the largest connected region among them, and use it as the target template; Step 7, obtaining the center of the maximum connected area, that is, the location of the target in the first frame image, then obtaining the location of the target in the second frame image according to the center, and updating the target template; Step 8, obtain the target position in the next frame image according to the target position in two consecutive frames of images; Step 9, obtain the correlation coefficient between the target and the target template in the next frame of image, and determine whether to repeat step 8 to continue tracking or stop tracking according to the correlation coefficient; if the correlation coefficient Q is greater than or equal to the preset threshold p, then repeat Step 8 Continue tracking; otherwise, increase the number of failures n by 1, and judge the relationship between n and the preset threshold q of the number of failures, if n≥q, stop tracking, otherwise expand the size of the wave gate in pixel-level units, repeat Step 8 continue following. 2. DSP-based target detection and tracking method according to claim 1, is characterized in that, in step 1: The initializing the size of the wave gate according to the collected first frame image is specifically: initializing the size of the wave gate according to the size of the first frame image and the size of the target to be tracked in the image, Assuming that the size of the collected image is w×h, the size of the minimum circumscribed rectangle of the target to be tracked is x×y, and the size of the wave gate is w'×h', where x<w'≤w, y<h'≤h; The determining the size of the background around the wave gate according to the size of the wave gate is specifically: extending the boundary of the wave gate by several pixels. 3. The DSP-based target detection and tracking method according to claim 1 or 2, characterized in that, in step 3, the threshold value of binarization is determined according to the grayscale histogram, specifically: the grayscale histogram represents the The gray value corresponding to the trough between the two peaks of the target to be tracked and its background is used as the threshold for binarization. 4 . The DSP-based target detection and tracking method according to claim 3 , wherein the filtering process in step 4 specifically adopts the open operation process. 5 . 5. DSP-based target detection and tracking method according to claim 1, is characterized in that, described in step 5, utilizes neighborhood labeling algorithm to mark each area of the image in the wave gate after filtering, wherein neighborhood labeling algorithm specifically for: Scan the filtered image in the wave gate pixel by pixel from left to right and from top to bottom until all the pixels in the image in the wave gate are scanned, if the current scanned pixel value is 0, move directly to the bottom One pixel; if the current scanned pixel value is 1, it is marked according to the left pixel and the upper pixel of the current pixel, and the size of the marked value increases with the appearance of a new connected domain, including the following four situations: (1) The pixel values on the left side and the upper side are both 0, indicating that the currently scanned pixel is the boundary of a new connected domain, and then a new label value is given to the currently scanned pixel; (2) There is only one pixel value of 1 in the pixel values of the left side and the upper side, then give the currently scanned pixel and the pixel value of the pixel value the same mark value as the pixel mark value of 1; (3) the pixel values on the left side and the upper side are both 1 and the mark value is the same, then the currently scanned pixel is given the same mark value as the mark value; (4) The pixel values on the left side and the upper side are both 1 but the flag values are different, and the flag value assigned to the currently scanned pixel is the smallest flag value among the flag values. 6. DSP-based target detection and tracking method according to claim 1, is characterized in that, described in step 7, obtains the target position in the second frame image according to the center of the largest connected area, specifically: Step 7-1, move the center of the wave gate to the center of the maximum connected area; Step 7-2: Match the image in the wave gate with the target template through a template matching algorithm to obtain the position of the target in the second frame of image. 7 . The DSP-based target detection and tracking method according to claim 6 , wherein the template matching algorithm in step 7-2 specifically adopts a diamond search algorithm. 8 . 8. DSP-based target detection and tracking method according to claim 7, wherein the update target template described in step 7 is specifically: Suppose T _k is the target template used for template matching of the k-th frame image, and T _k+1 is the updated target template used for template matching of the k+1-th frame image. The formula for updating the target template is:

In the formula, M _k is the best matching of the target template T _k obtained by the template matching algorithm in the kth frame image with the target position as the center and the template size as the coverage, α is the update weight, c _max is the template matching The output correlation coefficient, τ _t is the threshold for whether the set target template is updated. It can be seen from the formula that when the correlation coefficient c _max is greater than the threshold τ _t , the target template is updated, otherwise it remains unchanged. 9. DSP-based target detection and tracking method according to claim 1, is characterized in that, described in step 8, obtains the target position in the next frame image according to the target position in two consecutive frames of images, specifically: Step 8-1. Use the least squares method to process the location of the target in two consecutive frames of images, thereby obtaining the area where the target is located in the next frame of image; Step 8-2: Move the wave gate to the center of the area, perform template matching between the image in the wave gate and the updated target template, and obtain the position of the target in the current frame image. 10. DSP-based target detection and tracking method according to claim 1, is characterized in that, in step 9, the correlation coefficient between the target in the next frame image and the target template is obtained, and according to the correlation coefficient, it is determined whether Repeat step 8 to continue tracking or stop tracking, specifically: Step 9-1, taking the position of the target in the next frame image described in step 8 as the center, intercept the area of the same size as the target template as the target in the next frame image; Step 9-2, to obtain the correlation coefficient Q between the target and the target template in the next frame of image, the formula used is:

In the formula, x _i is the gray value of the ith pixel in the target template,

is the pixel gray mean value of the target template, y _i is the gray value of the ith pixel in the target in the image,

is the pixel gray mean value of the target in the image; Step 9-3, determine the relationship between the correlation coefficient Q and the preset threshold p, if Q≥p, repeat step 8 to continue tracking; otherwise, increase the number of failures n by 1, and perform step 9-4; wherein, the number of failures The initial value of n is 0; Step 9-4: Determine the relationship between n and the preset threshold q for the number of failures. If n ≥ q, stop tracking, otherwise expand the size of the wave gate in pixel-level units, and repeat step 8 to continue tracking.

Images