WO2023072290A1 - Tracking target occlusion determination method and apparatus, device and storage medium - Google Patents

Abstract

Embodiments of the present disclosure disclose a tracking target occlusion determination method and apparatus, a device and a storage medium. The method comprises: acquiring first and second image information of a template image and a target image, respectively; determining an occlusion determination value on the basis of the first and second image information and an occlusion determination function, the occlusion determination function comprising a first similarity function between the target image and the template image, a second similarity function between the target image and a background image, and a change degree function between the size of the target image and the size of the template image; and determining an occlusion situation on the basis of the occlusion determination value.

Description

A tracking target occlusion determination method, device, equipment and storage medium

Cross References to Related Applications

This application is based on the Chinese patent application with the application number 202111274073.6, the application date is October 29, 2021, and the application name is "A method, device, equipment and storage medium for tracking target occlusion determination", and requires the Chinese patent application Priority, the entire content of this Chinese patent application is hereby incorporated into this application by reference.

technical field

The present disclosure relates to the field of computer vision, and in particular to a tracking target occlusion determination method, device, equipment and storage medium.

Background technique

Target tracking technology is an important technology in the field of computer vision, and it has been widely used in scenarios such as smart transportation, smart security, and military weapons. In target tracking technology, how to achieve accurate target tracking when the target is occluded is extremely important, and accurately judging whether the target is occluded is the basis of target tracking technology. When it is determined that the target is occluded, the size of the tracking frame needs to be adjusted in time. Therefore, the accuracy of the determination result of tracking target occlusion will affect the adjustment efficiency of the tracking frame, and then affect the tracking effect. However, the existing tracking target occlusion determination methods have problems such as low accuracy and poor robustness.

Contents of the invention

Embodiments of the present disclosure expect to provide a tracking target occlusion determination method, device, device, and storage medium, which can improve the accuracy and robustness of the occlusion determination method.

The technical scheme of the embodiment of the present disclosure is realized in this way:

In the first aspect, an embodiment of the present disclosure provides a method for determining occlusion of a tracking target, including:

Determining a template frame from the video, and including a template image of a tracking target in the template frame; obtaining first image information of the template image; tracking the tracking target of the current frame in the video, and determining a target image; Acquiring second image information of the target image; determining an occlusion determination value corresponding to the current frame based on the first image information, the second image information, and an occlusion determination function; wherein, the occlusion determination function includes: the The first similarity function between the target image and the template image, the second similarity function between the target image and the background image, the change degree function between the size of the target image and the size of the template image; the occlusion determination When the value is greater than or equal to the first preset threshold, it is determined that the tracking target is blocked.

In some embodiments of the present disclosure, the first image information includes: the color autocorrelation histogram of the template image; the second image information includes: the color autocorrelation histogram of the target image; wherein, the The color autocorrelation histogram includes: under a preset distance threshold, the ratio of the number of pixels of each color level in the entire image to the total number of pixels in the entire image.

In some embodiments of the present disclosure, the determining the occlusion determination value corresponding to the current frame based on the first image information, the second image information and the occlusion determination function includes: combining the first image information with the second Substituting the image information into the first similarity function to obtain the first similarity; substituting the second image information into the second similarity function to obtain the second similarity; combining the target image size with the template image Substituting the size into the change degree function to determine the change degree value; making a difference between the first similarity and the second similarity to obtain a similarity difference; if the similarity difference is greater than or equal to the second preset threshold, then use the similarity difference as the occlusion judgment value; if the similarity difference is smaller than the second preset threshold, use the product of the difference similarity and the change degree value as The occlusion judgment value.

In some embodiments of the present disclosure, the first similarity function is:

In the formula,

When the preset distance threshold in the target image is d, the ratio of the number of pixels with color level i to the total number of pixels in the target image;

When the preset distance threshold in the template image is d, the number of pixels whose color grade is i accounts for the ratio of the total number of pixels in the template image; D is a set of preset distance thresholds; q is the number of the color grades.

In some embodiments of the present disclosure, the first image information further includes: a template image size, and the second image information further includes: a target image size; the substituting the target image size and the template image size into the The change degree function, determining the change degree value includes: calculating the size change rate of the target image size compared with the template image size based on the change degree function; if the size change rate is greater than or equal to a third preset Threshold, determine the value of the change degree as 1; if the size change rate is less than the third preset threshold, determine the value of the change degree as 0.

In some embodiments of the present disclosure, the second image information further includes: a color autocorrelation histogram of at least one image block in the target image, a color autocorrelation histogram of a background block of the image block;

The second similarity function is:

In the formula,

When the preset distance threshold is d in the jth image block, the ratio of the number of pixels with color level i to the total number of pixels in the jth image block;

When the preset distance threshold is d in the jth background block, the number of pixels whose color level is i accounts for the ratio of the total number of pixels in the jth background block; D is the set of preset distance thresholds; q is the color level number, u is the number of the image block.

In some embodiments of the present disclosure, the method further includes: longitudinally dividing the target image to obtain a first image block and a second image block; wherein, the first image block is located at the edge of the second image block Upper part; the target image is horizontally divided to obtain a third image block and a fourth image block; wherein the third image block is located on the left of the fourth image block; wherein the at least one image block includes The 1st image block, the 2nd image block, the 3rd image block and the 4th image block; in the current frame, select the side above the 1st image block adjacent to the side of the 1st image block, and have the same size In the current frame, select an area located below the second image block adjacent to the side of the second image block and having the same size as the second background block; in the current frame In the frame, select an area located on the left side of the third image block adjacent to the side of the third image block and has the same size as the third background block; in the current frame, select an area located on the fourth image block The area on the right adjacent to the side of the fourth image block with the same size is used as the fourth background block.

In some embodiments of the present disclosure, after it is determined that the tracking target is blocked, the method further includes: determining the intersection of the target image and the template image based on the position information of the target image and the template image The intersecting area, and the area of the template image; calculating the ratio of the intersecting area to the area of the template image; determining a target occlusion degree value based on the ratio.

In some embodiments of the present disclosure, the method further includes: when the occlusion determination function value of the current frame is smaller than the first preset threshold, using the current frame as a new template frame; based on the new template frame, determine the value of the occlusion decision function for the next frame.

In a second aspect, an embodiment of the present disclosure further provides a tracking target occlusion determination device, including: a processing module configured to determine a template frame from a video, and the template frame includes a template image of the tracking target; an acquisition module configured to Acquire the first image information of the template image; the processing module is also configured to track the tracking target in the current frame of the video, and determine the target image; the acquisition module is also configured to acquire the target The second image information of the image; the processing module is further configured to determine the occlusion determination value corresponding to the current frame based on the first image information, the second image information and the occlusion determination function; wherein, the occlusion determination function Including: a first similarity function between the target image and the template image, a second similarity function between the target image and the background image, and a change degree function between the size of the target image and the size of the template image; The processing module is further configured to determine that the tracking target is blocked when the occlusion determination value is greater than or equal to a first preset threshold.

In a third aspect, an embodiment of the present disclosure provides a tracking target occlusion determination device, including: a processor and a memory configured to store a computer program that can run on the processor, wherein the processor is configured to run the computer program , execute the steps of the method for determining the occlusion of the tracking target as described in the first aspect.

In a fourth aspect, an embodiment of the present disclosure provides a computer storage medium on which a computer program is stored, wherein when the computer program is executed by a processor, the step of determining the occlusion of the tracking target as described in the first aspect is implemented.

The technical solution of the embodiment of the present disclosure, by setting the occlusion judgment that combines the similarity function between the template image and the target image, the similarity function between the target image and the background image, and the change degree function between the size of the target image and the size of the template image The function, when judging the occlusion situation, combines multiple influencing factors for judging the occlusion situation, that is, the first similarity and change degree value between the target image and the template image, and the second similarity between the target image and the background image Combined to determine and judge the occlusion situation, improve the accuracy and robustness of the occlusion judgment method.

Description of drawings

FIG. 1 is a schematic flowchart of a first flow chart of a method for determining occlusion of a tracking target in an embodiment of the present disclosure;

FIG. 2 is a first schematic diagram of a target image in a current frame in an embodiment of the present disclosure;

FIG. 3 is a second schematic diagram of a target image in a current frame in an embodiment of the present disclosure;

FIG. 4 is a schematic flowchart of a method for determining an occlusion judgment value corresponding to a current frame in an embodiment of the present disclosure;

5 is a schematic diagram of an image block and a background block of a target image in an embodiment of the present disclosure;

FIG. 6 is a second schematic flowchart of a method for determining occlusion of a tracking target in an embodiment of the present disclosure;

FIG. 7 is a schematic flowchart of a third flowchart of a method for determining occlusion of a tracking target in an embodiment of the present disclosure;

FIG. 8 is a schematic diagram of the composition and structure of a tracking target occlusion determination device in an embodiment of the present disclosure;

FIG. 9 is a schematic diagram of the composition and structure of a tracking target occlusion determination device in an embodiment of the present disclosure.

Detailed ways

In order to understand the characteristics and technical content of the embodiments of the present disclosure in more detail, the implementation of the embodiments of the present disclosure will be described in detail below in conjunction with the accompanying drawings. The attached drawings are only for reference and description, and are not intended to limit the embodiments of the present disclosure.

FIG. 1 is a schematic flow chart of a first method for identifying a network device in an embodiment of the present disclosure. As shown in Figure 1, the method for judging the occlusion of the tracking target may specifically include:

Step 101: determining a template frame from the video, and the template frame includes a template image of the tracking target;

Here, the video is the video captured by the video capture device, and the tracking target is the target object to be tracked in the video, such as a certain vehicle or a certain person in the video. The template frame is a frame with a template function in the video, which is used for comparison with the current frame of the video, and based on the comparison result, it is judged whether the tracking target in the current frame is blocked. Exemplarily, in some embodiments, the template frame may be an initial frame of the video, and may also be an image frame in the video where the tracking target is not occluded.

Exemplarily, in an actual application, the image within the tracking frame in the template frame is used as the template image. Exemplarily, in some embodiments, the method further includes: using the first frame of the video as a template frame; selecting a region in the first frame that contains the tracking target as a template image.

Step 102: Acquiring first image information of the template image;

The first image information includes template image related information. Exemplarily, in some embodiments, the first image information includes: a color autocorrelation histogram of the template image. Wherein, the color autocorrelation histogram includes: under a preset distance threshold, the ratio of the number of pixels of each color level in the entire image to the total number of pixels in the entire image.

It should be noted that the color autocorrelation histogram not only contains the quantity distribution information of different levels of color in the entire image, but also contains the spatial distribution information of the same level of color, and it is beneficial to obtain more accurate image similarity by using it for similarity calculation .

Step 103: track the tracking target of the current frame in the video, and determine the target image;

Here, the current frame is an image frame in the video that needs to track the tracking target and determine the occlusion of the tracking target. Tracking the tracking target of the current frame in the video can be achieved by a tracking algorithm. Exemplary, the tracking algorithm includes but is not limited to a single target tracking algorithm, such as Siamese Region Proposal Networks (SiamRPN, Siamese Region Proposal Networks) Target tracking algorithm, Discriminative Scale Space Tracker (DSST, Discriminatiive Scale Space Tracker,) target tracking algorithm, etc.

Here, the target image is the image area containing the tracking target in the current frame. Exemplarily, in practical applications, the image within the tracking frame in the current frame may be used as the target image. Exemplarily, FIG. 2 is a first schematic diagram of a target image in a current frame in an embodiment of the present disclosure, and FIG. 3 is a second schematic diagram of a target image in a current frame in an embodiment of the present disclosure. As shown in Figure 2 and Figure 3, the vehicle image inside the box is the target image in the current frame.

Step 104: acquiring second image information of the target image;

The second image information includes target image related information. Exemplarily, in some embodiments, the second image information includes: the color autocorrelation histogram of the target image; wherein, the color autocorrelation histogram includes: under a preset distance threshold, each The ratio of the number of pixels of the color level to the total number of pixels of the entire image.

Step 105: Determine an occlusion determination value corresponding to the current frame based on the first image information, the second image information, and the occlusion determination function;

Wherein, the occlusion determination function includes: a first similarity function between the target image and the template image, a second similarity function between the target image and the background image, a size of the target image and a size of the template image The degree of change function between.

Here, since the template image is the image when the tracking target in the video is not occluded, the target image is compared with the template image to determine the similarity between the two (namely the first similarity), if the similarity between the two The larger the value, the closer the target image is to the template image, that is, the closer the tracking target in the target image is to the complete tracking target. Therefore, the occlusion situation can be determined by the similarity between the target image and the template image. Incorporating the first similarity into the occlusion judgment function can improve the accuracy of the occlusion judgment value, and further improve the accuracy of the occlusion judgment result.

Here, the background image is a partial image area around the target image in the current frame. If the similarity between the background image and the target image (that is, the second similarity) is greater, it indicates that the background image is more similar to the target image, indicating that the probability of the tracking target being blocked is greater. Therefore, the occlusion situation can be determined by comparing the similarity between the target image and the background image. Integrating the second similarity into the occlusion determination function can improve the accuracy of the occlusion determination value, and further improve the accuracy of the occlusion determination result.

Here, there are two main reasons for introducing the change degree function into the occlusion determination function: on the one hand, the current more advanced tracking algorithm can solve the problem of partial occlusion of the tracking target, so not all occlusion situations need to be dealt with . In the first schematic diagram of the target image shown in Figure 2, the car on the left is the tracking target. Although there is a partial occlusion, it can be determined that it is not occluded without processing the occlusion situation; on the other hand, The current excellent target tracking algorithms all have good scale adaptive capabilities, that is, when the target is occluded, the tracking algorithm will adjust the size of the tracking frame in time. In the second schematic diagram of the target image shown in Figure 3, the car on the left is When tracking the target, when the tracking target is occluded, the size of the tracking frame (target image size) changes greatly. Therefore, the scale change of the tracking frame is correlated with the target occlusion, and the robustness of the occlusion determination method can be improved by incorporating the change degree function between the target image size and the template image size into the occlusion determination function.

In summary, the occlusion judgment function is jointly determined by the first similarity function between the target image and the template image, the second similarity function between the target image and the background image, and the change degree function between the size of the target image and the size of the template image. When judging the occlusion situation, multiple influencing factors for judging the occlusion situation are combined, that is, the first similarity and change degree value between the target image and the template image, and the second similarity between the target image and the background image are combined to obtain Determine and judge the occlusion situation, and improve the accuracy and robustness of the occlusion judgment method.

Step 106: When the occlusion determination value is greater than or equal to a first preset threshold, determine that the tracking target is occluded.

Here, the first preset threshold may be constant, and may be determined according to actual requirements.

Exemplarily, in some embodiments, the method further includes: when the occlusion determination function value of the current frame is smaller than the first preset threshold, using the current frame as a new template frame; based on the new template frame to determine the value of the occlusion decision function for the next frame.

Here, when the occlusion determination function value of the current frame is smaller than the first preset threshold, it is determined that the tracking target is not occluded. The current frame where the target is not occluded is used as the template frame, and the occlusion judgment function value of the next frame is carried out, so as to realize the occlusion judgment of the tracking target in the whole video.

Here, the subject of execution of steps 101 to 106 may be a processor of the tracking target occlusion determination device. Exemplarily, the tracking target occlusion determination method in the present disclosure can also be coupled to a specific target tracking algorithm, which has good scalability.

In the technical solution of the present disclosure, by setting an occlusion judgment function that combines the similarity function between the template image and the target image, the similarity function between the target image and the background image, and the change degree function between the size of the target image and the size of the template image, When judging the occlusion situation, the factors that can be used to judge the occlusion situation are combined, that is, the first similarity, the second similarity and the change degree value are combined, and the occlusion situation is determined and judged by the combined result, which improves the occlusion judgment method. accuracy and robustness.

In order to illustrate the color autocorrelation histogram, the process of determining the color autocorrelation histogram of the target image is taken as an example for further illustration.

Exemplarily, in practical applications, the method for determining the color autocorrelation histogram of the target image includes:

Step 201: convert the target image into HSV color space by RGB color space, and extract the H (hue) component of HSV color space;

Step 202: Non-uniformly quantize the H color component of the target image into q intervals according to the following formula, where each interval corresponds to a color level. Exemplarily, the value of q here is 14.

Among them, the H (hue) component in the HSV color space represents color information, and its value ranges from 0° to 360°, and its value corresponds to the wavelength of each color in the visible spectrum, so the H component range corresponding to each color It is non-uniformly distributed; this method first non-uniformly quantizes the H color component of the target image into 7 intervals according to the seven colors of red, orange, yellow, green, blue, blue, and purple, and then divides each quantization interval into two In the small interval, the H color component is finally non-uniformly quantized into 14 intervals. Exemplarily, if the H component of pixel i in the target image is between 0° and 23°, it indicates that the color level of the pixel is 0. If the H component of pixel i in the target image is located between 330° and 360°, it indicates that the color level of the pixel is 1.

Step 203: Calculate the color autocorrelation histogram of the target image in the tth frame (current frame) according to the following formula, and normalize the histogram:

Η(r,d)=Num({(p ₁ ,p ₂ )|p ₁ ∈I,p ₂ ∈I,r _s ∈I _R ,|p ₁ -p ₂ |=d})

Among them, p ₁ =(x ₁ ,y ₁ ) and p ₂ =(x ₂ ,y ₂ ) represent the pixels in image I (target image), and I _R =(0,1,...,13) represents step 202 The set of color levels determined in , |p ₁ -p ₂ |=max(|x ₁ -x ₂ |, |y ₁ -y ₂ |) is used to calculate the distance between pixel points p ₁ and p ₂ , D is A set of preset distance values d, Num( ) is used to count the number of pixels that meet the conditions, and for each preset distance value, a color autocorrelation histogram with 14 bins (equivalent to a scale) is obtained

Exemplarily, the preset distance value d may be 1, 3, 5, where d∈D=(1, 3, 5).

Based on the same method as above steps 201-203, the color autocorrelation histogram of the template image can be obtained

and color autocorrelation histograms for other images.

On the basis of the above-mentioned embodiments, the method of determining the occlusion determination value corresponding to the current frame in step 105 based on the first image information, the second image information and the occlusion determination function is further illustrated with an example. Exemplarily, FIG. 4 is a schematic flowchart of a method for determining an occlusion determination value corresponding to a current frame in an embodiment of the present disclosure.

Exemplarily, in some embodiments, the determining the occlusion determination value corresponding to the current frame based on the first image information, the second image information and the occlusion determination function includes:

Step 401: Substituting the first image information and the second image information into the first similarity function to obtain a first similarity;

Exemplarily, in some embodiments, the first similarity function is:

In the formula,

When the preset distance threshold in the target image is d, the ratio of the number of pixels with color level i to the total number of pixels in the target image;

When the preset distance threshold in the template image is d, the number of pixels whose color grade is i accounts for the ratio of the total number of pixels in the template image; D is a set of preset distance thresholds; q is the number of the color grades.

Step 402: Substituting the second image information into the second similarity function to obtain a second similarity;

Exemplarily, in some embodiments, the second image information further includes: a color autocorrelation histogram of at least one image block in the target image, a color autocorrelation histogram of a background block of the image block;

The second similarity function is:

In the formula,

When the preset distance threshold is d in the jth image block, the ratio of the number of pixels with color level i to the total number of pixels in the jth image block;

When the preset distance threshold is d in the jth background block, the number of pixels whose color level is i accounts for the ratio of the total number of pixels in the jth background block; D is the set of preset distance thresholds; q is the color level number, u is the number of the image block. Exemplarily, the value of u may be 4.

Here, at least one image block is a plurality of image blocks obtained by dividing the target image into blocks.

Exemplarily, in some embodiments, the method further includes: longitudinally dividing the target image to obtain a first image block and a second image block; wherein, the first image block is located in the second image The upper part of the block; the target image is horizontally divided to obtain the third image block and the fourth image block; wherein the third image block is located at the left part of the fourth image block; wherein the at least one image The blocks include a first image block, a second image block, a third image block and a fourth image block;

In the current frame, select an area above the first image block that is adjacent to the side of the first image block and has the same size as the first background block; in the current frame, select an area located on the first image block The area below the 2 image block adjacent to the side of the 2nd image block and having the same size is used as the 2nd background block; in the current frame, select the area located on the left side of the 3rd image block and The area with adjacent sides and the same size is used as the third background block; in the current frame, select the area on the right side of the fourth image block that is adjacent to the side of the fourth image block and has the same size as the first background block 4 background blocks.

When dividing the target image into blocks, unequal division may be performed. For example, the unequal division can be: dividing the left two-thirds of the target image as an image block; dividing the right two-thirds of the target image as an image block; dividing the upper two-thirds of the target image as an image block , divide the lower two-thirds of the target image as an image block. The following takes equal division as an example to describe in detail.

FIG. 5 is a schematic diagram of an image block and a background block of a target image in an embodiment of the present disclosure. In Fig. 5, o1 is the upper half image of the target image, which is equivalent to the above-mentioned first image block; o2 is the lower half image of the target image, which is equivalent to the above-mentioned second image block; o3 is the left half image of the target image, which is equivalent to In the above-mentioned third image block; o4 is the right half image of the target image, which is equivalent to the above-mentioned fourth image block. b1, b2, b3, and b4 correspond to the above-mentioned first background block, second background block, third background block, and fourth background block, respectively.

Step 403: Substituting the size of the target image and the size of the template image into the degree of change function to determine a degree of change value;

Exemplarily, in some embodiments, the first image information further includes: a template image size, and the second image information further includes: a target image size.

The step of substituting the target image size and the template image size into the change degree function to determine the change degree value includes: calculating the size of the target image size compared with the template image size based on the change degree function Rate of change; if the rate of change in size is greater than or equal to a third preset threshold, determine that the value of the degree of change is 1; if the rate of change in size is less than the third preset threshold, determine that the value of the degree of change is 0 .

Exemplarily, in practical applications, the size of the tracking frame in the current frame may be used as the target image size, and the size of the tracking frame in the template frame may be used as the size of the template image. Exemplarily, the size of the target image includes: the width and height of the tracking frame in the current frame; the size of the template image includes: the width and height of the tracking frame in the template frame.

Exemplarily, the change degree function can be represented by the following formula.

In the formula, w _t and h _t represent the width and height of the tracking frame in the tth frame (current frame), respectively, w _T and h _T represent the width and height of the tracking frame in the template frame, respectively, and μ is the third preset threshold. Exemplarily, the value of μ may be 0.7.

Step 404: making a difference between the first similarity and the second similarity to obtain a similarity difference;

Step 405: If the similarity difference is greater than or equal to a second preset threshold, use the similarity difference as the occlusion determination value; if the similarity difference is smaller than the second preset threshold, The product of the difference similarity and the change degree value is then used as the occlusion determination value.

Here, the method of determining the occlusion judgment value can be expressed by the following formula:

In the formula, φ represents the similarity difference, that is, the greater the similarity between the target image and the target template, and the smaller the similarity between the target image and the background image, the greater the value of φ.

is an occlusion determination function, and η is a second preset threshold. Exemplary, n can take a value of 0.3, when φ is less than the second preset threshold,

It is the product of the degree of change and the value of φ.

The current excellent target tracking algorithms have good scale adaptive capabilities, that is, when the target is occluded, the tracking algorithm will adjust the size of the tracking frame in time. At this time, it is not accurate enough to judge whether the target is occluded by only using the similarity of the color distribution. of. Because the color autocorrelation histogram not only contains the quantity distribution information of different grade colors in the whole image, but also contains the spatial distribution information of the same grade color. Therefore, by determining the first similarity and the second similarity based on the color autocorrelation histogram of the image, it is beneficial to obtain a more accurate image similarity, thereby improving the accuracy of the tracking target occlusion determination method; by combining the target image size with The variation degree value between template image sizes is integrated into the determination process of the occlusion judgment value, which can improve the robustness of the occlusion judgment method.

In order to better reflect the purpose of the present disclosure, on the basis of the embodiments of the present disclosure, further illustrations are made. FIG. 6 is a second schematic flow chart of a method for determining occlusion of a tracking target in an embodiment of the present disclosure. As shown in Figure 6, the tracking target occlusion determination method may specifically include:

Step 601: Determine a template frame from the video, and the template frame includes a template image of the tracking target;

Exemplarily, in some embodiments, the method further includes: using the first frame of the video as a template frame; selecting a region in the first frame that contains the tracking target as a template image.

Step 602: Obtain first image information of the template image;

Wherein, the first image information includes: the color autocorrelation histogram of the template image and the size of the template image. Wherein, the color autocorrelation histogram includes: under a preset distance threshold, the ratio of the number of pixels of each color level in the entire image to the total number of pixels in the entire image.

Step 603: Track the tracking target in the current frame of the video, and determine the target image;

Here, tracking the tracking target in the current frame of the video may be implemented by a tracking algorithm. Exemplarily, the tracking algorithm includes but not limited to a single target tracking algorithm SiamRPN, DSST target tracking algorithm, and the like. Here, the target image is the image area containing the tracking target in the current frame. Exemplarily, in practical applications, the target image is the image area within the tracking frame.

Step 604: Obtain second image information of the target image;

Here, the second image information includes: the color autocorrelation histogram of the target image, the size of the target image, the color autocorrelation histogram of at least one image block in the target image, and the color autocorrelation histogram of the background block of the image block.

Step 605: Determine an occlusion determination value corresponding to the current frame based on the first image information, the second image information, and the occlusion determination function;

Wherein, the occlusion determination function includes: a first similarity function between the target image and the template image, a second similarity function between the target image and the background image, a size of the target image and a size of the template image The degree of change function between.

Specifically, based on the first image information, the second image information, and the occlusion determination function, determining the occlusion determination value corresponding to the current frame includes:

Substituting the first image information and the second image information into the first similarity function to obtain a first similarity; substituting the second image information into the second similarity function to obtain a second similarity; Substituting the size of the target image and the size of the template image into the degree of change function to determine the degree of change; if the difference in similarity is greater than or equal to a second preset threshold, use the difference in similarity as the An occlusion determination value; if the similarity difference is smaller than the second preset threshold, the product of the difference similarity and the change degree value is used as the occlusion determination value.

Step 606: When the occlusion determination value is greater than or equal to a first preset threshold, determine that the tracking target is occluded;

Step 607: Based on the position information of the target image and the template image, determine the intersection area of the target image and the template image and the area of the template image;

Step 608: Calculate the ratio of the intersection area to the area of the template image;

Step 609: Determine a target occlusion degree value based on the ratio.

Exemplarily, in some embodiments, the calculation method of the occlusion degree value can be expressed by the following formula:

w ₀ =min(x _T +h _T ,x _t +h _t )-max(x _T ,x _t ) _;

h ₀ ＝min(y _T +w _T ,y _t +w _t )-max(y _T ,y _t )

In the formula, w ₀ h ₀ represents the area of the intersection of the current frame tracking frame and the template frame tracking frame, w _T h _T represents the area of the template frame tracking frame; w _T , h _T represent the width and High; ρ represents the degree of occlusion, and its value range is [0, 1]. The larger the value, the higher the degree of occlusion of the target.

Here, the subject of execution of steps 601 to 609 may be a processor of the tracking target occlusion determination device.

In the technical solution of the present disclosure, by setting an occlusion judgment function that combines the similarity function between the template image and the target image, the similarity function between the target image and the background image, and the change degree function between the size of the target image and the size of the template image, When judging the occlusion situation, multiple influencing factors for judging the occlusion situation can be combined, that is, the first similarity and change degree value between the target image and the template image, and the second similarity between the target image and the background image. Combined, to determine and judge the occlusion situation, improve the accuracy and robustness of the occlusion judgment method.

In order to better reflect the purpose of the present disclosure, on the basis of the embodiments of the present disclosure, a further example is given. FIG. 7 is a schematic flowchart of a third method for determining occlusion of a tracking target in an embodiment of the present disclosure. As shown in Figure 7, the method for judging the occlusion of the tracking target may specifically include:

Step 701: Obtain the tth frame;

Step 702: Determine whether t is equal to 1; if so, execute step 703; if not, execute step 704;

Step 703: Frame the target and determine the template frame and template image;

Specifically, the first frame is used as the template frame, and the tracking frame containing the tracking target is selected, and the image in the frame is used as the template image;

Step 704: Obtain the size and color autocorrelation histogram of the template image;

Specifically, the size of the template image is the size of the tracking frame, and the color autocorrelation histogram is the color autocorrelation histogram of the template image. Here, the template image may be a preset image area containing a tracking target in a preset frame;

Step 705: Determine the target image in the tth frame based on the tracking algorithm;

Step 706: Obtain the size of the target image, the color autocorrelation diagram of the image block and the color autocorrelation histogram of the background block;

Here, the size of the target image is the size of the tracking box in the current frame.

Step 707: Calculating the first similarity, second similarity, and change degree values;

Here, the first similarity is the similarity between the target image and the template image, the second similarity is the similarity between the target image and the background image, and the change degree value is the change degree value between the target image size and the template image size.

Step 708: Calculate the occlusion judgment value;

Step 709: Whether it is blocked; if yes, execute steps 711 and 712; if not, execute step 710;

Step 710: use the tth frame as a template frame;

Step 711: set t=t+1, and return to step 704;

Step 712: Calculate the intersection area of the intersecting part between the target image and the template image and the area of the template image;

Step 713: Determine the target occlusion degree value.

In the technical solution of the present disclosure, by setting an occlusion judgment function that combines the similarity function between the template image and the target image, the similarity function between the target image and the background image, and the change degree function between the size of the target image and the size of the template image, When judging the occlusion situation, the factors that can be used to judge the occlusion situation can be combined, and the occlusion situation can be determined and judged by the combined results, so as to improve the accuracy and robustness of the occlusion judgment method; The degree of occlusion is calculated by the intersection of the intersection, which can more accurately describe the occlusion of the tracking target; the first similarity and the second similarity are determined by the color autocorrelation histogram based on the image, which is beneficial to obtain a more accurate image similarity, and then improve Accuracy of tracking object occlusion determination method.

Fig. 8 is a schematic diagram of the composition and structure of a tracking target occlusion determination device in an embodiment of the present disclosure, showing a tracking target occlusion determination device 80, the tracking target occlusion determination device 80 specifically includes:

The processing module 801 is configured to determine a template frame from the video, and the template frame includes a template image of the tracking target;

An acquiring module 802, configured to acquire first image information of the template image;

The processing module 801 is further configured to track the tracking target in the current frame of the video, and determine a target image;

The obtaining module 802 is further configured to obtain second image information of the target image;

The processing module 801 is further configured to determine an occlusion determination value corresponding to the current frame based on the first image information, the second image information, and an occlusion determination function; wherein the occlusion determination function includes: the target image A first similarity function with the template image, a second similarity function between the target image and the background image, a change degree function between the size of the target image and the size of the template image;

The processing module 801 is further configured to determine that the tracking target is blocked when the occlusion determination value is greater than or equal to a first preset threshold.

In some embodiments, the first image information includes: the color autocorrelation histogram of the template image; the second image information includes: the color autocorrelation histogram of the target image; wherein, the color autocorrelation histogram The correlation histogram includes: the ratio of the number of pixels of each color level in the entire image to the total number of pixels in the entire image under a preset distance threshold.

In some embodiments, the processing module 801 is configured to substitute the first image information and the second image information into the first similarity function to obtain the first similarity; and substitute the second image information into the The second similarity function is used to obtain the second similarity; the target image size and the template image size are substituted into the change degree function to determine the change degree value; for the first similarity and the second similarity If the similarity difference is greater than or equal to the second preset threshold, then use the similarity difference as the occlusion judgment value; if the similarity difference is less than the set If the second preset threshold is used, the product of the difference similarity and the change degree value is used as the occlusion determination value.

In some embodiments, the first similarity function is:

In the formula,

When the preset distance threshold in the target image is d, the ratio of the number of pixels with color level i to the total number of pixels in the target image;

When the preset distance threshold in the template image is d, the number of pixels whose color grade is i accounts for the ratio of the total number of pixels in the template image; D is a set of preset distance thresholds; q is the number of the color grades.

In some embodiments, the first image information further includes: template image size, and the second image information further includes: target image size; the processing module 801 is configured to calculate the target based on the change degree function The size change rate of the image size compared with the size of the template image; if the size change rate is greater than or equal to a third preset threshold, determine that the change degree value is 1; if the size change rate is less than the third preset threshold A preset threshold is used to determine that the value of the degree of change is 0.

In some embodiments, the second image information further includes: a color autocorrelation histogram of at least one image block in the target image, a color autocorrelation histogram of a background block of the image block;

The second similarity function is:

In the formula,

When the preset distance threshold is d in the jth image block, the ratio of the number of pixels with color level i to the total number of pixels in the jth image block;

When the preset distance threshold is d in the jth background block, the number of pixels whose color level is i accounts for the ratio of the total number of pixels in the jth background block; D is the set of preset distance thresholds; q is the color level number, u is the number of the image block.

In some embodiments, the processing module 801 is further configured to vertically divide the target image to obtain a first image block and a second image block; wherein, the first image block is located in the second image block The upper part; the target image is horizontally divided to obtain the third image block and the fourth image block; wherein the third image block is located at the left part of the fourth image block; wherein the at least one image block Including the first image block, the second image block, the third image block and the fourth image block; in the current frame, select the side adjacent to the first image block located above the first image block, and the size The same area is used as the first background block; in the current frame, select an area located below the second image block adjacent to the side of the second image block and of the same size as the second background block; in the In the current frame, select an area located on the left side of the third image block that is adjacent to the side of the third image block and has the same size as the third background block; in the current frame, select an area located in the fourth image The right side of the block is adjacent to the fourth image block, and the area with the same size is used as the fourth background block.

In some embodiments, the processing module 801 is further configured to determine the intersection area of the intersection of the target image and the template image based on the position information of the target image and the template image, and the template image calculating the ratio of the intersecting area to the area of the template image; determining a target occlusion degree value based on the ratio.

In some embodiments, the processing module 801 is further configured to use the current frame as a new template frame when the occlusion determination function value of the current frame is smaller than the first preset threshold; based on the new Template frame, which determines the occlusion judgment function value of the next frame.

Based on the hardware implementation of each unit in the above-mentioned tracking target occlusion determination device, the embodiment of the present disclosure also provides another tracking target occlusion determination device,

FIG. 9 is a schematic diagram of the composition and structure of a tracking target occlusion determination device in an embodiment of the present disclosure. As shown in FIG. 9 , the device 90 includes: a processor 901 and a memory 902 configured to store a computer program that can run on the processor; wherein, when the processor 901 is configured to run the computer program, execute the methods in the foregoing embodiments step.

Certainly, in actual application, as shown in FIG. 9 , various components in the tracking target occlusion determination device are coupled together through a bus system 903 . It can be understood that the bus system 903 is used to realize connection and communication between these components. In addition to the data bus, the bus system 903 also includes a power bus, a control bus and a status signal bus. However, the various buses are labeled bus system 903 in FIG. 9 for clarity of illustration.

In practical application, the above-mentioned processor can be application specific integrated circuit (ASIC, Application Specific Integrated Circuit), digital signal processing device (DSPD, Digital Signal Processing Device), programmable logic device (PLD, Programmable Logic Device), on-site At least one of a programmable gate array (FPGA, Field-Programmable Gate Array), a controller, a microcontroller, and a microprocessor. It can be understood that, for different devices, the electronic device used to implement the above processor function may also be other, which is not specifically limited in this embodiment of the present disclosure.

Above-mentioned memory can be volatile memory (volatile memory), such as random access memory (RAM, Random-Access Memory); Or non-volatile memory (non-volatile memory), such as read-only memory (ROM, Read-Only Memory), flash memory (flash memory), hard disk (HDD, Hard Disk Drive) or solid-state drive (SSD, Solid-State Drive); or a combination of the above types of memory, and provide instructions and data to the processor.

In an exemplary embodiment, an embodiment of the present disclosure further provides a computer-readable storage medium, such as a memory including a computer program, and the computer program can be executed by a processor of a network device identification device to complete the steps of the foregoing method.

It should be understood that the terminology used in the present disclosure is for the purpose of describing particular embodiments only and is not intended to be limiting of the present disclosure. As used in this disclosure and the appended claims, the singular forms "a", "the", and "the" are intended to include the plural forms as well, unless the context clearly dictates otherwise. It should also be understood that the term "and/or" as used herein refers to and includes any and all possible combinations of one or more of the associated listed items. The expressions "has", "may have", "comprises" and "comprises", or "may comprise" and "may comprise" in this disclosure may be used herein to indicate the presence of corresponding features (eg, such as values, functions, elements such as operations or components), but does not exclude the presence of additional features.

It should be understood that although the terms first, second, third, etc. may be used in the present disclosure to describe various information, the information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another and are not necessarily used to describe a specific order or sequence. For example, without departing from the scope of the present invention, first information may also be called second information, and similarly, second information may also be called first information.

The technical solutions described in the embodiments of the present disclosure may be combined arbitrarily if there is no conflict.

In the several embodiments of the present disclosure, it should be understood that the disclosed methods, devices and equipment can be implemented in other ways. The above-described embodiments are only illustrative. For example, the division of units is only a logical function division. In actual implementation, there may be other division methods, such as: multiple units or components can be combined, or can be integrated into Another system, or some features may be ignored, or not implemented. In addition, the coupling, or direct coupling, or communication connection between the components shown or discussed may be through some interfaces, and the indirect coupling or communication connection of devices or units may be electrical, mechanical or other forms of.

The units described above as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place or distributed to multiple network units; Part or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

In addition, each functional unit in each embodiment of the present disclosure may be integrated into one processing unit, or each unit may be used as a single unit, or two or more units may be integrated into one unit; the above-mentioned integration The unit can be realized in the form of hardware or in the form of hardware plus software functional unit.

The above is only a specific implementation of the present disclosure, but the scope of protection of the present disclosure is not limited thereto. Anyone skilled in the art can easily think of changes or substitutions within the technical scope of the present disclosure. should fall within the protection scope of the present disclosure.

Industrial Applicability

The present disclosure provides a tracking target occlusion determination method, device, device, and storage medium, including: acquiring first and second image information of a template image and a target image respectively; based on the first and second image information and an occlusion determination function, determining Blocking judgment value; wherein, the blocking judgment function includes: the first similarity function of the target image and the template image, the second similarity function of the target image and the background image, the change degree function between the size of the target image and the size of the template image ; Determine the occlusion situation based on the occlusion judgment value. In this way, by setting an occlusion judgment function that combines two similarity functions and change degree functions, when judging the occlusion situation, multiple influencing factors for judging the occlusion situation are combined, that is, the first similarity between the target image and the template image Degree and change degree values, and the second similarity between the target image and the background image are combined to determine and judge the occlusion situation, and improve the accuracy and robustness of the occlusion judgment method.

Claims (12)

1. A method for judging a tracking target occlusion, the method comprising:

   Determining a template frame from the video, and including a template image of the tracking target in the template frame;

   acquiring first image information of the template image;

   Tracking the tracking target in the current frame of the video, and determining the target image;

   acquiring second image information of the target image;

   Based on the first image information, the second image information, and an occlusion determination function, determine an occlusion determination value corresponding to the current frame; wherein the occlusion determination function includes: a first similarity between the target image and the template image A degree function, a second similarity function between the target image and the background image, a change degree function between the size of the target image and the size of the template image;

   When the occlusion determination value is greater than or equal to a first preset threshold, it is determined that the tracking target is occluded.
2. The method according to claim 1, wherein the first image information comprises: a color autocorrelation histogram of the template image; the second image information comprises: a color autocorrelation histogram of the target image; wherein , the color autocorrelation histogram includes: under a preset distance threshold, the ratio of the number of pixels of each color level in the entire image to the total number of pixels in the entire image.
3. The method according to claim 2, wherein said determining the occlusion determination value corresponding to the current frame based on the first image information, the second image information and the occlusion determination function comprises:

   Substituting the first image information and the second image information into the first similarity function to obtain a first similarity;

   Substituting the second image information into the second similarity function to obtain a second similarity;

   Substituting the target image size and the template image size into the change degree function to determine the change degree value;

   Making a difference between the first similarity and the second similarity to obtain a similarity difference;

   If the similarity difference is greater than or equal to a second preset threshold, the similarity difference is used as the occlusion judgment value;

   If the similarity difference is smaller than the second preset threshold, the product of the difference similarity and the change degree value is used as the occlusion determination value.
4. The method according to claim 3, wherein the first similarity function is:

   

   In the formula,

   

   When the preset distance threshold in the target image is d, the ratio of the number of pixels with color level i to the total number of pixels in the target image;

   

   When the preset distance threshold in the template image is d, the number of pixels whose color grade is i accounts for the ratio of the total number of pixels in the template image; D is a set of preset distance thresholds; q is the number of the color grades.
5. The method according to claim 3, wherein the first image information further includes: template image size, and the second image information further includes: target image size;

   The step of substituting the target image size and the template image size into the change degree function to determine the change degree value includes:

   calculating a size change rate of the target image size compared to the template image size based on the change degree function;

   If the dimensional change rate is greater than or equal to a third preset threshold, determine that the change degree is 1;

   If the size change rate is less than the third preset threshold, determine that the change degree is 0.
6. The method according to claim 3, wherein the second image information further comprises: a color autocorrelation histogram of at least one image block in the target image, a color autocorrelation histogram of a background block of the image block;

   The second similarity function is:

   

   In the formula,

   

   When the preset distance threshold is d in the jth image block, the ratio of the number of pixels with color level i to the total number of pixels in the jth image block;

   

   When the preset distance threshold is d in the jth background block, the number of pixels whose color level is i accounts for the ratio of the total number of pixels in the jth background block; D is the set of preset distance thresholds; q is the color level number, u is the number of the image block.
7. The method according to claim 6, wherein the method further comprises:

   The target image is divided vertically to obtain a first image block and a second image block; wherein, the first image block is located on the upper part of the second image block;

   The target image is horizontally divided to obtain a third image block and a fourth image block; wherein the third image block is located on the left of the fourth image block; wherein the at least one image block includes the first image block, the second image block, the third image block and the fourth image block;

   In the current frame, select an area above the first image block that is adjacent to the side of the first image block and has the same size as the first background block;

   In the current frame, select an area located below the second image block adjacent to the side of the second image block and having the same size as the second background block;

   In the current frame, select an area located on the left side of the third image block adjacent to the side of the third image block and having the same size as the third background block;

   In the current frame, select an area on the right side of the fourth image block adjacent to the fourth image block and having the same size as the fourth background block.
8. The method according to claim 1, wherein, after determining that the tracking target is blocked, the method further comprises:

   Based on the position information of the target image and the template image, determine the intersection area of the intersection part of the target image and the template image, and the area of the template image;

   calculating the ratio of the intersection area to the area of the template image;

   A target occlusion degree value is determined based on the ratio.
9. The method according to claim 1, wherein the method further comprises:

   When the occlusion determination function value of the current frame is less than the first preset threshold, the current frame is used as a new template frame;

   Based on the new template frame, an occlusion decision function value of the next frame is determined.
10. A tracking target occlusion determination device, said device comprising:

    A processing module configured to determine a template frame from the video, and the template frame includes a template image of the tracking target;

    an acquisition module configured to acquire the first image information of the template image;

    The processing module is further configured to track the tracking target in the current frame of the video, and determine the target image;

    The acquiring module is further configured to acquire second image information of the target image;

    The processing module is further configured to determine an occlusion determination value corresponding to the current frame based on the first image information, the second image information, and an occlusion determination function; wherein, the occlusion determination function includes: the target image and The first similarity function of the template image, the second similarity function between the target image and the background image, the change degree function between the size of the target image and the size of the template image;

    The processing module is further configured to determine that the tracking target is blocked when the occlusion determination value is greater than or equal to a first preset threshold.
11. A tracking target occlusion determination device, wherein the device includes: a processor and a memory configured to store a computer program that can run on the processor,

    Wherein, the processor is configured to execute the steps of the method according to any one of claims 1-9 when running the computer program.
12. A computer-readable storage medium, on which a computer program is stored, wherein, when the computer program is executed by a processor, the steps of the method described in any one of claims 1-9 are implemented.

Images