TW202026940A - Target tracking method - Google Patents

Abstract

A target tracing method comprises: capaturing a first result tracking window having a target object and a first reference region near to the first result tracking window in a first frame; capaturing a second result tracking window out of the first result tracking window and a second reference region near to the second result tracking window; calculating a first and a second similarities of the first and the second reference regions relative to a default reference feature respectively by a comparison algorithm; and calculating a first difference ratio of the first feature value of the first result tracking window and a default feature value of an target object template of the tarjet object and a second difference ratio of the second feature value of the second result tracking window and the default deature value by a template matching algorithm, selecting the first result tracking window or the second result tracking window as an initial tracking window of the second frame after the determination according to the first similarity, the second similarity, the first difference ratio and the second differentce ratio.

Description

Target tracking method

The present invention relates to a target tracking method, in particular to a target tracking method using a reference area and a target object.

The conventional Camshift (Continuously Adaptive Mean Shift Algorithm) algorithm uses the characteristics of color distribution to track a moving object. First, select an object as the tracking target, and then create the color histogram of the object, then get the back projection of the entire scene in the tracked scene (frame picture), and then find the centroid position of the object as the object position. Object tracking is achieved by repeatedly performing the same steps as above on the next scene.

If the Camshift algorithm is applied to human tracking, the target tracking frame is usually locked on the face, and tracking is performed according to the characteristics of the skin color of the face. Under normal circumstances, this method can track the target person smoothly. However, once the skin color is disturbed, the tracking accuracy will be reduced. In addition, if the tracking frame deviates from the original person to be tracked due to some factors, the correct face may not be retrieved, resulting in a series of subsequent incorrect tracking results.

For example, when the face to be tracked is completely obscured by another face, since the skin color of the face is similar, according to the judgment of the Camshift algorithm, another person will be tracked at this time. Even if the person you originally wanted to track appears on the screen again, because the tracking frame of the Camshift algorithm has been locked to another face, the tracking frame can no longer return to the face of the person you originally wanted to track, resulting in incorrect tracking results. .

Therefore, the clothes color of the tracked person can be added to assist in judging the situation when the person is interlaced. However, if two people's clothes are of the same color, it is still impossible to accurately distinguish which one is the object originally intended to be tracked according to the above method.

In view of this, the present invention provides a target tracking method to solve the above-mentioned problems.

A target tracking method according to an embodiment of the present invention is suitable for tracking a target object in a continuous first frame and a second frame. The target tracking method includes: in the first frame, An iterative algorithm is executed according to a first initial tracking frame to extract a first result tracking frame and a first reference area, the first result tracking frame includes the target object, and the first reference area is adjacent to the first result tracking Frame; capture a second result tracking frame and a second reference area outside the first result tracking frame, the second reference area being adjacent to the second result tracking frame; and after capturing the second result tracking frame, The method includes: calculating a first degree of similarity of the first reference area with respect to a predetermined reference feature and a second degree of similarity of the second reference area with respect to the predetermined reference feature by using a comparison algorithm; and The board matching algorithm calculates a first feature value of the first result tracking frame, a first difference degree of the first feature value and a preset feature value, a second feature value of the second result tracking frame, and the A second degree of difference between the second characteristic value and the predetermined characteristic value; wherein the predetermined characteristic value is associated with a target template of the target object; when the first degree of similarity is not equal to the second degree of similarity, Use the first result tracking frame or the second result tracking frame corresponding to the larger of the first degree of similarity and the second degree of similarity as the first initial tracking frame of the second frame; or when the When the first degree of similarity is equal to the second degree of similarity, the first result tracking frame or the second result tracking frame corresponding to the smaller of the first degree of difference and the second degree of difference is used as the second The first initial tracking frame of the frame.

According to an embodiment of the target tracking method of the present invention, the first characteristic value, the second characteristic value and the preset characteristic value are a brightness, a chroma, a density, a hue, and a hue of the target object. Saturation, or lightness.

A target tracking method according to an embodiment of the present invention, before capturing the second result tracking frame, further includes: updating the preset feature value of the target template at least once; and capturing the second result tracking frame Later, it also includes: suspend updating the target template.

According to an embodiment of the target tracking method of the present invention, extracting a second result tracking frame includes: executing the iterative algorithm to obtain a plurality of second initial tracking frames outside the first result tracking frame; and sequentially selecting One of the second initial tracking frames is calculated, and a confidence index of the selected second initial tracking frame relative to the target object is calculated; wherein when the confidence index should be less than a threshold value, the first The other of the two initial tracking frames, and the target template is updated at least once according to a characteristic value of the target object; when the confidence index is greater than or equal to the threshold value, the selected second initial tracking frame is used as the The second result tracking frame, and the update of the target template is suspended.

According to an embodiment of the target tracking method of the present invention, the confidence indicator is a ratio of the plurality of pixel values of the second result tracking frame associated with the target object.

According to an embodiment of the target tracking method of the present invention, there is a fixed relative position relationship between the first reference area and the first result tracking frame, and there is a fixed relative position relationship between the second reference area and the second result tracking frame The fixed relative position relationship.

According to a target tracking method according to an embodiment of the present invention, the target object is a human face.

According to an embodiment of the target tracking method of the present invention, the iterative algorithm is the Camshift algorithm. The comparison algorithm is one of a correlation coefficient (Correlation) algorithm, Chi-square algorithm, Intersection algorithm, and Bhattacharyya distance algorithm. The template matching algorithm is Mean Absolute Differences (MAD), Sum of Absolute Differences (SAD), Sum of Squared Differences (Sum of Squared Differences, SSD), and Mean Square Differences (Mean Square Differences, MSD), Normalized Cross Correlation (NCC), Sequential Similiarity Detection Algorithm (SSDA) or Sum of Absolute Transformed Difference (SATD).

The above description of the disclosure and the following description of the implementation manners are used to demonstrate and explain the spirit and principle of the present invention, and to provide a further explanation of the patent application scope of the present invention.

The detailed features and advantages of the present invention are described in detail in the following embodiments. The content is sufficient to enable anyone familiar with the relevant art to understand the technical content of the present invention and implement it accordingly, and according to the content disclosed in this specification, the scope of patent application and the drawings Anyone who is familiar with relevant skills can easily understand the purpose and advantages of the present invention. The following examples further illustrate the viewpoints of the present invention in detail, but do not limit the scope of the present invention by any viewpoint.

FIG. 1 is a block diagram of a target tracking device 100 according to an embodiment of the invention. As shown in FIG. 1, the target tracking device 100 includes a reading unit 110, an iterative processing unit 120 and a judgment unit 130. The iterative processing unit 120 is electrically connected to the reading unit 110, and the judging unit 130 is electrically connected to the reading unit 110 and the iterative processing unit 120. The reading unit 110, the iterative processing unit 120, and the judging unit 130 can be implemented by various chips or processors, which are not limited here.

The target tracking device 100 can execute the target tracking method according to an embodiment of the present invention to track the target object in consecutive multiple frames. The target object is, for example, a human face. The following uses consecutive first and second frames as an example. Please refer to FIGS. 1, 2 and 3A to 3E together. FIG. 2 shows a flowchart of a target tracking method according to an embodiment of the present invention, and FIGS. 3A to 3E show how to track a target object according to an embodiment of the present invention. Schematic. When the target tracking method of an embodiment of the present invention is started to be executed in step S300, the first frame of picture is first read by the reading unit 110.

Please refer to step S310 to capture the first result tracking frame 10 and the first reference area 30. In detail, in the first frame, the iterative processing unit 120 executes an iterative algorithm according to the first initial tracking frame to capture the first result tracking frame 10 and a first reference area 30. The first result tracking frame 10 includes the target Object, the first reference area 30 is adjacent to the first result tracking frame 10. In practice, the user can set the first initial tracking frame through the target tracking device 100 or by other methods. The iterative algorithm is, for example, the Camshift algorithm (Continue Adaptive Mean Shift Algorithm). After the iterative algorithm is executed, the first result tracking frame 10 containing the face of the person 1 can be obtained, as shown in FIG. 3A. There may be a fixed relative position relationship between the first reference area 30 and the first result tracking frame 10. For example, the first reference area 30 is located under the first result tracking frame 10 and separated by 1/2 the length of the tracking frame, and the first reference area 30 is the same size as the first result tracking frame 10, but it is not limited thereto. The first reference area 30 corresponds to the part in front of the chest of the jacket of the person 1. Generally speaking, the difference in clothing color between different people is greater than the difference in skin color. Therefore, referring to the color of the clothes as an auxiliary face to be tracked can increase the accuracy of the tracking result.

In step S310, the determining unit 130 extracts the first reference feature from the first reference area 30. The first reference feature, for example, represents the color of clothes corresponding to the face to be tracked. The first reference feature is, for example, a histogram of hue and saturation of the first reference area 30 in the HSV color space. In addition, the first reference feature of the first reference area 30 corresponding to the tracking frame 10 obtained by the first result obtained after the iterative algorithm is executed for the first time can be used as the preset reference feature.

Please refer to step S320 to update the target template at least once. In detail, the target pattern (Image Patch) is the feature value of the target object (face). The characteristic value is, for example, the brightness (Luminance), chrominance (Chrominance), density (Chrominance), hue (Hue), saturation (Saturation), or lightness (Saturation) of the target object. When the person's posture is not moving, that is, when the face does not change from a front face to a side face, and the head is not raised or lowered, the target model obtained according to the feature value will basically not change much. In other words, the target template can be used to represent the characteristics of a human face. However, in practice, since the tracked person may not always appear in the image with the front face, the target template needs to be updated to increase the accuracy of subsequent tracking judgments. For example, when there are no other faces in the first frame of the current picture, the target template corresponding to the target object can be updated at regular intervals, but the update timing is not limited to this. In addition, after the target tracking method is executed in step S300 and the target object to be tracked is selected, the target template of the target object can be obtained at the same time as the initial target template.

Please refer to step S330 to capture the second result tracking frame and the second reference area 40. In detail, the second result tracking frame and the second reference area 40 are captured outside the first result tracking frame 10, and the second reference area 40 is adjacent to the second result tracking frame. In practice, similar to the fixed relative position relationship between the first reference area 30 and the first result tracking frame 10, the second reference area 40 and the second result tracking frame also have the same fixed relative position relationship. However, the present invention is not limited to this. Retrieving the second result tracking frame includes the following steps: executing an iterative algorithm to obtain a plurality of second initial tracking frames outside the first result tracking frame 10, as shown in FIG. 3C and FIG. 3D 20a~20h; selecting these second ones in sequence One of the initial tracking frames is calculated, and a confidence index of the selected second initial tracking frame relative to the target object is calculated. The confidence indicator is used to determine whether the selected second initial tracking frame can be used as the second result tracking frame. In practice, the confidence indicator is, for example, a ratio of the plurality of pixel values of the second result tracking frame associated with the target object. In other words, the proportion of skin color in the second result tracking frame is used to determine whether there is a human face in the second result tracking frame. Therefore, when the judging unit 130 judges that the confidence index is less than a threshold value, it selects the other one of the other second initial tracking frames in sequence, and at the same time selectively updates the target template according to a characteristic value of the target object; for example, you can choose not to update the target Template (this is because there is no skin color target around it does not mean that there are no other people around, it may be that two people overlap), or after detecting that there is no skin color target around and lasting for a period of time, choose to update the target template. The situation of the above-mentioned update target template is merely an illustrative description and is not intended to limit the present invention. On the other hand, when the confidence index is greater than or equal to the threshold value, the judgment unit 130 uses the second initial tracking frame selected by it as the second result tracking frame. For example, whenever a second initial tracking frame is selected, it is further calculated whether the percentage of skin color in the entire tracking frame is less than a threshold value, so as to determine whether the selected second initial tracking frame includes a face of sufficient size. As shown in FIGS. 3C to 3E, after calculation one by one, for example, the skin tone ratio of the second result tracking frame 21f obtained through the sixth second initial tracking frame 20f is greater than or equal to the threshold value.

It should be noted that after capturing the second result tracking frame, it is equivalent to there are more than two target objects in the frame at this time, so it is necessary to pause the update of the target template to avoid updating to the face template of the non-original tracker . However, the present invention is not limited to this. In practice, it is also possible to suspend updating the template when the second initial tracking frame is obtained.

Please refer to step S340 to calculate the first and second similarity degrees, and the first and second difference degrees. In detail, after capturing the second result tracking frame, the following steps are executed, including: the determining unit 130 executes a comparison algorithm to calculate the first similarity degree and the second reference area of the first reference area 30 with respect to the preset reference feature 40 The second degree of similarity relative to the preset reference feature. The comparison algorithm is, for example, a correlation coefficient (Correlation) algorithm, a Chi-square algorithm, an intersection (Intersection) algorithm, or a Bhattacharyya distance algorithm. In practice, when the tracked person is obscured, as shown in Figures 3B to 3C, the reference features in the first and second reference areas 40 (equivalent to the upper parts of the characters 1 and 2) can be compared with the preset reference The feature (the coat part of the original tracked person 1) is compared to calculate the feature value of the reference area corresponding to which result tracking frame is closer to the feature value of the preset reference feature, that is, which result is in the tracking frame The coat color is more similar to the coat color of the original tracked person 1.

In addition, step S340 further includes using a template matching algorithm to calculate the first feature value of the first result tracking frame 10, the first degree of difference between the first feature value and the preset feature value, and the second feature of the second result tracking frame Value and a second degree of difference between the second characteristic value and the preset characteristic value, wherein the preset characteristic value is associated with a target template. In practice, template matching algorithms are for example: Mean Absolute Differences (MAD), Sum of Absolute Differences (Sum of Absolute Differences, SAD), Sum of Squared Differences (Sum of Squared Differences, SSD), and Mean Absolute Differences (SSD) Mean Square Differences, MSD), Normalized Cross Correlation (NCC), Sequential Similiarity Detection Algorithm (SSDA) or Sum of Absolute Transformed Difference (SATD). The first and second degree of difference respectively reflect the degree of difference between the face in the tracking frame and the originally preset face template in the first and second results. In an embodiment of the present invention, considering that only relying on the aforementioned degree of similarity may encounter a situation where the correct target object cannot be tracked (for example, two people in the current frame are wearing the same or highly similar clothes), so add The steps to calculate the degree of these two differences can improve the tracking accuracy.

Please refer to step S350 to determine whether the first degree of similarity and the second degree of similarity are equal. In detail, the determining unit 130 first determines whether the person 1 locked by the first result tracking frame 10 and the person 2 locked by the second result tracking frame are both wearing clothes of the same color. If the judgment result is no, please continue to refer to step S370. When the first degree of similarity is not equal to the second degree of similarity, the first result tracking frame 10 corresponding to the larger of the first degree of similarity and the second degree of similarity is used. Or the second result tracking frame is used as the first initial tracking frame of the second frame. That is, the person corresponding to the tracking frame whose clothes color is closer to the original tracking target is used as the follow-up tracking target. Conversely, when the judgment result of step S350 is yes, please continue to refer to step S360, that is, when the first degree of similarity is equal to the second degree of similarity (or when the difference between the two is less than a threshold, for example, the difference is within 5%) If the first and second degrees of similarity are both equal), the first or second result tracking frame corresponding to the smaller of the first degree of difference and the second degree of difference is used as the first initial tracking of the second frame frame. In other words, when the first target tracking frame and the second target tracking frame are respectively locked by two persons, the colors of the clothes of the two persons are quite similar to those of the previously tracked object. Therefore, in step S340, the degree of difference between the face template of the two persons and the original (updated) face template of the tracked person is pre-calculated, and in step S360, the difference degree is smaller, that is, the face in the tracking frame is the result. The one with the smaller difference between the template feature value and the face template feature value of the original tracked person is used as the first initial tracking frame of the second frame. This is shown in step S370.

In summary, in the target tracking method of an embodiment of the present invention, in the initial setting, the face in the tracking frame detected by the face detection is used as the target template, and the clothes color in the reference area is recorded. The reference area can be set to the distance under the face 1/2 the height of the face and the same size as the target tracking frame (it can also be an area of any size sufficient to represent the color of the clothes); while recording the color of the clothes, all the pixels of the target template The eigenvalues are recorded as a face model.

Therefore, when the tracker is fully obscured, other targets that are not obscured will be tracked according to the Camshift algorithm. At this point, you can first judge according to the color of the clothes represented by the target area. If the clothes are of different colors, when the two people are separated again, the second result tracking box will select the target person corresponding to the clothes with the same color or a higher degree of similarity Keep tracking. If the color of the clothes of the two is the same as the original color, continue to compare the difference between the face template of the two result tracking frames and the original target template, and then track the difference of the face template in the next frame The target object corresponding to the smaller one. In addition, in practice, because the face framed by the subsequent first or second result tracking frame may not correspond to the position corresponding to the initial first result tracking frame 10. Therefore, the technique of motion estimation can be used to find the closest location nearby.

Based on the above description, the effect of the present invention is mainly to improve the accuracy of person tracking. Its architecture is based on an iterative algorithm (such as Camshift), and then uses the surrounding result tracking frame as the starting point to predict the possible tracking person , And use the judgment of the color of the clothes as an aid. When the color of the clothes is the same, continue to compare the face models as the basis for judgment. In this way, when the color of the clothes is the same, the person who originally wanted to be tracked can still be tracked correctly. At the same time, the face template will be updated when it is judged that there are no other obscured people in the current frame, so that when the face of the tracking object moves to different angles, the accuracy of identifying the difference degree with the face template can still be maintained.

Although the present invention is disclosed in the foregoing embodiments, it is not intended to limit the present invention. All changes and modifications made without departing from the spirit and scope of the present invention fall within the scope of patent protection of the present invention. For the scope of protection defined by the present invention, please refer to the attached patent scope.

100: Target tracking device 110: Reading unit 120: Iterative processing unit 130: Judgment Unit 1, 2: characters 10: The first result tracking box 20a~20h: The second initial tracking frame 21f: The second result tracking box 30: The first reference area 40: second reference area S330~S370: steps

FIG. 1 is a block diagram of a target tracking device according to an embodiment of the invention. FIG. 2 is a flowchart of a target tracking method according to an embodiment of the invention. 3A to 3E are schematic diagrams of tracking target objects according to an embodiment of the present invention.

S300~S370: Steps of the target tracking method

Claims (10)

A target tracking method is suitable for tracking a target object in a continuous first frame and a second frame. The target tracking method includes: in the first frame, according to a first initial tracking frame An iterative algorithm is executed to extract a first result tracking frame and a first reference area, the first result tracking frame includes the target object, the first reference area is adjacent to the first result tracking frame; in the first result Extracting a second result tracking frame and a second reference area outside the tracking frame, the second reference area being adjacent to the second result tracking frame; and after capturing the second result tracking frame, including: performing a comparison calculation Method to calculate a first degree of similarity between the first reference area and a predetermined reference feature and a second degree of similarity between the second reference area and a predetermined reference feature; and calculate the first degree of similarity with the same plate matching algorithm A first feature value of a result tracking frame, a first degree of difference between the first feature value and a predetermined feature value, a second feature value of the second result tracking frame, and the second feature value and the prediction Set a second degree of difference of the characteristic value; wherein the preset characteristic value is associated with a target template of the target object; when the first degree of similarity is not equal to the second degree of similarity, the first degree of similarity and the The first result tracking frame or the second result tracking frame corresponding to the larger of the second degree of similarity is used as the first initial tracking frame of the second frame; or when the first degree of similarity is the same as the second When the degree of similarity is equal, the first result tracking frame or the second result tracking frame corresponding to the smaller of the first difference degree and the second difference degree is used as the first initial tracking of the second frame frame. The target tracking method according to claim 1, wherein the first characteristic value, the second characteristic value and the predetermined characteristic value are a brightness, a chroma, a density, a hue, and a saturation of the target object , Or a lightness. The target tracking method according to claim 1, wherein before capturing the second result tracking frame, it further comprises: updating the preset characteristic value of the target template at least once; and after capturing the second result tracking frame , And include: Pause to update the target template. The target tracking method according to claim 1, wherein capturing a second result tracking frame includes: executing the iterative algorithm to obtain a plurality of second initial tracking frames outside the first result tracking frame; and sequentially selecting the One of the second initial tracking frames is calculated, and a confidence indicator of the selected second initial tracking frame relative to the target object is calculated; wherein when the confidence indicator should be less than a threshold value, the second The other one in the initial tracking frame, and the target template is updated at least once according to a characteristic value of the target object; when the confidence index is greater than or equal to the threshold value, the selected second initial tracking frame is used as the first 2. Results tracking frame, and suspend updating the target template. The target tracking method according to claim 3, wherein the confidence indicator is a ratio of a plurality of pixel values of the second result tracking frame associated with the target object. The target tracking method according to claim 1, wherein the first reference area and the first result tracking frame have a fixed relative positional relationship, and the second reference area and the second result tracking frame have the Fixed relative position relationship. The target tracking method according to claim 1, wherein the target object is a human face. The target tracking method according to claim 1, wherein the iterative algorithm is the Camshift algorithm. The target tracking method according to claim 1, wherein the comparison algorithm is a correlation coefficient (Correlation) algorithm, Chi-square algorithm, Intersection algorithm, and Bhattacharyya distance) one of the algorithms. The target tracking method as described in claim 1, the template matching algorithm is Mean Absolute Differences (MAD), Sum of Absolute Differences (Sum of Absolute Differences, SAD), and Sum of Squared Differences (Sum of Squared Differences, SSD) ), Mean Square Differences (MSD), Normalized Cross Correlation (NCC), Sequential Similiarity Detection Algorithm (SSDA) or Sum of Absolute Transformed Difference, SATD).

Images