TW201828158A - Method of video object tracking and apparatus thereof - Google Patents

Abstract

A method of video object tracking and an apparatus are provided. The method comprises following steps. Video streams are acquired. The face area is recognized according to the face detection algorithm, and a first to-be-tracked target corresponding to the first video frame is obtained. The first to-be-tracked target is extracted through the face features to obtain the first face features, and the first face features are stored in a feature library corresponding to the target to be tracked. The face region is recognized in the current video frame for obtaining a current to-be-tracked target corresponding to the current video frame. The current to-be-tracked target is extract through the face features to obtain second face features. The current to-be-tracked target is matched with the first to-be-tracked target according to the second face feature and the feature library, so that the first to-be-tracked target is tracked from the first video frame. The feature library is updated according to the extracted updated face features in the tracking process.

Description

   Video target tracking method and device   

The present application relates to the field of computer technology, and in particular, to a video target tracking method and device. This application claims priority from a Chinese patent application filed on January 17, 2017, with application number 201710032132.6, and the invention name is "Video Target Tracking Method and Device", the entire contents of which are incorporated herein by reference. .

Target tracking technology has always been one of the focuses in the field of computer vision and image processing, and is widely used in intelligent monitoring, intelligent transportation, visual navigation, human-machine interaction, and defense reconnaissance.

Target tracking algorithms usually use one or several simple traditional feature matching algorithms to distinguish targets, such as using the color, shape and other features of the image itself.

The embodiments of the present application provide a video target tracking method and device, which can improve tracking continuity and robustness.

An embodiment of the present application provides a method for video target tracking, which is applied to a terminal or a server. The method includes: acquiring a video stream, identifying a face area according to a face detection algorithm, and obtaining a first to-be-tracked corresponding to a first video frame. A target; a first facial feature is obtained by extracting a facial feature based on a deep neural network on the first target to be tracked, and the first facial feature is stored in a feature library corresponding to the first target to be tracked ; Identify the face area according to the face detection algorithm in the current video frame, obtain the current target to be tracked corresponding to the current video frame, and obtain a second face from the deep neural network-based face feature extraction for the current target to be tracked Feature, matching the current target to be tracked with the first target to be tracked according to the second face feature and the feature library to track the first target to be tracked from the first video frame, During the tracking process, the feature library is updated according to the extracted updated face features.

An embodiment of the present application further provides a video target tracking device. The device includes a processor and a memory connected to the processor. The memory stores a machine-readable instruction module executable by the processor. The machine-readable instruction module includes: a detection module for obtaining a video stream, identifying a face area according to a face detection algorithm, and obtaining a first target to be tracked corresponding to a first video frame; and extracting facial features A module for extracting a first facial feature from the first neural network-based facial feature extraction for the first target to be tracked, and storing the first facial feature into the first target to be tracked Feature library; the detection module is further configured to identify a face area according to a face detection algorithm in the current video frame to obtain a current target to be tracked corresponding to the current video frame; the face feature extraction module is further configured to A second facial feature is obtained by extracting a facial feature based on a deep neural network from the current target to be tracked; a tracking module is configured to convert the target according to the second facial feature and the feature database. Feature matching between the former target to be tracked and the first target to be tracked to track the first target to be tracked from the first video frame; a learning module for tracking the updated facial features based on the extracted features Update the feature library.

An embodiment of the present application further provides a non-volatile computer-readable storage medium. The storage medium stores machine-readable instructions, and the machine-readable instructions can be executed by a processor to complete the following operations: acquiring a video stream, according to The face detection algorithm recognizes a face region, and obtains a first target to be tracked corresponding to the first video frame. The first target to be tracked is obtained through facial feature extraction based on a deep neural network, and Storing the first face feature into a feature library corresponding to the first target to be tracked; identifying a face area according to a face detection algorithm at the current video frame, obtaining the current target to be tracked corresponding to the current video frame, The current target to be tracked obtains a second face feature through facial feature extraction based on a deep neural network, and the current target to be tracked and the first target to be tracked are performed according to the second face feature and the feature library. Feature matching to track the first target to be tracked from the first video frame, and update the features according to the extracted updated face features during the tracking process Library.

110‧‧‧Terminal

120‧‧‧Server

130‧‧‧video capture device

140‧‧‧Internet

1101‧‧‧System Bus

1102‧‧‧Processor

1103‧‧‧Graphics Processing Unit

1104‧‧‧Storage media

1105‧‧‧Memory

1106‧‧‧Interface

1107‧‧‧display

1108‧‧‧Input device

11041‧‧‧Operating System

11042‧‧‧The first video target tracking device

1201‧‧‧System Bus

1202‧‧‧Processor

1203‧‧‧Storage media

1204‧‧‧Memory

1205‧‧‧Interface

12031‧‧‧ Operating System

12032‧‧‧Database

12033‧‧‧Second video target tracking device

310‧‧‧Tracking Module

320‧‧‧Detection Module

330‧‧‧Learning Module

410‧‧‧testing module

411‧‧‧Image Feature Extraction Unit

412‧‧‧Identity matching unit

413‧‧‧First tracking target determination unit

414‧‧‧The first recommended unit

415‧‧‧Second recommended unit

416‧‧‧Second tracking target determination unit

420‧‧‧Face feature extraction module

430‧‧‧Tracking Module

440‧‧‧Learning Module

450‧‧‧ Feature Identity Processing Module

510‧‧‧ processor

520‧‧‧Memory

521‧‧‧Detection Module

522‧‧‧Face feature extraction module

523‧‧‧Tracking Module

524‧‧‧Learning Module

525‧‧‧ Feature Identity Processing Module

530‧‧‧Interface

AdaBoost‧‧‧ Iterative Algorithm

conv‧‧‧ convolution layer

FC‧‧‧ means fully connected layer

I (x, y, t) ‧‧‧ primitive

(dx, dy) ‧‧‧distance

dt‧‧‧time

LRN‧‧‧Regional Response Normalization Layer

max pool‧‧‧Maximum pooling layer

NPD‧‧‧normalized feature differences

ROM‧‧‧Read Only Memory

RAM‧‧‧ random storage memory

S210, S220, S230‧‧‧ steps

S231, S232, S233, S234, S235, S236, S237‧‧‧ steps

TLD‧‧‧ single target long-term tracking

VGG‧‧‧Visualized Geometric Group

VGG-S‧‧‧Face Feature Extraction Algorithm

In order to explain the technical solution in the embodiments of the present invention more clearly, the drawings used in the description of the embodiments are briefly introduced below: FIG. 1 is an application environment diagram of a video target tracking method in an embodiment of the present application; FIG. 2 is an internal structure diagram of the terminal in FIG. 1 in an embodiment of the present application; FIG. 3 is an internal structure diagram of the server in FIG. 1 in an embodiment of the present application; FIG. 4 is a video target tracking method in an embodiment of the present application Fig. 5 is a flowchart of obtaining a current target to be tracked in one embodiment of the application; Fig. 6 is a flowchart of updating a feature database in one embodiment of the application; Fig. 7 is a video target tracking in one embodiment of the application Schematic comparison between algorithm and template matching algorithm; Figure 8 is another flowchart of obtaining the current target to be tracked in one embodiment of the application; Figure 9 is a schematic diagram of the target tracking system corresponding to the video target tracking method in one embodiment of the application ; FIG. 10 is a schematic diagram of a video tracking result obtained by a video target tracking algorithm in an embodiment of the present application; FIG. 11 is a TLD following Schematic diagram of video tracking results obtained by the tracking algorithm; FIG. 12 is a schematic diagram of the structure of a video target tracking device in an embodiment of the application; FIG. 13 is another schematic diagram of the structure of a video target tracking device in an embodiment of the application; Another schematic diagram of the structure of the video target tracking device in one embodiment of the application; FIG. 15 is another schematic diagram of the structure of the video target tracking device in one embodiment of the application; FIG. 16 is another schematic diagram of the video target tracking device in one embodiment of the application Schematic.

Please refer to the drawings, wherein the same component symbols represent the same components or similar components. The principle of the present invention is exemplified by being implemented in a suitable computing environment. The following description is based on the exemplified specific embodiments of the present invention, which should not be construed as limiting other specific embodiments of the present invention which are not described in detail herein.

FIG. 1 is an application environment diagram of a video target tracking method running in an embodiment of the present application. As shown in FIG. 1, the application environment includes a terminal 110, a server 120, and a video capture device 130. The terminal 110, the server 120, and the video capture device 130 communicate through a network 140.

In some embodiments of the present application, the terminal 110 may be a smart phone, a tablet computer, a notebook computer, a desktop computer, etc., but is not limited thereto. The video capture device 130 may be a camera, which is arranged at a position such as an entrance of a building. The network 140 may be a wired network or a wireless network. In some embodiments of the present application, the video capture device 130 may send the collected video stream to the terminal 110 or the server 120, and the terminal 110 or the server 120 may perform target tracking on the video stream. In other embodiments of the present application, the video capture device 130 may also directly perform target tracking on the video stream, and send the tracking result to the terminal 110 for display.

In an embodiment of the present application, the internal structure of the terminal 110 in FIG. 1 is shown in FIG. 2. The terminal 110 includes a processor 1102, a graphics processing unit 1103, a storage medium 1104, and a memory 1105 connected through a system bus 1101. , Network interface 1106, display screen 1107, and input device 1108. The storage medium 1104 of the terminal 110 stores an operating system 11041 and a first video target tracking device 11042. The device 11042 is configured to implement a video target tracking method suitable for the terminal 110. The processor 1102 is used to provide computing and control capabilities to support the operation of the entire terminal 110. The graphics processing unit 1103 in the terminal 110 is configured to provide at least a drawing capability of a display interface. The memory 1105 provides an environment for the operation of the first video object tracking device 11042 in the storage medium 1104. The network interface 1106 is configured to perform network communication with the video capture device 130, such as receiving a video stream collected by the video capture device 130. A display screen 1107 is used to display tracking results and the like. The input device 1108 is used to receive commands or data input by a user. For the touch-sensitive terminal 110, the display screen 1107 and the input device 1108 may be touch-sensitive. The structure shown in FIG. 2 is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the terminal 110 to which the solution of the present application is applied. The specific terminal 110 may include Show more or fewer components, or combine certain components, or have different component arrangements.

In an embodiment of the present application, the internal structure of the server 120 in FIG. 1 is shown in FIG. 3. The server 120 includes a processor 1202, a storage medium 1203, a memory 1204, and a network interface connected through a system bus 1201. 1205. The storage medium 1203 of the server 120 stores an operating system 12031, a database 12032, and a second video target tracking device 12033. The database 12032 is used for storing data. The second video target tracking device 12033 is configured to implement a video target tracking method suitable for the server 120. The processor 1202 of the server 120 is used to provide computing and control capabilities to support the operation of the entire server 120. The memory 1204 of the server 120 provides an environment for the operation of the second video target tracking device 12033 in the storage medium 1203. The network interface 1205 of the server 120 is used to communicate with an external video capture device 130 through a network connection, such as receiving a video stream sent by the video capture device 130.

As shown in FIG. 4, in an embodiment of the present application, a video target tracking method is provided, which is applied to the terminal 110, the server 120, or the video capture device 130 in the above application environment, and the method may be implemented by any one of the present application. The video target tracking device provided by the example is executed and includes the following steps:

Step S210: Obtain a video stream, identify a face region according to a face detection algorithm, and obtain a first target to be tracked corresponding to a first video frame.

Specifically, the video stream can be acquired by a video acquisition device distributed at the entrance of the building. If the video target tracking method is applied to a video capture device, the video stream can be obtained directly from the memory of the video capture device. If the video target tracking method is applied to a terminal or a server, the video capture device can immediately send the captured video stream to the terminal or the server.

Face detection refers to searching for a given image using a certain strategy to determine whether it contains a face, and if so, returning the position, size and pose of the face. In some embodiments of the present application, a face area (such as a rectangular frame shown in FIG. 10) may be displayed in the form of a recommended frame to obtain a first target to be tracked corresponding to the first video frame. Face detection is performed on the video stream until a human face is detected, and the face area is determined as the first target to be tracked. Since multiple faces may be detected in one frame, there may be multiple first targets to be tracked. If there are multiple first targets to be tracked, different face areas may be identified through different identification information, for example, different face areas may be identified through different color recommendation frames. Face detection algorithms can be customized as needed, such as using NPD (Normalized Pixel Difference) face detection algorithms, or combining NPD face detection algorithms with other algorithms to improve determination The accuracy of the target to be tracked.

In step S220, the first facial feature is extracted from the facial feature based on the deep neural network for the first target to be tracked, and the first facial feature is stored in a feature database corresponding to the first target to be tracked.

Specifically, the deep neural network is a machine learning model under deep learning. Deep learning is a branch of machine learning. It is a high-level abstraction algorithm that uses multiple processing layers that contain complex structures or consist of multiple non-linear transformations. Deep neural networks can use the Visual Geometry Group (VGG) network structure. The VGG network structure has a higher recall and accuracy rate than distinguishing targets by feature matching algorithms.

Assign a target identifier to the first target to be tracked and establish a feature database, establish an association relationship between the target identifier and the feature database, and save the association relationship. When there are multiple first to-be-tracked targets, a target identifier can be assigned to each first to-be-tracked target and a feature database can be established, and an association relationship can be established between each first to-be-tracked target and its corresponding first face feature. The association relationship and the first face feature are stored in a feature library corresponding to the first target to be tracked. By referencing face features for feature matching, the problem that the target tracking algorithm does not make good use of the face features, so frequent tracking errors, follow-up errors, and follow-up losses cannot be retrieved correctly after the tracking target is solved.

Step S230: Identify the face area according to the face detection algorithm in the current video frame, obtain the current target to be tracked corresponding to the current video frame, and obtain a second face by extracting facial features based on the deep neural network for the current target to be tracked. Feature, matching the current target to be tracked with the first target to be tracked according to the second face feature and the feature database to track the first target to be tracked from the first video frame, and according to the extracted updated person during the tracking process Face features update feature library.

Specifically, feature matching is performed on the second face feature and each first face feature corresponding to the first target to be tracked in the feature database. The specific algorithm for feature matching can be customized. For example, the Euclidean distance of the vector corresponding to the facial features can be directly calculated, and whether the matching can be determined based on the Euclidean distance. If the second facial feature matches the first facial feature successfully, it is determined that the current target to be tracked is a continuous moving target of the first target to be tracked. If there are multiple current targets to be tracked, each current target group to be tracked constitutes the current target set to be tracked, and the second face features corresponding to each current target to be tracked in the current target set to be tracked are respectively stored in the feature database. The facial features corresponding to each historical target to be tracked are matched. If the matching is successful, the target identifier of the historical target to be tracked is used as the target identifier of the current target to be tracked, and the position of the current target to be tracked is the position of the historical target to be tracked after movement. .

In some embodiments of the present application, the feature database may be updated during the tracking process based on the extracted updated face features. For example, when the lighting continuously changes or the face is updated, updated faces of the first target to be tracked in other frames may be obtained. Feature, if the updated face feature is different from the first face feature, the updated face feature with the difference may be added to a feature library corresponding to the first target to be tracked, and the updated face feature and the first face feature may be different. Establish an association relationship between the target identifiers of the target to be tracked, and store the association relationship in a feature database, so that when the first target to be tracked has a larger-angle side face or a larger light intensity change in other frames, Feature matching between the second face feature corresponding to the current target to be tracked and the updated face feature of the first target to be tracked is smaller than when the feature is directly matched with the first face feature, thereby increasing feature matching The probability of success reduces the sensitivity of the target tracking process to changes in the tracking target, tilt, cover, and light changes, and improves the continuity and robustness of the tracking. Moreover, a large number of facial features corresponding to the first target to be tracked in different frames can be saved through the feature database. In the case that the first target to be tracked disappears and then appears, the feature library corresponding to the first target to be tracked can be used to save Feature matching is performed on the facial features before the first target to be tracked disappears, so as to achieve a good tracking effect on the intermittently appearing targets. Updating the feature database is to update a positive and negative sample database through tracking and detection, which is equivalent to a half-line tracking algorithm. Compared with a completely offline tracking algorithm, it has a better recall rate. Compared to a completely online tracking algorithm, Can show higher accuracy.

In the embodiment of the present application, by obtaining a video stream and identifying a face region according to a face detection algorithm, a first target to be tracked corresponding to a first video frame is obtained, and a person based on a deep neural network is used to pass the first target to the tracked person. Face features are extracted to obtain the first face feature, and the first face feature is added to the feature database. The face area is identified according to the face detection algorithm in the current video frame, and the current target to be tracked corresponding to the current video frame is obtained. The target to be tracked obtains a second face feature through facial feature extraction based on a deep neural network, and the current target to be tracked is matched with the first target to be tracked according to the second face feature and the feature library, so that A video frame starts to track the first target to be tracked. During the tracking process, the feature database is updated based on the extracted updated face features, and feature matching is performed by referring to deep neural network-based face features. Make good use of facial features, frequently failing to retrieve the tracking target correctly after following errors, follow-ups, and follow-ups. It saves the resources of the terminal or server equipment, and improves the processing speed of the processor of the terminal or server. At the same time, the feature database is continuously updated during the tracking process, which can save different facial features corresponding to the target to be tracked in different states, thereby improving the success rate of facial feature matching, reducing the changes, tilt, The sensitivity of cover and light changes improves the continuity and robustness of the tracking, thereby increasing the processing speed of the processor of the terminal or server.

In an embodiment of the present application, the method further includes: identifying the facial identity information corresponding to each target to be tracked through a face recognition algorithm according to the face state of each target to be tracked, and extracting the algorithm through image features The target features corresponding to the facial identity information are obtained.

In some embodiments of the present application, the state of the face refers to the state of the deflection angle of the face. When a human face is detected as a positive face, the corresponding face identity information can be obtained through recognition by a face recognition algorithm. Face identity information is used to describe the identity corresponding to the face. Face recognition refers to searching and matching feature information of the extracted face image with a feature template stored in a database, such as a face feature template, and determining face identity information based on similarity. For example, when performing face recognition on employees who enter the enterprise, a feature template of each employee in the enterprise, such as a face feature template, is stored in advance in the database, so that the feature data and database of the currently extracted face image are stored in the database. The facial feature templates stored in the comparison are used to obtain the employee's facial identity information. The specific content of the face identity information can be customized according to needs, such as employee name, job number, and department.

The image feature extraction algorithm extracts feature data based on the features of the image itself, such as color features, texture features, shape features, and spatial relationship features, to obtain the target features, where the target features are all the feature data extracted. set. Establish associations between target features and face identity information, such as clothing color, clothing texture, body shape, height ratio, and other characteristics, and store the associations in the database. In this way, when the face is deflected and covered, other target features can be used to identify the identity and determine the face area. In an embodiment of the present application, as shown in FIG. 5, in step S230, the face region is identified according to the face detection algorithm in the current video frame, and the steps of obtaining the current target to be tracked corresponding to the current video frame include:

In step S231, it is determined whether the current video frame recognizes a face area based on the face detection algorithm. If no face area is recognized, the current video frame corresponding to the current video frame is obtained according to the image feature extraction algorithm.

Specifically, if the facial area is not recognized in the current video frame according to the face detection algorithm, the detection may also be due to the side of the face. At this time, the algorithm corresponding to the current image frame needs to be obtained to obtain the corresponding video frame. The current image feature.

Step S232: Compare the current image features with the target features to obtain matching target face identity information, and obtain the current target to be tracked corresponding to the current video frame according to the target face identity information.

Specifically, since the target feature has been associated with face identity information before, the current image feature can be compared with the target feature to calculate the similarity. If the similarity exceeds the threshold, the matching is successful and the corresponding target feature correspondence can be obtained. Target face identity information of, so as to obtain the current target to be tracked corresponding to the current video frame according to the target face identity information. Then, the current target to be tracked is matched with the first target to be tracked through the face identity information, so that the first target to be tracked is tracked.

In the embodiment of the present application, face identity information is introduced into target tracking, and image features are combined with face recognition, and the target tracking can be achieved when the face detection algorithm cannot identify the face area, which further improves the tracking accuracy. Continuity and robustness.

In an embodiment of the present application, step S220 may include: acquiring first facial identity information corresponding to the first target to be tracked, establishing a first facial feature set corresponding to the first facial identity information, and integrating the first facial feature Adding the first face feature set and storing the first face feature set to a feature library corresponding to the first target to be tracked.

Specifically, face recognition may be performed on the first target to be tracked to obtain first face identity information corresponding to the first target to be tracked. The first face feature set is used to store the first face feature of the first target to be tracked in different states during the movement, and the different states include different angles, different illuminations, different coverage ranges, and the like. The first facial feature obtained after the extraction of facial features is added to the first facial feature set, and an association relationship is established between the first facial feature set and the first facial identity information, and the association relationship and the first The facial feature set is stored in a feature library corresponding to the first target to be tracked.

In an embodiment of the present application, as shown in FIG. 6, the step of updating the feature database according to the extracted updated face features in the tracking process in step S230 may include:

Step S233: Acquire the current facial identity information corresponding to the current target to be tracked, and acquire the first facial feature set corresponding to the current facial identity information from the feature database.

Specifically, in one embodiment, the current face identity information corresponding to the current target to be tracked may be obtained by performing face recognition on the current target to be tracked. In another embodiment, the current image feature corresponding to the current target to be tracked can also be obtained by applying an image feature extraction algorithm to the current target to be tracked, and then the current image feature and the target feature are matched to match the matched target. The facial identity information corresponding to the feature is used as the current facial identity information, so that the current facial identity information can also be obtained when the target area to be tracked cannot recognize the facial area. According to the association between the facial identity information and the facial feature set, the first facial feature set corresponding to the current facial identity information is obtained, indicating that the current target to be tracked and the first target to be tracked are the same target.

Step S234: Calculate the difference between the first face feature and the second face feature in the first face feature set. If the difference exceeds a preset threshold, add a second face feature to the first face feature set .

Specifically, a custom algorithm may be used to calculate the difference between the second face feature and the first face feature in the first face feature set. If there are multiple first facial features in the first facial feature set, the difference between the second facial feature and each first facial feature is calculated separately to obtain multiple differences. The amount of difference indicates the difference between the second face feature and the face feature of the same tracking target that has been saved in the feature database. A larger difference indicates a greater change in the face state of the tracking target. If the difference exceeds a preset threshold, a second face feature is added to the first face feature set, and the added second face feature can be used for subsequent feature matching. The more face features stored in the face feature set, the more it can represent the features of the same tracking target in different states. As long as any one of the features can be successfully matched when the features are matched, the current target to be tracked is considered to be the first The matching of the target to be tracked is successful, thereby increasing the probability of successful matching, reducing the sensitivity of the target tracking process to changes in the tracking target, tilt, cover, and lighting changes, and improving the continuity and robustness of tracking.

In an embodiment of the present application, step S220 may include: performing facial feature extraction on the first target to be tracked through a deep neural network to obtain a first feature vector.

Specifically, a facial feature extraction model is obtained after training a deep neural network, and the first feature vector is obtained by inputting the primitive value corresponding to the target to be tracked, and the dimension of the first feature vector is determined by the facial feature extraction model. .

Step S230 includes: performing facial feature extraction on the current target to be tracked through a deep neural network to obtain a second feature vector, and calculating the Euclidean distance between the first feature vector and the second feature vector. If the Euclidean distance is less than a preset threshold, then It is determined that the characteristics of the first target to be tracked and the current target to be tracked are successfully matched.

Specifically, by inputting the primitive value corresponding to the current target to be tracked into the aforementioned facial feature extraction model, a second feature vector can be obtained. The Euclidean distance between the first feature vector and the second feature vector represents the similarity between the current target to be tracked and the first target to be tracked. If the Euclidean distance is less than a preset threshold, it is determined that the characteristics of the current target to be tracked and the first target to be tracked are successfully matched, indicating that the current target to be tracked is the same target as the first target to be tracked, and the tracking purpose is achieved.

In an embodiment of the present application, the network structure of the deep neural network may be an 11-layer network layer, including a stacked convolutional neural network and a fully connected layer. The stacked convolutional neural network routes multiple Convolution layer and maxpool layer, the specific network structure is: conv 3-64 * 2 + LRN + max pool

conv 3-128 + max pool

conv 3-256 * 2 + max pool

conv 3-512 * 2 + max pool

conv 3-512 * 2 + max pool

FC 2048

FC 1024, where conv 3 represents a convolution layer with a radius of 3, LRN represents the LRN layer, "maxpool" represents the max pooling layer, and FC represents a fully connected layer. In an embodiment of the present application, the LRN (Local Response Normalization) layer is defined as a local response normalization layer, which is used for normalization processing. In an embodiment of the present application, the maximum pooling layer is defined as a pattern that is non-overlapping and divided into several small blocks of the same size (pooling size). Only the largest number is taken in each small block, and other nodes are discarded. Keep the original plane structure.

Specifically, this network structure is a simplified deep neural network VGG network structure, where 64 * 2 represents 2 64 groups, the LRN layer is a parameterless layer to help training, and FC2048 represents a full connection of 2048-dimensional vectors. Layer, the output of the last fully connected layer FC1024 is the face feature obtained by feature extraction, which is a 1024-dimensional vector. The optimized facial features obtained through the simplified VGG network structure perform much better on the random block matching of the test set than the matching performance of the matching module in the TLD (Tracking-Learning-Detection). , And greatly improve the efficiency of facial feature extraction, and achieve the instantaneousness required by the tracking algorithm. In one embodiment of the present application, the resolution of the target to be tracked can be controlled to be 112 * 112 graphics primitives to reduce the computational complexity. FIG. 7 is a schematic diagram of the matching comparison between the facial feature extraction algorithm VGG-S and the template matching algorithm (match template) corresponding to the VGG network structure. As shown in Figure 7, the abscissa represents the recall rate and the ordinate represents the accuracy rate. It can be seen that the facial feature extraction algorithm corresponding to this VGG network structure has a better accuracy rate when performing feature matching, which improves the accuracy of target tracking rate.

In an embodiment of the present application, in step S230, the step of identifying a face region based on a face detection algorithm in the current video frame to obtain a current target to be tracked corresponding to the current video frame may include: based on normalized feature differences of the primitives And the human body half body recognition algorithm recognizes the face area in the current video frame, and obtains the current target to be tracked corresponding to the current video frame.

Specifically, face detection is performed based on a normalized pixel difference feature (NPD), and the obtained return value is used as a face region recommendation frame. For example, AdaBoost (iterative algorithm) can be used to construct the NPD feature A strong classifier is used to recognize and distinguish faces. The human body half body recognition algorithm can be defined as required, and upper body detection can be performed. The face area recommendation frame can be filtered based on the upper body detection, which can filter out some of the incorrectly recognized face area recommendation frames, which greatly improves the recall rate and Accuracy improves the overall performance of target tracking.

In an embodiment of the present application, as shown in FIG. 8, in step S230, the face region is identified according to the face detection algorithm in the current video frame, and the step of obtaining the current target to be tracked corresponding to the current video frame may include:

In step S235, a face region is identified based on the normalized feature differences of the primitives, and a first recommended region is obtained in the current video frame.

In step S236, a second recommended region corresponding to the first target to be tracked in the current video frame is obtained according to an optical flow analysis algorithm.

Specifically, the optical flow analysis algorithm assumes the light intensity of a picture element I ( x , y , t ) in the first frame, which moves a distance of ( dx , dy ) to the next frame, using dt time. Because the primitive points are the same, the light intensity has not changed. According to the historical trajectory of the first target to be tracked, the vector velocity model corresponding to the first target to be tracked is calculated using the optical flow analysis principle. The current video frame and the previous frame of the current video frame and the first target to be tracked are input to the vector speed model At the position of the previous frame, a second recommended area corresponding to the first target to be tracked in the current video frame, that is, the position where the first target to be tracked may appear in the current video frame.

Step S237: Obtain the current target to be tracked according to the first recommendation area and the second recommendation area.

Specifically, the second recommended area obtained according to the optical flow analysis algorithm is an area to which the first target to be tracked may move based on the historical motion speed, and the distance between the position of the second recommended area and the distance of the second recommended area may exceed the predetermined Set the first recommendation area of the range to get the current target to be tracked. The first recommendation area and the second recommendation area may also be used as the current target to be tracked. If there are multiple first targets to be tracked, each first target to be tracked has a corresponding second recommendation area.

In this embodiment, the normalized feature differences of the picture elements are combined with the optical flow analysis algorithm to obtain the current target to be tracked, because the addition of a priori information makes the accuracy rate increase in subsequent feature matching.

In one embodiment, step S237 may include: performing motion prediction according to the inter-frame correlation to obtain an expected motion range, and filtering the first recommendation region and the second recommendation region according to the expected motion range to obtain a current target to be tracked.

Specifically, inter-frame correlation uses historical position information and motion trajectories to predict the position of the target in the next frame or frames, which is equivalent to using prior information to adjust the credibility of the NPD algorithm. The first recommendation area and the second recommendation area outside the expected motion range are filtered to obtain the current target to be tracked, which reduces the number of matching of subsequent feature matching calculations, and improves the matching efficiency and accuracy.

In an embodiment of the present application, the video target tracking method may complete video target tracking through three modules as shown in FIG. 9, including a tracking module 310, a detection module 320, and a learning module 330. Specifically, a video stream is obtained, a face area is identified according to a face detection algorithm, a first target to be tracked corresponding to the first video frame is obtained, and tracking is started from the video frame where the first target to be tracked is located. A target to be tracked obtains a first face feature by extracting a face feature based on a deep neural network, and adds the first face feature to a feature database. The learning module 330 updates the feature database according to the tracking situation, and the detection module 320 Constantly find better current targets to be tracked from the current video frame to prevent errors and follow-ups. The tracking module 310 matches the current target to be tracked with the first target to be tracked according to the updated feature library to track the first Target to be tracked.

In an embodiment of the present application, a schematic diagram of a tracking area obtained by using the foregoing video target tracking method may be shown in FIG. 10, and a schematic diagram of a tracking area obtained by using a TLD tracking algorithm may be shown in FIG. 11. By comparison, it can be found that when the face is sideways, the tracking area of the video target tracking method proposed in the embodiment of the present application is more accurate than the tracking area of the TLD tracking algorithm, and the TLD tracking algorithm will appear when the face is completely deflected The tracking failure phenomenon, and the video target tracking method proposed in the embodiment of the present application can still track success when the human face is completely deflected. The accuracy rate and recall rate are improved compared to the TLD tracking algorithm. The specific information is as follows: No head detection version: The accuracy rate is increased by about 5 percentage points, the error rate is reduced by 100%, and the target tracking loss rate is reduced by 25%.

Human head detection version: the accuracy rate is increased by about 1 percentage point, the error rate is reduced by 100%, and the target tracking loss rate is reduced by 15%.

In terms of performance, for example, at a resolution of 640 * 480, a CPU with a 3.5G frequency and a Nvidia GeforceGtx 775m graphics card model machine, the single frame processing time is about 40ms, and the frame rate is above 25FPS (frames per second).

The above video target tracking method is more accurate than the traditional method, which provides the possibility and convenience for the follow-up personnel flow statistics, identity recognition and behavior analysis. The good performance also meets the needs of online processing and improves the monitoring and analysis system. Accuracy, scalability, and applicability have further improved the processing speed of the hardware processor and the processing performance of the processor.

In an embodiment of the present application, as shown in FIG. 12, a video target tracking device is provided. The device may include a detection module 410 for acquiring a video stream, identifying a face area according to a face detection algorithm, and obtaining The first target to be tracked corresponding to the first video frame.

A face feature extraction module 420 is configured to obtain a first face feature by extracting a face feature based on a deep neural network on the first target to be tracked, and store the first face feature in the first A feature library corresponding to the target to be tracked.

The detection module 410 is further configured to identify a face area according to a face detection algorithm in a current video frame, and obtain a current target to be tracked corresponding to the current video frame.

The facial feature extraction module 420 is further configured to obtain a second facial feature by extracting a facial feature based on a deep neural network for the current target to be tracked.

The tracking module 430 is configured to perform feature matching between the current target to be tracked and the first target to be tracked according to the second face feature and the feature library, so as to track the first target to be tracked from the first video frame.

A learning module 440 is configured to update the feature database according to the extracted updated face features during the tracking process.

In an embodiment of the present application, as shown in FIG. 13, the device further includes a feature identity processing module 450 configured to obtain corresponding face identity information through a face recognition algorithm according to the face state of the target to be tracked. According to the image feature extraction algorithm, a target feature corresponding to the facial identity information is obtained, and an association relationship is established between the target feature and the facial identity information.

The detection module 410 may include: an image feature extraction unit 411, configured to determine whether a face area is recognized according to a face detection algorithm in the current video frame; if no face area is identified, an algorithm is extracted based on the image features Get the current image feature corresponding to the current video frame.

An identity matching unit 412 is configured to compare current image features with target features to obtain matching target face identity information based on the association relationship.

The first tracking target determining unit 413 is configured to obtain a current target to be tracked corresponding to the current video frame according to the target face identity information.

In an embodiment of the present application, the facial feature extraction module 420 is further configured to obtain first facial identity information corresponding to the first target to be tracked, establish a first facial feature set corresponding to the first facial identity information, and The first face feature is added to the first face feature set and the first face feature set is stored in the feature database.

The learning module 440 is also used to obtain the current face identity information corresponding to the current target to be tracked, obtain the first face feature set corresponding to the current face identity information from the feature library, and calculate the first person in the first face feature set. The amount of difference between the face feature and the second face feature. If the difference exceeds a preset threshold, a second face feature is added to the first face feature set.

In an embodiment of the present application, the detection module 410 is further configured to identify a face region in a current video frame based on a normalized feature difference of a picture element and a human body half-length recognition algorithm to obtain a current target to be tracked corresponding to the current video frame. .

In an embodiment of the present application, as shown in FIG. 14, the detection module 410 may include a first recommendation unit 414 for identifying a face region based on the normalized feature differences of the primitives, and obtaining the first region in the current video frame. Recommended area.

The second recommendation unit 415 obtains, according to the optical flow analysis algorithm, a second recommendation region corresponding to the first target to be tracked in the current video frame.

The second tracking target determination unit 416 is configured to obtain a current target to be tracked according to the first recommendation area and the second recommendation area.

In an embodiment of the present application, the second tracking target determination unit 416 is further configured to perform motion prediction according to the inter-frame correlation to obtain an expected motion range, and filter the first recommended region and the second recommended region to obtain the current target to be tracked according to the expected motion range. .

In an embodiment of the present application, the network structure of the deep neural network is an 11-layer network layer, including a stacked convolutional neural network and a fully connected layer. The stacked convolutional neural network routes multiple volumes. It consists of multi-layer and maxpool layer. The specific network structure is: conv 3-64 * 2 + LRN + max pool

conv 3-128 + max pool

conv 3-256 * 2 + max pool

conv 3-512 * 2 + max pool

conv 3-512 * 2 + max pool

FC 2048

FC 1024, where conv 3 represents a convolution layer with a radius of 3, LRN represents the LRN layer, maxpool represents the maximum pooling layer, and FC represents a fully connected layer.

In an embodiment of the present application, the facial feature extraction module 420 is further configured to perform facial feature extraction on the first target to be tracked through a deep neural network to obtain a first feature vector, and to perform a deep neural network on the current target to be tracked. Face feature extraction is performed to obtain a second feature vector.

The tracking module 430 is further configured to calculate the Euclidean distance between the first feature vector and the second feature vector. If the Euclidean distance is less than a preset threshold, it is determined that the features of the first target to be tracked and the current target to be tracked are successfully matched. .

FIG. 15 is another schematic structural diagram of a video target tracking device according to an embodiment of the present application. As shown in FIG. 15, the video target tracking device includes a processor 510, a memory 520 connected to the processor 510, and an interface 530 for sending and receiving data. The memory 520 stores a machine-readable instruction module that can be executed by the processor 510. The machine-readable instruction module includes a detection module 521 for acquiring a video stream, and detecting the video stream based on a face. The algorithm recognizes the face area and obtains the first target to be tracked corresponding to the first video frame.

A face feature extraction module 522 is configured to obtain the first face feature by extracting the face feature based on a deep neural network from the first target to be tracked, and store the first face feature into the first A feature library corresponding to the target to be tracked.

The detection module 521 is further configured to identify a face area according to a face detection algorithm in a current video frame, and obtain a current target to be tracked corresponding to the current video frame.

The facial feature extraction module 522 is further configured to obtain a second facial feature by using a deep neural network-based facial feature extraction for the current target to be tracked.

The tracking module 523 is configured to perform feature matching between the current target to be tracked and the first target to be tracked according to the second face feature and the feature library, so as to track the first target to be tracked from the first video frame.

A learning module 524 is configured to update the feature database according to the extracted updated face features during the tracking process.

In an embodiment of the present application, as shown in FIG. 16, the machine-readable instruction module may further include: a feature identity processing module 525, configured to recognize by a face recognition algorithm according to a face state of a target to be tracked. The corresponding face identity information is obtained, and the target feature corresponding to the face identity information is obtained according to the image feature extraction algorithm, and an association relationship is established between the target feature and the face identity information.

In the embodiment of the present application, the specific functions and implementation methods of the detection module 521, face feature extraction module 522, tracking module 523, learning module 524, and feature identity processing module 525 can refer to the aforementioned module 410. Relevant descriptions up to 450 are not repeated here.

A person of ordinary skill in the art can understand that all or part of the processes in the method of the above embodiments can be implemented by using a computer program to instruct related hardware. The program can be stored in a non-volatile computer-readable storage medium. In the embodiment of the present application, the program may be stored in a storage medium of a computer system and executed by at least one processor in the computer system to implement the processes including the embodiments of the methods described above. The storage medium may be a magnetic disk, an optical disk, a read-only memory (Read-Only Memory, ROM), or a random access memory (Random Access Memory, RAM).

Through the description of the above embodiments, those skilled in the art can clearly understand that the embodiments of the present application can be implemented by means of software plus necessary universal hardware platform, that is, the relevant hardware is instructed by machine-readable instructions. Implementation can of course also be done in hardware, but in many cases the former is a better implementation. Based on such an understanding, the technical solutions of the embodiments of the present application that are essentially or contribute to the existing technology can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions to make A terminal device (for example, a mobile phone, a personal computer, a server, or a network device) executes the methods described in the embodiments of the present application.

The technical features of the embodiments described above can be arbitrarily combined. In order to make the description concise, all possible combinations of the technical features in the above embodiments have not been described. However, as long as there is no contradiction in the combination of these technical features, It should be considered as the scope described in this specification.

Although the present application has been disclosed as above with preferred embodiments, it is not intended to limit the present application. Those with ordinary knowledge in the technical field to which this application belongs may make various changes and modifications without departing from the spirit and scope of this application. Retouching, so the protection scope of this application shall be determined by the scope of the appended patent application.

Claims (22)

    A video target tracking method applied to a terminal or a server. The method includes: acquiring a video stream, identifying a face area according to a face detection algorithm, and obtaining a first target to be tracked corresponding to a first video frame; A target to be tracked obtains a first face feature through facial feature extraction based on a deep neural network, and the first face feature is stored in a feature library corresponding to the first target to be tracked; The face detection algorithm recognizes a face region, obtains a current target to be tracked corresponding to the current video frame, and obtains a second face feature for the current target to be tracked based on facial feature extraction based on a deep neural network. Two face features and the feature library perform feature matching between the current target to be tracked and the first target to be tracked to track the first target to be tracked from the first video frame, and are extracted according to the tracking process. The updated face features update the feature library.         According to the method described in item 1 of the patent application scope, the method further comprises: identifying the corresponding face identity information through a face recognition algorithm according to the face state of the target to be tracked, and extracting the algorithm to obtain Describe the target feature corresponding to the face identity information, and establish an association relationship between the target feature and the face identity information; and identify the face area according to the face detection algorithm in the current video frame to obtain the current target corresponding to the current video frame. The step of tracking the target includes: judging whether a face area is recognized according to a face detection algorithm in the current video frame, and if no face area is identified, obtaining a current image feature corresponding to the current video frame according to an image feature extraction algorithm ; Comparing the current image features with the target features to obtain matching target face identity information based on the association relationship; and obtaining the current target to be tracked corresponding to the current video frame based on the target face identity information.         The method according to item 1 of the scope of patent application, wherein the first face feature is extracted from the deep neural network-based face feature to the first target to be tracked, and the first face feature is obtained. The step of storing the feature database corresponding to the first target to be tracked includes: obtaining first face identity information corresponding to the first target to be tracked; establishing a first face feature set corresponding to the first face identity information, Adding the first facial feature to the first facial feature set and storing the first facial feature set to the feature database; and updating the updated facial feature based on the extracted updated facial feature during the tracking process The steps of the feature database include: obtaining current face identity information corresponding to the current target to be tracked; obtaining a first face feature set corresponding to the current face identity information from the feature database; and calculating the first face feature The amount of difference between the first face feature in the set and the second face feature. If the difference exceeds a preset threshold, the second face feature is added to the first face feature set.         According to the method described in item 1 of the scope of patent application, the step of identifying a face region according to a face detection algorithm in a current video frame to obtain a current target to be tracked corresponding to the current video frame includes: based on a normalized primitive The difference feature and the human body half body recognition algorithm identify the face area in the current video frame, and obtain the current target to be tracked corresponding to the current video frame.         According to the method described in item 1 of the scope of patent application, the step of identifying a face region according to a face detection algorithm in a current video frame to obtain a current target to be tracked corresponding to the current video frame includes: based on a normalized primitive Differential features identify the face area, and obtain a first recommended area in the current video frame; calculate a second recommended area corresponding to the first target to be tracked in the current video frame according to the optical flow analysis algorithm; and according to the first recommended area And the second recommendation area to obtain the current target to be tracked.         According to the method described in claim 5, the step of obtaining the current target to be tracked according to the first recommendation area and the second recommendation area includes: performing motion prediction based on inter-frame correlation to obtain an expected motion. Range, and filtering the first recommendation region and the second recommendation region according to the expected motion range to obtain the current target to be tracked.     According to the method described in any of claims 1 to 6, the network structure of the deep neural network is an 11-layer network layer, including a stacked convolutional neural network and a fully connected layer. The stacked convolutional neural network consists of multiple convolutional layers and maxpool layers. The specific network structure is: conv 3-64 * 2 + LRN + max pool conv 3-128 + max pool conv 3-256 * 2 + max pool conv 3-512 * 2 + max pool conv 3-512 * 2 + max pool FC 2048 FC 1024, where conv 3 represents a convolution layer with a radius of 3, LRN represents the LRN layer, maxpool represents the maxpool layer, and FC represents a fully connected layer .     According to the method described in any one of claims 1 to 6, the first target feature is obtained by extracting a facial feature based on a deep neural network on the first target to be tracked, and the first The step of storing facial features into a feature library corresponding to the first target to be tracked includes: performing facial feature extraction on the first target to be tracked to obtain a first feature vector through a deep neural network; The target to be tracked obtains a second face feature by extracting a facial feature based on a deep neural network, and performs feature matching between the current target to be tracked and the first target to be tracked according to the second face feature and the feature library. The step of tracking the first target to be tracked from the first video frame includes: performing facial feature extraction on the current target to be tracked through a deep neural network to obtain a second feature vector; and calculating the first feature vector. The Euclidean distance between the feature vector and the second feature vector. If the Euclidean distance is less than a preset threshold, it is determined that the features of the first target to be tracked and the current target to be tracked are successfully matched.         A video target tracking device includes a processor and a memory connected to the processor. The memory stores a machine-readable instruction module executable by the processor; the machine may The read instruction module includes: a detection module for obtaining a video stream, identifying a face area according to a face detection algorithm, and obtaining a first target to be tracked corresponding to a first video frame; a face feature extraction module for detecting The first target to be tracked obtains a first face feature by extracting a facial feature based on a deep neural network, and stores the first face feature in a feature library corresponding to the first target to be tracked; The detection module is further configured to identify a face area according to a face detection algorithm in the current video frame to obtain a current target to be tracked corresponding to the current video frame; the face feature extraction module is further configured to perform a target detection on the current target A second facial feature is obtained through facial feature extraction based on a deep neural network; a tracking module is configured to compare the current target to be tracked with the first facial feature according to the second facial feature and the feature library. Feature matching is performed on a target to be tracked to track the first target to be tracked from the first video frame; a learning module is used to update the feature database according to the extracted updated face features during the tracking process.         According to the device described in claim 9 of the patent application scope, the device further includes: a feature identity processing module, which is used to obtain corresponding face identity information through a face recognition algorithm according to the face state of the target to be tracked. An image feature extraction algorithm obtains a target feature corresponding to the face identity information, and establishes an association relationship between the target feature and the face identity information; the detection module includes an image feature extraction unit for determining whether Whether the current video frame recognizes a facial area based on the face detection algorithm. If no facial area is recognized, the algorithm extracts the current image features corresponding to the current video frame according to the image feature extraction algorithm; the identity matching unit is used to The association relationship compares the current image feature with the target feature to obtain matching target face identity information; a first tracking target determination unit is configured to obtain a corresponding video frame corresponding to the current video frame according to the target face identity information; The target currently being tracked.         According to the device described in item 9 of the scope of patent application, the face feature extraction module is further configured to obtain first face identity information corresponding to a first target to be tracked, and establish a first face identity information corresponding to the first face identity information. A face feature set, adding the first face feature to the first face feature set and storing the first face feature set to the feature database; the learning module is further configured to obtain the current The current face identity information corresponding to the target to be tracked, obtain a first face feature set corresponding to the current face identity information from the feature database, and calculate a first face feature and a first face feature set in the first face feature set. A difference amount of the second face feature, and if the difference amount exceeds a preset threshold, adding the second face feature to the first face feature set.         According to the device described in item 9 of the scope of patent application, the detection module is further configured to identify a face region in a current video frame based on a normalized feature difference of a picture element and a human half-length recognition algorithm to obtain a current video frame correspondence. The current target to track.         According to the device described in claim 9 of the patent application scope, the detection module includes: a first recommendation unit for identifying a face region based on the normalized feature differences of the picture elements, and obtaining the first recommendation region in the current video frame; A second recommendation unit calculates and obtains, according to an optical flow analysis algorithm, a second recommendation area corresponding to the first target to be tracked in the current video frame; a second tracking target determination unit is configured to, according to the first recommendation area and the The second recommendation area obtains the current target to be tracked.         According to the device described in claim 13 of the patent application scope, the second tracking target determination unit is further configured to perform motion prediction based on inter-frame correlation to obtain an expected motion range, and filter the first recommended region and The second recommendation region obtains the current target to be tracked.         According to the device described in any of claims 9 to 14, the facial feature extraction module is further configured to perform facial feature extraction on the first target to be tracked through a deep neural network to obtain a first feature vector. , Performing facial feature extraction on the current target to be tracked through a deep neural network to obtain a second feature vector; the tracking module is further configured to calculate an Euclidean distance between the first feature vector and the second feature vector, if If the Euclidean distance is less than a preset threshold, it is determined that the features of the first target to be tracked and the characteristics of the current target to be tracked are successfully matched.         A non-transitory computer-readable storage medium storing machine-readable instructions stored in the storage medium. The machine-readable instructions can be executed by a processor to complete the following operations: acquiring a video stream and identifying the video stream based on a face detection algorithm. A face region to obtain a first target to be tracked corresponding to a first video frame; to obtain a first face feature from a deep neural network-based face feature extraction for the first target to be tracked, and Face features are stored in a feature library corresponding to the first target to be tracked; the face region is identified according to a face detection algorithm in the current video frame to obtain the current target to be tracked corresponding to the current video frame, and the current target to be tracked is passed A second facial feature is obtained by extracting a facial feature based on a deep neural network, and the current target to be tracked and the first target to be tracked are subjected to feature matching according to the second facial feature and the feature library to obtain The first video frame starts tracking the first target to be tracked, and the feature database is updated according to the extracted updated face features during the tracking process.         As the non-volatile computer-readable storage medium described in item 16 of the scope of patent application, the machine-readable instructions may be executed by the processor to complete the following operations: face recognition through face recognition according to the face state of the target to be tracked The corresponding facial identity information is obtained by algorithm recognition, and the target feature corresponding to the facial identity information is obtained by extracting the algorithm according to image features, and an association relationship is established between the target feature and the facial identity information; The frame recognizes the face area according to the face detection algorithm and obtains the current target to be tracked corresponding to the current video frame. The steps include: judging whether the face area is recognized according to the face detection algorithm in the current video frame, and if no face is recognized Region, the current image feature corresponding to the current video frame is obtained according to the image feature extraction algorithm; based on the association relationship, the current image feature is compared with the target feature to obtain matching target face identity information; according to The target face identity information obtains the current target to be tracked corresponding to the current video frame.         According to the non-volatile computer-readable storage medium described in item 16 of the scope of the patent application, the first face feature is extracted from the face feature based on a deep neural network for the first target to be tracked, and The step of storing the first face feature into a feature library corresponding to the first target to be tracked includes: obtaining first face identity information corresponding to the first target to be tracked; and establishing a first face identity information corresponding to the first face to be tracked. A first face feature set, adding the first face feature to the first face feature set, and storing the first face feature set to the feature database; the tracking process is based on the extracted Updated face features The steps of updating the feature database include: obtaining current face identity information corresponding to the current target to be tracked; obtaining the first face feature set corresponding to the current face identity information from the feature database; calculating The amount of difference between the first face feature and the second face feature in the first face feature set, and if the difference exceeds a preset threshold, adding the Describe The two facial features.         According to the non-volatile computer-readable storage medium described in item 16 of the scope of the patent application, the step of identifying a face area based on a face detection algorithm in a current video frame and obtaining a current target to be tracked corresponding to the current video frame includes : Recognize the face area in the current video frame based on the normalized feature differences of the primitives and the human half-length recognition algorithm to obtain the current target to be tracked corresponding to the current video frame.         According to the non-volatile computer-readable storage medium described in item 16 of the scope of the patent application, the step of identifying a face area based on a face detection algorithm in a current video frame and obtaining a current target to be tracked corresponding to the current video frame includes : Recognizing a face region based on normalized feature differences of pixels and obtaining a first recommended region in the current video frame; calculating a second recommended region corresponding to the first target to be tracked in the current video frame according to an optical flow analysis algorithm Obtaining the current target to be tracked according to the first recommendation area and the second recommendation area.         According to the non-transitory computer-readable storage medium described in item 20 of the scope of patent application, the step of obtaining the current target to be tracked according to the first recommended area and the second recommended area includes: Correlation is used to perform motion prediction to obtain an expected motion range, and the first recommendation region and the second recommendation region are filtered according to the expected motion range to obtain the current target to be tracked.         According to the non-transitory computer-readable storage medium according to any one of claims 16 to 21, the first human face is obtained by extracting facial features based on a deep neural network on the first target to be tracked. And storing the first face feature into a feature library corresponding to the first target to be tracked includes: performing facial feature extraction on the first target to be tracked to obtain the first feature through a deep neural network A vector; the second facial feature is obtained by extracting facial features based on a deep neural network for the current target to be tracked, and comparing the current target to be tracked with the second facial feature and the feature library The first target to be tracked is subjected to feature matching to track the first target to be tracked from the first video frame. The step includes: performing facial feature extraction on the current target to be tracked through a deep neural network to obtain a second Feature vector; calculating the Euclidean distance between the first feature vector and the second feature vector, and if the Euclidean distance is less than a preset threshold, determining the first target to be tracked and the current target to be tracked The target feature matches successfully.