CN109948560B - Mobile robot target tracking system fusing bone recognition and IFace-TLD - Google Patents

Abstract

A mobile robot target tracking system integrating skeleton recognition and IFace-TLD comprises an original color picture of a human body and a skeleton picture of an upper limb which are obtained through a Kinect sensor, an IFace-TLD unit used for tracking and positioning a target on the color picture and a skeleton recognition unit used for tracking and positioning the target on the skeleton picture, wherein a region frame where the target is located is obtained and sent to an image target positioning unit, the image target positioning unit marks a target region on the original color picture according to the obtained region frame where the target is located, and the target region is fed back to the IFace-TLD unit. The invention effectively solves the problem of short sequence tracking, and can realize better tracking on the target face no matter the length of the tracking sequence. The invention can realize stable recognition effect even under the condition that the human face faces back to the camera. The online processing based on the bone recognition is realized, and the tracking accuracy and robustness are improved.

Description

Mobile robot target tracking system integrating skeleton recognition and IFace-TLD

Technical Field

The present invention relates to a mobile robot target tracking system, and more particularly to a mobile robot target tracking system integrating skeleton recognition and IFace-TLD.

Background Art

Object tracking has a wide range of applications in security, robotics, human-computer interaction, etc. In the actual tracking process, due to factors such as the rapid movement of the target, changes in lighting, and occlusion, achieving robust and efficient tracking is a very challenging task.

Faces are highly distinguishable. In order to achieve better tracking results, we choose to track the target face. The face-based tracking-detection-learning algorithm (Face-TLD) can track faces for a long time. However, because this is a long tracking algorithm, when the tracking sequence is short, the learning part of Face-TLD is not fully trained, the tracking effect is poor, and even large drift may occur. In addition, in actual application scenarios, the rotation of the face is highly random, and it cannot be guaranteed that the face is facing the camera at all times. In some cases, the target face may be completely facing away from the camera. At this time, the tracking algorithm based on image appearance will fail.

Human face has unique biological characteristics and has been used in many occasions. However, most high-precision face recognition algorithms are time-consuming, and these time-consuming algorithms cannot be applied to mobile robot target tracking, which has high real-time requirements.

Based on the TLD algorithm, Face-TLD can track faces for a long time and robustly. The original TLD is a single target tracking algorithm that can track an unknown object for a long time in a video stream. The original TLD can be divided into three parts, namely, the tracking part, the learning part, and the detection part. Because of its good tracking effect, there are many improvements based on this algorithm. Face-TLD is one of them. Face-TLD combines face detection and TLD to achieve long-term tracking of faces. In the original Face-TLD, the detector can be divided into two parts, namely, the face detection part and the verifier part. The face detection part processes all image blocks, and the verifier part outputs the confidence coefficient of the image block containing a specific face. However, when the tracking sequence is short, Face-TLD cannot achieve a satisfactory tracking effect due to insufficient training of the learning part. Specifically, the learning part is introduced to deal with various uncertainties. However, in order to ensure accuracy, the learning part requires sufficient training data, and this is a time-consuming process. Short sequences cannot provide enough pictures to train the learning part. What’s more serious is that in the early stage of tracking, if the targets have similar appearances, the original Face-TLD is likely to lose the target.

Microsoft Kinect sensor can directly collect human skeleton information and is relatively robust to human movement. Even if the human body is completely facing away from Kinect, relatively stable skeletons can be collected. Due to the advantages of the sensor, skeleton-based human recognition has good robustness to changes in lighting, movement, and target appearance. However, existing skeleton-based human recognition algorithms all collect a certain amount of data and then process it offline. They need to manually set training labels, making it difficult to achieve online recognition. This obviously cannot meet the requirements of online tracking of mobile robots.

Summary of the invention

The technical problem to be solved by the present invention is to provide a mobile robot target tracking system which integrates skeleton recognition and IFace-TLD and can achieve better tracking of a target face and improve tracking accuracy and robustness.

The technical solution adopted by the present invention is: a mobile robot target tracking system integrating skeleton recognition and IFace-TLD, comprising obtaining an original color picture of a human body and a skeleton picture of an upper limb through a Kinect sensor, obtaining a target area frame through an IFace-TLD unit for tracking and locating the target on the color picture, and a skeleton recognition unit for tracking and locating the target on the skeleton picture, and sending the target area frame to an image target positioning unit, the image target positioning unit marks a target area in the original color picture according to the obtained target area frame, and feeds the target area back to the IFace-TLD unit.

The IFace-TLD unit includes a tracking part, a learning part, a detection part, and an integrator for respectively acquiring the original color picture, wherein the tracking part uses an optical flow tracker to estimate the motion trajectory of the target in the acquired original color picture between two adjacent frames, and sends the trajectory to the learning part and the integrator respectively; the detection part independently scans and processes all image blocks in the first frame of the acquired original color picture, separates the target face and the background, and sends the target face to the learning part and the integrator respectively, and scans and processes only the target area and the surrounding area fed back by the image target positioning unit for the original color picture after the first frame, separates the target face and the background, and sends the target face to the learning part and the integrator respectively; the integrator calculates the confidence coefficient of the most likely target position in the original color picture according to the obtained motion trajectory of the target and the target face between two adjacent frames, and sends the calculation result to the learning part, and the skeleton recognition unit or the image target positioning unit; the learning part is trained according to the original color picture and the results obtained from the tracking part, the detection part and the integrator, and updates and corrects the errors in the tracking part and the detection part according to the training results.

The detection part includes a face detection part for detecting the area of the face in the original color picture according to the acquired original color picture, the target area of the learning part and the target area information fed back by the image target positioning unit, a face recognition part for identifying the target face area according to the area of the face in the original color picture obtained from the face detection part, and a verifier part for judging whether the target face area recognized by the face recognition part is correct. The verification results of the verifier part are respectively sent to the learning part and the integrator.

The skeleton recognition unit includes a motion cycle extraction part, a skeleton feature extraction part and a support vector data description part, wherein the motion cycle extraction part calculates the motion cycle of the human body according to the acquired skeleton image, and the skeleton feature extraction part calculates the skeleton features within the obtained motion cycle of the human body. When the result output by the integrator in the IFace-TLD unit is the target area frame, the skeleton features obtained by the skeleton feature extraction part are sent to the training part in the support vector data description part for training. When the result output by the integrator in the IFace-TLD unit is empty, the skeleton features obtained by the skeleton feature extraction part are sent to the prediction part in the support vector data description part. The prediction part predicts the target area frame according to the training result of the training part, and sends the predicted target area frame to the image target positioning unit.

The motion cycle extraction part calculates the motion cycle of the human body according to the acquired skeleton image using the following formula:

和

表示第k帧图像左手腕和肩膀中心的三维点坐标；N表示序列中总的图像总帧数。Where dist _k is the distance between the left wrist and shoulder center of the kth frame image in the Kinect coordinate system;

and

Represents the 3D point coordinates of the center of the left wrist and shoulder of the kth frame image; N represents the total number of image frames in the sequence.

The bone feature extraction part calculates the bone features within the obtained motion cycle of the human body, including:

First, the gait half-cycle is defined as T _w , then the skeletal features based on the upper limbs of the human body are expressed as:

Trajectory feature: The center point of the shoulder is selected as the fixed point, and the relative positions of other upper limb bone points relative to the fixed point are calculated by the following formula to obtain a 9-dimensional feature P:

表示第j个人体骨骼在步态半周期T_w中的第t帧图像的位置，

被表示为：in,

represents the position of the jth human skeleton in the tth frame image in the gait half cycle _Tw ,

is represented as:

表示相机坐标系下肩膀中心点的位置，上肢骨骼中其余坐标点的位置表示为

用P的协方差矩阵来表示包含人体行走习惯的轨迹特征矩阵F_T，定义in,

represents the position of the center point of the shoulder in the camera coordinate system, and the positions of the remaining coordinate points in the upper limb bones are expressed as

The covariance matrix of P is used to represent the trajectory feature matrix F _T containing the walking habits of human beings, and the definition is

和

分别表示测试数据和训练数据的轨迹特征矩阵；

是

和

之间的广义特征值，满足

x是相应的广义右特征向量；in,

and

Represent the trajectory feature matrices of test data and training data respectively;

yes

and

The generalized eigenvalues between

x is the corresponding generalized right eigenvector;

Area and distance features: The area feature _FA represents the area of the closed area enclosed by the upper limbs of the human body. The distance feature _FD is represented by the distance between different human body centers. _FA is expressed as:

和

分别表示肩膀中心点、头部、左肩和右肩在步态半周期T_w中的第t帧的位置；in,

and

denote the positions of the shoulder center, head, left shoulder, and right shoulder in the tth frame in the gait half cycle Tw _, respectively;

右手中心

和左手中心

通过以下公式计算得到：To calculate the distance feature _FD , we first calculate the centers of the three closed polygons in the upper limb area. The three center points are the center of the head.

Right hand center

and left hand center

Calculated by the following formula:

是肩膀中心点和头部点围成多边形的中心；右手中心

是右肩点，右肘点和右手腕点围成多边形的中心；左手中心

是左肩点，左肘点和左手腕点围成多边形的中心；右手中心

分别与头部中心

和左手中心

之间的欧几里得距离f_t ^d1和f_t ^d2写为：Head Center

It is the center of the polygon formed by the center point of the shoulder and the head; the center of the right hand

It is the center of the polygon formed by the right shoulder point, right elbow point and right wrist point; the left hand center

It is the center of the polygon formed by the left shoulder point, left elbow point and left wrist point; the center of the right hand

Head center

and left hand center

The Euclidean distance between f _t ^d1 and f _t ^d2 is written as:

这样，整个距离特征表示为make

In this way, the entire distance feature is expressed as

和

分别表示f^di的均值、方差和最大值，其中,i＝1,2；in,

and

Respectively represent the mean, variance and maximum value of f ^di , where i = 1, 2;

Static features: Static features are represented by a 5-dimensional vector F _S = [f _h ,f _lua ,f _rua ,f _lf ,f _rf ] ^T , where f _h represents the height of the target, and f _lua ,f _rua ,f _lf and f _rf represent the length of the left upper arm, the length of the right upper arm, the length of the left forearm and the length of the right forearm, respectively. They are obtained by the following formulas:

分别表示在相机坐标系下，肩部中心点、左肩点、左肘点、左腕点、左手点、右肩点、右肘点、右腕点和右手点的-位置；in,

Respectively represent the positions of the shoulder center point, left shoulder point, left elbow point, left wrist point, left hand point, right shoulder point, right elbow point, right wrist point and right hand point in the camera coordinate system;

Frequency feature and amplitude feature: The frequency feature F _Fre is the number of skeleton image frames in a gait half cycle, and the difference between adjacent local maximum and local minimum is the amplitude feature F _Amp ;

构成基于人体上肢的骨骼特征。Finally, we get a 23-dimensional mixed feature

The composition is based on the skeletal features of the human upper limbs.

The mobile robot target tracking system of the present invention, which integrates skeleton recognition and IFace-TLD, adds face recognition based on principal component analysis (PCA) to the original Face-TLD algorithm, effectively solves the short sequence tracking problem, and the present invention is referred to as IFace-TLD. In the present invention, regardless of the length of the tracking sequence, IFace-TLD can achieve better tracking of the target face. At the same time, SIFace-TLD seamlessly integrates IFace-TLD with skeleton-based human body recognition. When IFace-TLD tracks successfully, the extracted skeleton features are used to train support vector data description (SVDD). When its tracking fails, the newly extracted skeleton features can be sent to the trained SVDD for identification. In this way, even when the face is facing away from the camera, a stable recognition effect can be achieved. Not only online processing based on skeleton recognition is achieved, but also the accuracy and robustness of tracking are improved.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG1 is a block diagram of a mobile robot target tracking system integrating skeleton recognition and IFace-TLD according to the present invention;

Fig. 2 is a schematic diagram of a human skeleton containing 20 joints;

FIG3 is a graph of dist _k .

In the figure

101: IFace-TLD Unit 101.1: Tracking Section

101.2: Learning part 101.3: Detection part

101.31 Face detection part 101.32: Face recognition part

101.33: Validator Section 101.4: Integrator

102: Skeleton Recognition Unit 102.1: Motion Cycle Extraction Part

102.2: Skeleton feature extraction part 102.3: Support vector data description part

102.31: Training part 102.32: Prediction part

103: Image target positioning unit

DETAILED DESCRIPTION

The mobile robot target tracking system integrating skeleton recognition and IFace-TLD of the present invention is described in detail below in conjunction with the embodiments and drawings.

As shown in FIG1 , the mobile robot target tracking system integrating skeleton recognition and IFace-TLD of the present invention comprises obtaining an original color image A of a human body and a skeleton image B of an upper limb through a Kinect sensor, obtaining a target region frame through an IFace-TLD unit 101 for tracking and locating the target on the color image A, and a skeleton recognition unit 102 for tracking and locating the target on the skeleton image B, and sending the frame to an image target positioning unit 103. The image target positioning unit 103 marks a target region in the original color image A according to the obtained target region frame, and feeds the target region back to the IFace-TLD unit 101.

The IFace-TLD unit 101 includes a tracking part 101.1, a learning part 101.2, a detection part 101.3, and an integrator 101.4 for respectively acquiring the original color image A, wherein the tracking part 101.1 uses an optical flow tracker to estimate the motion trajectory of the target in the original color image A between two adjacent frames, and sends the estimates to the learning part 101.2 and the integrator 101.4 respectively; the detection part 101.3 independently scans and processes all image blocks in the first frame of the original color image A acquired, separates the target face and the background, and sends the target face to the learning part 101.2 and the integrator 101.4 respectively, and only scans and processes the original color image A after the first frame acquired. 03, scan and process the target area and the surrounding area fed back by the tracking part 101.1, separate the target face from the background, and send the target face to the learning part 101.2 and the integrator 101.4 respectively; the integrator 101.4 calculates the confidence coefficient of the most likely target position in the original color picture A according to the obtained motion trajectory of the target between two adjacent frames and the target face, and sends the calculation result to the learning part 101.2, and the skeleton recognition unit 102 or the image target positioning unit 103; the learning part 101.2 is trained according to the original color picture A and the results obtained from the tracking part 101.1, the detection part 101.3 and the integrator 101.4, and updates and corrects the errors in the tracking part 101.1 and the detection part 101.3 according to the training results.

The detection part 101.3 includes a face detection part 101.31 for detecting the area of the face in the original color picture A based on the acquired original color picture A, the target area of the learning part 101.2 and the target area information fed back by the image target positioning unit 103, a face recognition part 101.32 for identifying the target face area in the original color picture A based on the face obtained from the face detection part 101.31, and a verifier part 101.33 for judging whether the target face area recognized by the face recognition part 101.32 is correct. The verification results of the verifier part 101.33 are respectively sent to the learning part 101.2 and the integrator 101.4.

The IFace-TLD of the present invention combines Face-TLD and face recognition based on principal component analysis. The Face-TLD was proposed by Zdenek Kalal, Krystian Mikolajczyk and Jiri Matas in an article entitled "Face-TLD: Tracking-Learning-Detection Applied to Faces" in 2010. Compared with the original Face-TLD, IFace-TLD has an additional face detection part, so that the tracking performance is enhanced. In the IFace-TLD unit 101, face recognition based on principal component analysis is added after face detection, so as to distinguish similar image blocks. In order to reduce the number of image blocks and improve the processing speed, the present invention only considers the target area of feedback, and the size of the target area of feedback is twice the size of the bounding box containing the target face obtained before. Face detection is used to detect all image blocks containing faces, and then the face recognition part filters out faces that are not the target. Finally, all the remaining image blocks are sent to the verifier to further determine whether they contain the target face.

The skeleton recognition unit 102 includes a motion cycle extraction part 102.1, a skeleton feature extraction part 102.2 and a support vector data description part 102.3, wherein the motion cycle extraction part 102.1 calculates the motion cycle of the human body according to the acquired skeleton image B, and the skeleton feature extraction part 102.2 calculates the skeleton features within the obtained motion cycle of the human body. When the result output by the integrator 101.4 in the IFace-TLD unit 101 is the target area frame, the skeleton feature extraction part 102.2 obtains The skeletal features are sent to the training part 102.31 in the support vector data description part 102.3 for training. When the result output by the integrator 101.4 in the IFace-TLD unit 101 is empty, the skeletal features obtained by the skeletal feature extraction part 102.2 are sent to the prediction part 102.32 in the support vector data description part 102.3. The prediction part 102.32 predicts the target area box according to the training result of the training part 102.31, and sends the predicted target area box to the image target positioning unit 103.

The human skeleton with 20 joints is shown in Figure 2, and the numbers in the figure are shown in Table 1:

Table 1

1 Hip center 11 Right wrist point 2 Ridge Point 12 Right hand point 3 Shoulder center point 13 Left hip joint 4 Head Point 14 Left knee point 5 Left shoulder point 15 Left ankle point 6 Left elbow point 16 Left foot point 7 Left wrist point 17 Right hip joint 8 Left hand point 18 Right knee point 9 Right shoulder point 19 Right ankle point 10 Right elbow point 20 Right foot point

The present invention realizes online human body recognition using ten skeleton points of the upper limbs, which are connected by black solid lines. The remaining ten skeleton points are connected by black dotted lines. The gait cycle is obtained by calculating the distance between the left wrist and the center of the shoulder in the Kinect coordinate system.

The motion cycle extraction part 102.1 calculates the motion cycle of the human body according to the acquired skeleton image B using the following formula:

和

表示第k帧图像左腕点和肩部中心点的三维点坐标；N表示序列中总的图像总帧数，dist_k曲线如图3所示。点线表示最原始的距离曲线。为了降低噪声的干扰，将原始数据进行均值滤波。滤波后的曲线由实线表示。本发明定义步态全周期是相邻局部最大(或最小)的帧数间隔。相邻局部最大和局部最小之间的帧数数目定义为步态半周期。因为骨骼特征是在步态周期内提取的，为了得到更多的骨骼特征，本发明在步态半周期内进行提取。Wherein, dist _k is the distance between the left wrist point and the shoulder center point of the kth frame image in the Kinect coordinate system;

and

Represents the three-dimensional point coordinates of the left wrist point and the shoulder center point of the k-th frame image; N represents the total number of image frames in the sequence, and the dist _k curve is shown in Figure 3. The dotted line represents the most original distance curve. In order to reduce the interference of noise, the original data is mean filtered. The filtered curve is represented by a solid line. The present invention defines the full gait cycle as the frame interval between adjacent local maximums (or minimums). The number of frames between adjacent local maximums and local minimums is defined as the gait half cycle. Because the skeletal features are extracted within the gait cycle, in order to obtain more skeletal features, the present invention extracts them within the gait half cycle.

The bone feature extraction part 102.2 calculates the bone features within the obtained human body motion cycle, including:

First, the gait half-cycle is defined as T _w , then the skeletal features based on the upper limbs of the human body are expressed as:

Trajectory feature: The center point of the shoulder is selected as the fixed point, and the relative positions of other upper limb bone points relative to the fixed point are calculated by the following formula to obtain a 9-dimensional feature P:

表示第j个人体骨骼在步态半周期T_w中的第t帧图像的位置，

被表示为：in,

represents the position of the jth human skeleton in the tth frame image in the gait half cycle _Tw ,

is represented as:

表示相机坐标系下肩部中心点的位置，上肢骨骼中其余坐标点的位置表示为

用P的协方差矩阵来表示包含人体行走习惯的轨迹特征矩阵F_T，定义in,

represents the position of the center point of the shoulder in the camera coordinate system, and the positions of the remaining coordinate points in the upper limb bones are expressed as

The covariance matrix of P is used to represent the trajectory feature matrix F _T containing the walking habits of human beings, and the definition is

和

分别表示测试数据和训练数据的轨迹特征矩阵；

是

和

之间的广义特征值，满足

x是相应的广义右特征向量；in,

and

Represent the trajectory feature matrices of test data and training data respectively;

yes

and

The generalized eigenvalues between

x is the corresponding generalized right eigenvector;

Area and distance features: The area feature _FA represents the area of the closed area enclosed by the upper limbs of the human body. The distance feature _FD is represented by the distance between different human body centers. _FA is expressed as:

和

分别表示肩部中心点、头部点、左肩点和右肩点在步态半周期T_w中的第t帧的位置；in,

and

denote the positions of the shoulder center point, head point, left shoulder point, and right shoulder point in the tth frame in the gait half cycle Tw _, respectively;

右手中心

和左手中心

通过以下公式计算得到：To calculate the distance feature _FD , we first calculate the centers of the three closed polygons in the upper limb area. The three center points are the center of the head.

Right hand center

and left hand center

Calculated by the following formula:

是肩部中心点和头部点围成多边形的中心；右手中心

是右肩点，右肘点和右腕点围成多边形的中心；左手中心

是左肩点，左肘点和左腕点围成多边形的中心；右手中心

分别与头部中心

和左手中心

之间的欧几里得距离f_t ^d1和f_t ^d2写为：Head Center

It is the center of the polygon formed by the center point of the shoulder and the head; the center of the right hand

It is the center of the polygon formed by the right shoulder point, right elbow point and right wrist point; the left hand center

It is the center of the polygon formed by the left shoulder point, left elbow point and left wrist point; the center of the right hand

Head center

and left hand center

The Euclidean distance between f _t ^d1 and f _t ^d2 is written as:

这样，整个距离特征表示为make

In this way, the entire distance feature is expressed as

和

分别表示f^di的均值、方差和最大值，其中,i＝1,2；in,

and

Respectively represent the mean, variance and maximum value of f ^di , where i = 1, 2;

Static features: Static features are represented by a 5-dimensional vector F _S = [f _h ,f _lua ,f _rua ,f _lf ,f _rf ] ^T , where f _h represents the height of the target, and f _lua ,f _rua ,f _lf and f _rf represent the length of the left upper arm, the length of the right upper arm, the length of the left forearm and the length of the right forearm, respectively. They are obtained by the following formulas:

分别表示在相机坐标系下，肩部中心点、左肩点、左肘点、左腕点、左手点、右肩点、右肘点、右腕点和右手点的-位置。in,

They respectively represent the positions of the shoulder center point, left shoulder point, left elbow point, left wrist point, left hand point, right shoulder point, right elbow point, right wrist point and right hand point in the camera coordinate system.

Frequency feature and amplitude feature: The frequency feature F _Fre is the number of skeleton image frames in a gait half cycle, and the difference between adjacent local maximum and local minimum is the amplitude feature F _Amp ;

构成基于人体上肢的骨骼特征。Finally, we get a 23-dimensional mixed feature

The composition is based on the skeletal features of the human upper limbs.

Claims (4)

1. A mobile robot target tracking system integrating skeleton recognition and IFace-TLD, characterized in that it comprises obtaining an original color picture (A) of a human body and a skeleton picture (B) of an upper limb through a Kinect sensor, obtaining a target area frame through an IFace-TLD unit (101) for tracking and locating the target on the color picture (A), and a skeleton recognition unit (102) for tracking and locating the target on the skeleton picture (B), and sending the frame to an image target positioning unit (103); the image target positioning unit (103) marks a target area on the original color picture (A) according to the obtained target area frame, and feeds the target area back to the IFace-TLD unit (101); The IFace-TLD unit (101) comprises a tracking part (101.1), a learning part (101.2), a detection part (101.3), and an integrator (101.4) for respectively acquiring an original color image (A), wherein the tracking part (101.1) uses an optical flow tracker to estimate the motion trajectory of a target in the acquired original color image (A) between two adjacent frames, and sends the estimated motion trajectory to the learning part (101.2) and the integrator (101.4) respectively; the detection part (101.3) independently scans and processes all image blocks in the first frame of the acquired original color image (A), separates the target face from the background, and sends the target face to the learning part (101.2) and the integrator (101.4) respectively, and only processes the image target positioning unit (101.3) for the original color images (A) acquired after the first frame. The target area and the surrounding area fed back by the tracking part (101.1) and the detection part (101.3) are scanned and processed to separate the target face and the background, and the target face is sent to the learning part (101.2) and the integrator (101.4) respectively; the integrator (101.4) calculates the confidence coefficient of the most likely target position in the original color image (A) according to the obtained target motion trajectory between two adjacent frames and the target face, and sends the calculation result to the learning part (101.2), and the skeleton recognition unit (102) or the image target positioning unit (103); the learning part (101.2) is trained according to the original color image (A) and the results obtained from the tracking part (101.1), the detection part (101.3) and the integrator (101.4), and the errors in the tracking part (101.1) and the detection part (101.3) are updated and corrected according to the training results; The skeleton recognition unit (102) comprises a motion cycle extraction part (102.1), a skeleton feature extraction part (102.2) and a support vector data description part (102.3), wherein the motion cycle extraction part (102.1) calculates the motion cycle of the human body according to the acquired skeleton image (B), and the skeleton feature extraction part (102.2) calculates the skeleton features within the obtained motion cycle of the human body. When the result output by the integrator (101.4) in the IFace-TLD unit (101) is the target area frame, the skeleton feature extraction part (102.2) obtains The skeletal features are sent to the training part (102.31) in the support vector data description part (102.3) for training. When the result output by the integrator (101.4) in the IFace-TLD unit (101) is empty, the skeletal features obtained by the skeletal feature extraction part (102.2) are sent to the prediction part (102.32) in the support vector data description part (102.3). The prediction part (102.32) predicts the target area frame according to the training result of the training part (102.31), and sends the predicted target area frame to the image target positioning unit (103). 2. According to the mobile robot target tracking system integrating skeleton recognition and IFace-TLD as described in claim 1, it is characterized in that the detection part (101.3) includes a face detection part (101.31) for detecting the area of the face in the original color picture (A) according to the acquired original color picture (A), the target area of the learning part (101.2) and the target area information fed back by the image target positioning unit (103), a face recognition part (101.32) for identifying the target face area according to the area of the face in the original color picture (A) obtained from the face detection part (101.31), and a verifier part (101.33) for judging whether the target face area recognized by the face recognition part (101.32) is correct, and the verification results of the verifier part (101.33) are respectively sent to the learning part (101.2) and the integrator (101.4). 3. The mobile robot target tracking system integrating skeleton recognition and IFace-TLD according to claim 1, characterized in that the motion cycle extraction part (102.1) calculates the motion cycle of the human body according to the acquired skeleton image (B) by using the following formula:

Where dist _k is the distance between the left wrist and shoulder center of the kth frame image in the Kinect coordinate system;

and

Represents the 3D point coordinates of the center of the left wrist and shoulder of the kth frame image; N represents the total number of image frames in the sequence. 4. The mobile robot target tracking system integrating skeleton recognition and IFace-TLD according to claim 1, characterized in that the skeleton feature extraction part (102.2) calculates the skeleton features within the obtained motion cycle of the human body, including: First, the gait half-cycle is defined as T _w , then the skeletal features based on the upper limbs of the human body are expressed as: Trajectory feature: The center point of the shoulder is selected as the fixed point, and the relative positions of other upper limb bone points relative to the fixed point are calculated by the following formula to obtain a 9-dimensional feature P:

in,

represents the position of the jth human skeleton in the tth frame image in the gait half cycle _Tw ,

is represented as:

in,

represents the position of the center point of the shoulder in the camera coordinate system, and the positions of the remaining coordinate points in the upper limb bones are expressed as

The covariance matrix of P is used to represent the trajectory feature matrix F _T containing the walking habits of human beings, and the definition is

in,

and

Represent the trajectory feature matrices of test data and training data respectively;

yes

and

The generalized eigenvalues between

x is the corresponding generalized right eigenvector; Area and distance features: The area feature _FA represents the area of the closed area enclosed by the upper limbs of the human body. The distance feature _FD is represented by the distance between different human body centers. _FA is expressed as:

in,

and

denote the positions of the shoulder center, head, left shoulder, and right shoulder in the tth frame in the gait half cycle Tw _, respectively; To calculate the distance feature _FD , we first calculate the centers of the three closed polygons in the upper limb area. The three center points are the center of the head.

Right hand center

and left hand center

Calculated by the following formula:

Head Center

It is the center of the polygon formed by the center point of the shoulder and the head; the center of the right hand

It is the center of the polygon formed by the right shoulder point, right elbow point and right wrist point; the left hand center

It is the center of the polygon formed by the left shoulder point, left elbow point and left wrist point; the center of the right hand

Head center

and left hand center

The Euclidean distance between f _t ^d1 and f _t ^d2 is written as:

make

In this way, the entire distance feature is expressed as

in,

and

Respectively represent the mean, variance and maximum value of f ^di , where i = 1, 2; Static features: Static features are represented by a 5-dimensional vector F _S = [f _h ,f _lua ,f _rua ,f _lf ,f _rf ] ^T , where f _h represents the height of the target, and f _lua ,f _rua ,f _lf and f _rf represent the length of the left upper arm, the length of the right upper arm, the length of the left forearm and the length of the right forearm, respectively. They are obtained by the following formulas:

in,

Respectively represent the positions of the shoulder center point, left shoulder point, left elbow point, left wrist point, left hand point, right shoulder point, right elbow point, right wrist point and right hand point in the camera coordinate system; Frequency feature and amplitude feature: The frequency feature F _Fre is the number of skeleton image frames in a gait half cycle, and the difference between adjacent local maximum and local minimum is the amplitude feature F _Amp ; Finally, we get a 23-dimensional mixed feature

The composition is based on the skeletal features of the human upper limbs.

Images