CN109614877B

CN109614877B - Method for identifying attribute of pedestrian with shielding in low-resolution monitoring scene

Info

Publication number: CN109614877B
Application number: CN201811370580.8A
Authority: CN
Inventors: 王琼; 张媛; 陶叔银; 徐锦浩
Original assignee: Nanjing University of Science and Technology
Current assignee: Nanjing University of Science and Technology
Priority date: 2018-11-17
Filing date: 2018-11-17
Publication date: 2022-09-27
Anticipated expiration: 2038-11-17
Also published as: CN109614877A

Abstract

The invention discloses a pedestrian attribute identification method with shielding in a low-resolution monitoring scene, which comprises the steps of removing shielding objects in a pedestrian image by utilizing metric learning; transversely cutting the repaired image, and respectively marking the corresponding image blocks as a head and shoulder part, an upper body part and a lower body part of the pedestrian; extracting features of each image of the attribute corresponding part to be identified and expressing each feature as a 16-bin histogram; and finally, taking the 16-bin histogram as a feature vector of the image, and inputting the feature vector of the image into a trained SVM classifier to obtain a recognition result. The invention has better classification effect and higher classification accuracy.

Description

Method for identifying attribute of pedestrian with shielding in low-resolution monitoring scene

Technical Field

The invention belongs to the pedestrian feature identification technology, and particularly relates to a pedestrian attribute identification method with shielding in a low-resolution monitoring scene.

Background

Pedestrians are the most major research subject in monitoring scenarios. It is very significant to identify the basic attributes of a pedestrian. To facilitate the query and retrieval tasks in conventional surveillance systems, it is necessary to manually tag the basic attributes of pedestrians, and then enter attribute tags for the pedestrians of interest. However, this work does not meet the need for pedestrian attribute labeling with enormous monitoring data. Automatic marking of pedestrian attributes in surveillance by computer vision techniques is an effective method.

The invention relates to an algorithm for acquiring pedestrian attributes in an image by processing an input image, and mainly relates to the algorithm steps of image acquisition, image preprocessing, image enhancement, attribute extraction, attribute classification and the like. Where image enhancement and attribute classification are two key issues.

Currently, research has been conducted on the scene of Pedestrian attribute identification, for example, Layne et al uses SVM algorithm to identify Pedestrian attributes in "Person re-identification by attributes", and Deng introduces Pedestrian database (PETA) in "Pedestrian attribute recognition at face distance" to optimize the identification algorithm. However, they lack the identification research in the low-resolution image scene, so there is a certain limitation in the practical monitoring scene.

In addition, people such as Sun also study the gender attribute recognition, for example, people utilize genetic algorithm to regard some feature points of human face as the classification feature, through constructing artificial neural network as the classifier of discernment; local Binary Pattern (LBP) features proposed by Luo solve multi-angle face recognition; amit Jain extracts the human face features by using Independent Component Analysis (ICA) and combines with SVM classifier for recognition. Also, they do not consider application in a monitoring scenario.

The appearance attribute is mainly reflected in understanding of the pedestrian clothes and is also an important content of pedestrian attribute identification. By understanding the appearance of the garment, semantic attributes can be provided, including the color of the garment, style, whether to wear glasses, and whether to carry packaging. In recent years, many researchers have effectively recognized the basic appearance of pedestrian images through understanding of pedestrian clothing, in combination with background information of the environment. Gallagher and Chen et al propose an algorithm for learning a garment model from a set of images to solve the problem of how to segment regions of the garment from the images. But also if the pedestrian image is occluded, the clothing region will be more difficult to segment.

Disclosure of Invention

The invention aims to provide a pedestrian attribute identification method with shielding in a low-resolution monitoring scene.

The technical solution for realizing the invention is as follows: the method for identifying the attribute of the pedestrian with shielding in the low-resolution monitoring scene comprises the following specific steps:

step 1, carrying out restoration operation on a blocked pedestrian image by utilizing metric learning, and removing a blocking object in the pedestrian image;

step 2, transversely cutting the repaired image, and respectively marking the corresponding image blocks as a head and shoulder part, an upper body part and a lower body part of the pedestrian;

step 3, determining the attribute to be identified, and extracting the characteristics of each image of the part corresponding to the attribute, wherein the specific characteristics comprise: color features, LBP features, Gabor filter features, Schmid filter features, and representing each feature as a 16-bin histogram;

and 4, taking the 16-bin histogram in the step 3 as a feature vector of the image, and inputting the feature vector of the image into a trained SVM classifier to obtain a recognition result.

Preferably, the specific steps of performing the repairing operation on the occluded pedestrian image by using metric learning in step 1 are as follows:

step 1-1, a pedestrian image data set P with complete information and without being shielded is given;

step 1-2, giving an occluded pedestrian image I, marking an occluded part of the image I, finding a pedestrian image T which is most similar to an unoccluded part of the image I by using a Hash algorithm in a pedestrian image data set P, and complementing the occluded part of the pedestrian image I by using the pedestrian image T.

Preferably, the specific step of using the hash algorithm to find the pedestrian image T most similar to the unobstructed part of the image I in the pedestrian image data set P in step 1-2 is as follows:

step 1-2-1, cutting out an unobstructed part I1 of the pedestrian image I according to the obstruction mark;

step 1-2-2, graying and normalizing the uncovered part I1 of the image to the size of N x N, and marking as an image G;

1-2-3, calculating a gray average value a of all pixels in the image G, comparing the gray value of each pixel point in the image G with the average value a, if the gray value of the pixel point is greater than the average value a, marking the pixel point as '1', and if the gray value of the pixel point is less than the average value a, marking the pixel point as '0', so as to convert the unoccluded part I1 into a character string in a 01 form, and marking the character string as a Hash fingerprint of the unoccluded part I1;

step 1-2-4, cutting out an image P1 at the position corresponding to the image unoccluded part I1 in each image in the data set P, and obtaining a Hash fingerprint of an image P1 by using the method of the step 1-2-2 to the step 1-2-3;

and 1-2-5, respectively comparing the similarity of the Hash fingerprint of the unoccluded part I1 of the image with the Hash fingerprint of each image in the image set P1, and finding out the pedestrian image T which is most similar to the image I in the data set P.

Preferably, the similarity between the Hash fingerprint of the image I in step 1-2-3 and the Hash fingerprint of each image in the data set P is a "hamming distance", which is defined as the number of different characters at corresponding positions in two compared character strings with equal length, and the smaller the number, the more similar the two images.

Preferably, the value range of N is 8-12.

Preferably, the correspondence of the attributes to be identified in step 3 to the body part is not: the attributes "glasses" and "hair" belong to the "head and shoulder parts"; attributes "V-collar", "trademark" and "backpack" belong to the "upper body part", attributes "shorts" and "jeans" belong to the "lower body part", and for global attributes "male" and "female" belong to the entire pedestrian image.

Preferably, the kernel function of the SVM classifier in step 4 is a histogram cross kernel, and the trained SVM classifier is an optimal classifier obtained by training using feature vectors of corresponding attributes of an unoccluded image set as a training set.

Compared with the prior art, the invention has the following remarkable advantages: (1) the invention has better classification effect and higher classification accuracy; (2) the invention has stronger robustness when facing the pedestrian image with low resolution; (3) according to the method, the occluded image can be recovered for attribute classification by using metric learning, and the identification effect is better.

The present invention is described in further detail below with reference to the attached drawing figures.

Drawings

FIG. 1 is a general flow chart of the method for identifying attributes of pedestrians with occlusion in a monitored scene at low resolution of the present invention.

Fig. 2 is an RGB spatial image and respective channel images.

Fig. 3 is a YCbCr space image and respective channel images.

Fig. 4 is an HSV aerial image and respective lane images.

Fig. 5 shows LBP features of an image, left being an RGB image, middle being a grayscale image, and right being an LBP map.

FIG. 6 is an image feature map extracted by a Gabor filter under different parameters

FIG. 7 is an image feature map extracted by a Schmid filter under different parameters

Detailed Description

The method for identifying the attribute of the pedestrian with the shielding in the low-resolution monitoring scene comprises the following specific steps:

step 1, repairing the shielded pedestrian image by using metric learning to remove the shielding object in the pedestrian image, and the method specifically comprises the following steps:

In a further embodiment, the specific step of using a hash algorithm to find the pedestrian image T most similar to the unobstructed part of the image I in the pedestrian image data set P in step 1-2 is as follows:

step 1-2-1, cutting out an unoccluded part I1 of the pedestrian image I according to the occlusion mark;

step 1-2-2, graying and normalizing the I1 of the part, which is not shielded, of the image to the size of N x N, and recording the size of N x N as an image G;

step 1-2-4, cutting out an image P1 at the position corresponding to the unoccluded part I1 of each image in the data set P, and obtaining a Hash fingerprint of the image P1 by using the method of the step 1-2-2 to the step 1-2-3;

and 1-2-5, respectively comparing the similarity of the Hash fingerprint of the unoccluded part I1 of the image with the Hash fingerprint of each image in the image set P1, and finding out the pedestrian image T which is most similar to the image I in the image P1.

In some embodiments, the similarity between the Hash fingerprint of image I and the Hash fingerprint of each image in dataset P is a "hamming distance", which is defined as the number of different characters at corresponding positions in two equal-length character strings being compared, with fewer characters being more similar for the two images.

Step 2, transversely cutting the repaired image, and respectively marking the corresponding image blocks as a head and shoulder part, an upper body part and a lower body part of the pedestrian; in some embodiments, the pedestrian image is transversely and averagely cut into 10 blocks, the 10 blocks are marked as 1-10 blocks, the 1 st block to the 3 rd block are marked as a pedestrian 'head and shoulder part', the 3 rd block to the 7 th block are marked as a pedestrian 'upper body part', and the 6 th block to the 10 th block are marked as a pedestrian 'lower body part';

step 3, determining the attribute to be identified, and extracting the characteristics of each image of the part corresponding to the attribute, wherein the specific characteristics comprise: color features, LBP features, Gabor filter features, Schmid filter features, and representing each feature as a 16-bin histogram; in some embodiments, the correspondence of the attribute to be identified to the body part is: the attributes "glasses" and "hair" belong to the "head and shoulder parts"; attributes "V-collar", "trademark" and "backpack" belong to the "upper body part", attributes "shorts" and "jeans" belong to the "lower body part", and for global attributes "male" and "female" belong to the entire pedestrian image.

In some embodiments, the training process of the trained SVM classifier is:

transversely cutting each image of the image set which is not shielded, and respectively marking the corresponding image blocks as a head and shoulder part, an upper body part and a lower body part of the pedestrian; and extracting the characteristics of each image of the part corresponding to the corresponding attribute, and representing each characteristic as a 16-bin histogram as a training set and a characteristic vector of the image of the test set to train the SVM classifier. And evaluating the classification result of the SVM classifier by using 5-fold cross validation, and taking the model with the optimal classification effect as the optimal SVM classifier.

Examples

As shown in fig. 1, the method for identifying the attribute of a pedestrian with occlusion in a low-resolution monitoring scene specifically comprises the following steps:

step 1, utilizing metric learning to carry out restoration operation on a blocked pedestrian image and removing a blocking object in the pedestrian image, and the method specifically comprises the following steps:

step 1-2, giving an occluded pedestrian image I, marking the occluded part of the image I, and cutting out an unoccluded part I1 of the pedestrian image I according to the occlusion mark;

graying and normalizing the part I1, which is not shielded by the image, to the size of N x N, and recording as an image G;

calculating the gray average value a of all pixels in the image G, comparing the gray value of each pixel point in the image G with the average value a, if the gray value of the pixel point is greater than the average value a, marking the pixel point as '1', and if the gray value of the pixel point is less than the average value a, marking the pixel point as '0', so as to convert the unoccluded part I1 into a character string in a 01 form, and marking the character string as a Hash fingerprint of the unoccluded part I1;

cutting out an image P1 at the position corresponding to the unoccluded part I1 of each image in the data set P, and obtaining a Hash fingerprint of the image P1 by using the method of the step 1-2-2 to the step 1-2-3;

And (4) complementing the blocked part of the pedestrian image I by using an image at the position corresponding to the blocked part of the image I in the pedestrian image T.

Step 2, cutting the repaired image, transversely and averagely cutting the pedestrian image into 10 pieces, marking the 10 pieces as 1-10 pieces, marking the 1 st piece to the 3 rd piece as a head and shoulder part of a pedestrian, marking the 3 rd piece to the 7 th piece as an upper body part of the pedestrian, and marking the 6 th piece to the 10 th piece as a lower body part of the pedestrian; different attributes are distributed to different body parts, the attributes "glasses" and "hair" belong to the "head and shoulder parts", the attributes "V collar", "trademark" and "backpack" belong to the "upper body part", the attributes "shorts" and "jeans" belong to the "lower body part", and for the global attributes "male" and "female" belong to the entire pedestrian image.

as shown in fig. 2, 3, and 4, it can be known that different channels in different color spaces have different color characteristics. The local texture features of the LBP feature extraction image are shown in fig. 5. The parameters of the Gabor filter are selected as shown in table 1, and the images extracted by the Gabor filter selected as shown in table 1 are shown in fig. 6. The parameters of the Schmid filter were chosen as shown in table 2, and the images extracted by the Gabor filter chosen as a parameter in table 1 are shown in fig. 7.

TABLE 1

TABLE 2

And 4, taking the 16-bin histogram in the step 3 as a feature vector of the image, and inputting the feature vector of the image into a trained SVM classifier to obtain a recognition result, wherein a kernel function of the SVM classifier is a histogram cross kernel.

TABLE 3

In this embodiment, the feature vectors of the attributes corresponding to the sub data set VIPeR in the PETA data set are used as the training set and the test set of the attribute classifier. The VIPeR dataset details are shown in table 3. In this embodiment, feature vectors of the training set and the test set are mapped to a high-dimensional space through a histogram cross kernel function, and are classified by using an SVM classifier.

The histogram cross kernel function is defined as:

in the formula, n is the number of training set samples, (x), (i), y (i) are all training set samples.

In this embodiment, a classifier is trained for each attribute, that is, a data set sample is cut, a pedestrian image is transversely and averagely cut into 10 blocks, the 10 blocks are marked as 1-10 blocks, the 1 st block to the 3 rd block are marked as a pedestrian 'head-shoulder part', the 3 rd block to the 7 th block are marked as a pedestrian 'upper body part', the 6 th block to the 10 th block are marked as a pedestrian 'lower body part', and the features of each image of the corresponding part of the attribute are extracted, wherein the specific features include: the method comprises the steps of representing each feature as a 16-bin histogram as a feature vector of an image, for unbalanced attribute training, processing unbalanced data by using random downsampling, evaluating the quality of a model by using 5-fold cross validation, and finally saving the model with the optimal classification effect as an optimal SVM classifier. Eight attributes are selected in this embodiment to test the performance of the attribute classification. For each attribute, the present embodiment samples the training data 10 times, and takes the average of the ten results as the final classification result. In this embodiment, the recognition accuracy is shown in table 4, comparing the method proposed in "Person re-identification by attributes".

TABLE 4

Claims

1. The method for identifying the attribute of the pedestrian with shielding in the low-resolution monitoring scene is characterized by comprising the following specific steps of:

step 1-2, giving a blocked pedestrian image I and marking the blocked part of the image I, finding a pedestrian image T which is most similar to the unblocked part of the image I by using a Hash algorithm in a pedestrian image data set P, and complementing the blocked part of the pedestrian image I by using the pedestrian image T, wherein the specific steps are as follows:

step 1-2-4, cutting out images at positions corresponding to the image unoccluded part I1 in each image in the data set P to form an image set P1, and obtaining a Hash fingerprint of the image P1 by using the methods of the step 1-2-2 to the step 1-2-3;

1-2-5, respectively comparing the similarity of the Hash fingerprint of the unoccluded part I1 of the image with the Hash fingerprint of each image in an image set P1, and finding out a pedestrian image T which is most similar to the image I in the data set P;

2. The method for identifying the attribute of the blocked pedestrian in the low-resolution monitored scene according to claim 1, wherein the similarity between the Hash fingerprint of the image I in the step 1-2-3 and the Hash fingerprint of each image in the data set P is a hamming distance, the hamming distance is defined as the number of different characters at corresponding positions in two compared equal-length character strings, and the smaller the number is, the more similar the two images are.

3. The method for identifying the attributes of the pedestrians with the shelters in the low-resolution monitoring scene according to claim 1, wherein the value range of N is 8-12.

4. The method for identifying the attribute of the pedestrian with the occlusion in the low-resolution monitored scene according to claim 1, wherein the corresponding relationship between the attribute to be identified in the step 3 and the body part is not: the attributes "glasses" and "hair" belong to the "head and shoulder parts"; attributes "V-collar", "trademark" and "backpack" belong to the "upper body part", attributes "shorts" and "jeans" belong to the "lower body part", and for global attributes "male" and "female" belong to the entire pedestrian image.

5. The method for identifying the attribute of the pedestrian with the occlusion in the low-resolution monitoring scene according to claim 1, wherein a kernel function of the SVM classifier in the step 4 is a histogram cross kernel, and the trained SVM classifier is an optimal classifier obtained by training by using a feature vector of a corresponding attribute of an image set which is not occluded as a training set.