CN111046789A - Pedestrian re-identification method - Google Patents

Pedestrian re-identification method Download PDF

Info

Publication number
CN111046789A
CN111046789A CN201911260981.2A CN201911260981A CN111046789A CN 111046789 A CN111046789 A CN 111046789A CN 201911260981 A CN201911260981 A CN 201911260981A CN 111046789 A CN111046789 A CN 111046789A
Authority
CN
China
Prior art keywords
pedestrian
color
image
extracting
features
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201911260981.2A
Other languages
Chinese (zh)
Inventor
闫保中
王帅帅
王晨宇
韩旭东
何伟
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Harbin Engineering University
Original Assignee
Harbin Engineering University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Harbin Engineering University filed Critical Harbin Engineering University
Priority to CN201911260981.2A priority Critical patent/CN111046789A/en
Publication of CN111046789A publication Critical patent/CN111046789A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/103—Static body considered as a whole, e.g. static pedestrian or occupant recognition
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/25—Fusion techniques
    • G06F18/253—Fusion techniques of extracted features
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/46—Descriptors for shape, contour or point-related descriptors, e.g. scale invariant feature transform [SIFT] or bags of words [BoW]; Salient regional features
    • G06V10/462—Salient features, e.g. scale invariant feature transforms [SIFT]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/56—Extraction of image or video features relating to colour
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G06V20/52—Surveillance or monitoring of activities, e.g. for recognising suspicious objects

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Multimedia (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Evolutionary Computation (AREA)
  • Evolutionary Biology (AREA)
  • General Engineering & Computer Science (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Artificial Intelligence (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Human Computer Interaction (AREA)
  • Image Analysis (AREA)

Abstract

The invention discloses a pedestrian re-identification method, which comprises the following steps: rapidly obtaining a moving target area by using a Vibe moving target detection algorithm; carrying out pedestrian detection on the candidate region by adopting an improved DPM model; quantizing the colors into 9 common colors by adopting a new HSV space nonlinear quantization strategy; tracking the pedestrians detected by the DPM, respectively extracting CN color characteristics of the upper and lower bodies of the pedestrians according to the height proportion of the pedestrians, and storing sequence information into a pedestrian database; firstly, adopting a pre-recognition method for the target to be detected, extracting color characteristics of the upper half body and the lower half body, comparing the color characteristics with a pedestrian database, and removing irrelevant targets; accurately identifying targeted pedestrians, and fusing CN middle-level color features and HOG, HSV and other low-level color features; and (5) carrying out similarity calculation by adopting a simple metric learning algorithm. The invention can effectively cope with the influences of illumination, resolution, pedestrian posture and the like, simultaneously provides a pre-recognition strategy and improves the detection efficiency.

Description

Pedestrian re-identification method
Technical Field
The invention belongs to the field of image search, relates to a pedestrian re-identification method, and particularly relates to a pedestrian detection method for improving a DPM (differential pulse-width modulation) model and a pedestrian re-identification method based on multi-feature fusion.
Background
In recent years, surveillance video networks are widely applied in various industries, so that the generated massive video data cannot meet the requirements of people only by using traditional manual processing, and an intelligent surveillance system combined with a computer vision technology becomes an important way for solving the problem. As an important task in an intelligent monitoring system, pedestrian Re-identification (Person Re-ID) is gradually attracting attention, and the pedestrian Re-identification refers to a process of matching pedestrian images under different cameras and finding a target pedestrian in videos shot by different cameras at different times. The research on the related technology and engineering application has important academic significance and great application value.
At present, the pedestrian re-identification recognition rate based on the data set is already high, but the re-identification technology is still rarely applied to practical application, and the pedestrian re-identification mainly has the following problems.
(1) In an actual recognition system, pedestrian detection is an indispensable part, and the result of pedestrian detection has a great influence on the result of pedestrian re-recognition. Most of the current researches on pedestrian re-identification rely on the processed data set which is already cut, and the method is greatly different from the practical application. In practical application, pedestrians in videos need to be processed, the pedestrians need to be extracted by using a pedestrian detection algorithm, but due to the influences of pedestrian shielding, deformation and the like, a good detection effect cannot be obtained often.
(2) Pedestrian re-identification mainly processes pedestrians in a monitoring video, and is limited by cost, storage memory and the like, the resolution ratio of a monitoring camera is often very low, so that people cannot utilize face features to perform pedestrian re-identification, and only can study the body features of the pedestrians. Meanwhile, pedestrians may come from different cameras, and due to the fact that shooting environments are different, the sizes of the pedestrians are changed, and the same pedestrian images may have great difference.
(3) At present, the people with higher pedestrian re-identification precision use deep learning re-identification algorithm. However, the re-recognition algorithm based on deep learning needs a very large pedestrian database, so that the requirements on the performance of a computer are extremely high, the training time is very long, different models need to be designed according to different problems, and the method is not favorable for popularization. Meanwhile, the deep learning method has the problem of large subjectivity in parameter adjustment.
Disclosure of Invention
In view of the prior art, the technical problem to be solved by the present invention is to provide a pedestrian re-identification method which can better detect the blocked pedestrian, improve the detection efficiency in case of a large data volume, and improve the accuracy of pedestrian re-identification.
In order to solve the technical problem, the pedestrian re-identification method comprises the following steps:
s1: acquiring a pedestrian monitoring video by using monitoring equipment;
s2: performing background modeling by using a Vibe algorithm to obtain a moving target area including pedestrians and automobiles in a detection video;
s3: carrying out pedestrian fine inspection by using an improved DPM model;
s4: a new HSV space quantization algorithm is adopted to divide the color space into 9 colors;
s5: tracking the pedestrian detected in the video, extracting the CN color characteristics of the upper and lower bodies, and storing the CN color characteristics into a database;
s6: extracting the colors of the upper and lower half bodies of the pedestrian target to be detected by adopting a pedestrian pre-recognition strategy, and comparing the colors with the data in a pedestrian database to obtain a quasi-target pedestrian sequence;
s7: and a new feature fusion algorithm and a simple metric learning algorithm are adopted to accurately detect the target pedestrian.
The invention also includes:
1, S2, the background modeling is performed by using the Vibe algorithm, and the specific steps of obtaining the moving target area in the detected video are as follows:
s2.1: in the training stage, a background sample set is established through a first frame image sequence of a monitoring video to initialize a background model;
s2.2: a detection stage, performing foreground detection, taking a pixel value of a next frame of image and a background sample set of an object to perform distance calculation, and calculating the distance between the background image and the image to be detected to obtain a moving foreground;
s2.3: and updating the model, namely updating the background model by combining a moving point counting strategy, and updating a point if the point is continuously detected as a moving point.
The step of performing pedestrian fine inspection by using the improved DPM model in S3 is specifically as follows:
s3.1: the PCA-HOG is combined with the LBP characteristic to express the pedestrian, and the HL characteristic corresponding to each image is expressed as
Figure BDA0002311569050000021
Wherein the HOG represents the characteristic of the HOG,
Figure BDA0002311569050000022
represents the LBP signature;
s3.2: and (3) carrying out DPM model training, endowing different components with different weights, and meeting the requirement of the improved component model target response score:
Figure BDA0002311569050000023
wherein, score (p)i) Represents the ith component target response score, F0Representing a root filter, and
Figure BDA0002311569050000024
multiplying to obtain a score, the weight of each root part filter being wiB represents the value of the deviation, FiRepresents the ith component filter, H represents the pyramid of the LBP + HOG fusion feature, piModel representing the ith part, diWhich represents the coefficient of the offset,
Figure BDA0002311569050000025
represents the offset penalty for the part and n represents the total number of parts.
S3.3: and carrying out pedestrian fine inspection on the trained DPM in the moving target area to obtain a pedestrian rectangular frame.
3, S4, adopting a new HSV space quantization algorithm to divide the color space into 9 colors, specifically:
s4.1: dividing HSV color space by using a bright color separation strategy, and dividing the HSV color space into a plurality of color discs V according to the size of V component valuesi，i＝1,2,…,m；
S4.2: dividing the color disc obtained in S4.1, and dividing V according to the size of S component valueiColor wheel classification into color wheel VSi j，j＝1,2,…,n；
S4.3 at VSijGenerating color blocks according to the value of the hue H component in the space, fixing S, V the value of the component, adjusting the size of the H component, and respectively determining the color label and the corresponding H interval of each block through artificial observation; in addition, the three colors of black, white and gray are tested by a controlled variable method, and when the brightness is more than or equal to V, the brightness is more than or equal to 0<At 40, the region appears black regardless of the hue H and saturation S values, when 40<V is less than or equal to 220 and is less than or equal to 0 and S<35, the area appears gray, 220<V is less than or equal to 255 and S is less than or equal to 0<25 area appears white; if the H, S, V value is other values, determining the corresponding color according to the H and S values, generating a color comparison table through manual marking, and obtaining the mapping from the HSV space to the 9 index colors through the color comparison table.
S5, the method includes the steps of tracking pedestrians detected in the video, extracting CN color features of the upper and lower bodies, and storing the CN color features into a database specifically:
and for the pedestrian rectangular frame obtained in the step S3, selecting 20% -50% of the area of the image from top to bottom as the upper half body area of the pedestrian, selecting 55% -75% of the area as the lower half body area of the pedestrian, traversing each pixel point of the selected area, judging the main color of the interval according to the step S4, storing the main color and the tracked pedestrian sequence into a pedestrian database, and setting the size of the pedestrian image to be 128 pixels multiplied by 48 pixels.
5, S6, the method for pre-identifying the pedestrian specifically comprises the following steps: accurate detection of target pedestrians by adopting new feature fusion algorithm and simple metric learning algorithm
S6.1: extracting the main color of the upper and lower half bodies of the target pedestrian image, and recording as QA、QB；
S6.2: calculating the main face of the upper body of the pedestrian to be detectedColor, denoted as SAWhen S isA＝QAIf so, indicating that the main colors of the upper body areas of the two images are consistent, continuously executing S6.3 to calculate the lower body area, otherwise executing S6.4;
s6.3: calculating the main color of the lower body region and recording as SBComparing with the dominant color of the template image if SB＝QBThe color of the lower half area of the two images is consistent, and the images are set as quasi-target images.
S6.4: the next frame of image is input and the process returns to S6.2 to re-identify the new image.
And obtaining the quasi-target pedestrian sequence through the steps.
7. The pedestrian re-identification method according to claim 1, characterized in that: s7, the accurate detection of the target pedestrian by adopting the new feature fusion algorithm and the simple metric learning algorithm is specifically as follows:
s7.1: extracting overall characteristics: firstly, HOG feature extraction is extracted; extracting HSV spatial features which comprise H, S, V three channels, and extracting HSV color histogram features of one scale from the image;
s7.2: the local feature extraction specifically comprises the following steps:
(1) firstly, zooming each pedestrian image into a size of 128 × 48, horizontally dividing each pedestrian image into 5 blocks according to the size of each block being 16 × 48, wherein 5 × 3 is equal to 15 blocks of areas, then adopting 16 × 16 sub-windows to sequentially slide on the pedestrian images, and setting the step length as 8 pixels to obtain 5 overlapped blocks in total;
(2) extracting color features, selecting 25 bins of each dimension of the CN features, extracting color namespace features of 9 dimensions for each block, wherein each dimension is divided into 25 bins, and finally obtaining a 225-dimensional CN color feature histogram;
(3) extracting SILTP texture features, extracting texture features from two scales, selecting the number of sampling pixel points to be 4, selecting the proportion coefficient to be 0.3, and selecting the radius of a circular neighborhood to be 3 and 5 respectively, namely respectively selecting the sampling pixel points from the two scales
Figure BDA0002311569050000041
And
Figure BDA0002311569050000042
the two scales are used for extracting the texture features of the pedestrian image, and a 3 is obtained through calculation4X 2 ═ 162 dimensional texture histogram; adding corresponding dimensions to obtain local features with dimensions 225+162 being 387, and then combining 9 histograms with dimensions 387 in the image block into 1;
s7.3: three-layer pyramid sampling: carrying out two times of reduction sampling on a pedestrian image, respectively reducing the pedestrian image by 2 times and 4 times, wherein the size of an original image is 128 multiplied by 48, the original image is respectively changed into sizes of 64 multiplied by 24 and 32 multiplied by 12 after the two times of reduction, the cell size is changed into sizes of 4 multiplied by 4 and 2 multiplied by 2 after the cell size is reduced from 8 multiplied by 8 in HOG feature extraction, the HOG feature is sampled for three times, the total size is 2700 multiplied by 3 which is 8100 dimension HOG feature, the HSV feature is 512 dimension, after the features such as SILTP and CN are respectively extracted, the total size of the local feature dimension 7740 dimension of each pedestrian image is calculated, and then feature fusion is carried out;
s7.4: and carrying out similarity calculation by comparing and selecting an XQDA metric learning algorithm.
The invention has the beneficial effects that:
1. the pedestrian re-identification not only uses the cut data set, but also combines the pedestrian detection and the pedestrian re-identification in the video, thereby being more in line with the practical application;
2. the method has the advantages that the Vibe algorithm is used for reducing the range of pedestrians in the searched image, then the improved DPM model is used for carrying out pedestrian fine inspection, the PCA-HOG is provided for solving the problem of insufficient description capability of a single HOG feature, the LBP fusion feature is combined, different weights are given to different components, different components have better distinguishability, and shielded pedestrians can be better detected;
3. a new color naming space is provided, the color space is quantized into 9 common colors, and an algorithm is easy to implement and can well describe the colors of clothes of pedestrians;
4. a pedestrian pre-identification strategy is provided, so that the detection efficiency in the case of large data volume is improved;
5. the intermediate-level features and the low-level features are fused, so that the accuracy of pedestrian re-identification is improved.
Drawings
Fig. 1 is a general flow diagram of a pedestrian re-identification system according to an embodiment of the invention.
Fig. 2 is a flow chart of a DPM pedestrian detection algorithm according to an embodiment of the invention.
Fig. 3 is a schematic diagram of a pedestrian pre-recognition process according to an embodiment of the invention.
FIG. 4 is a schematic diagram of a development interface of a pedestrian re-identification system according to an embodiment of the invention.
Detailed Description
The following further describes the embodiments of the present invention with reference to the drawings.
The invention discloses a method for re-identifying pedestrians in a video by adopting a traditional image identification algorithm, which comprises the following steps of: rapidly obtaining a moving target area by using a Vibe moving target detection algorithm; carrying out pedestrian detection on the candidate region by adopting an improved DPM model; quantizing the colors into 9 common colors by adopting a new HSV space nonlinear quantization strategy; tracking the pedestrians detected by the DPM, respectively extracting CN color characteristics of the upper and lower bodies of the pedestrians according to the height proportion of the pedestrians, and storing sequence information into a pedestrian database; a pre-recognition method is adopted for the target to be detected, the color characteristics of the upper half body and the lower half body are extracted and compared with a pedestrian database, irrelevant targets are eliminated, and the detection efficiency is improved; accurately identifying the target pedestrian, and fusing CN middle-level color features and HOG, HSV and other low-level color features by adopting a new feature fusion algorithm; similarity calculation is carried out by adopting a simple metric learning algorithm; and finally, developing a test system based on Qt and Opencv to complete verification of the algorithm. The technical scheme of the invention can realize stable detection and tracking of pedestrians under different backgrounds, the new CN characteristics and the characteristic fusion algorithm can effectively cope with the influences of illumination, resolution, pedestrian posture and the like, and meanwhile, the pre-recognition strategy is provided, so that the detection efficiency is improved, and the pedestrian detection and the pedestrian re-recognition are combined together. Meanwhile, the influence of interference such as illumination and shielding is reduced to a certain extent, and the method comprises the following steps:
s1: acquiring a pedestrian monitoring video by using monitoring equipment;
s2: obtaining a moving area in a detected video by using a Vibe algorithm to obtain a moving pedestrian area;
s3: carrying out pedestrian fine inspection by using an improved DPM model;
s4: a new HSV space quantization algorithm is adopted to divide the color space into 9 common colors;
s5: tracking the pedestrian detected in the video, extracting the CN color characteristics of the upper and lower bodies, and storing the CN color characteristics into a database;
s6: and extracting the colors of the upper and lower half bodies of the pedestrian target to be detected by adopting a pedestrian pre-recognition strategy, and comparing the colors with the data in the pedestrian database to obtain a quasi-target pedestrian sequence.
S7: a new feature fusion algorithm and a simple metric learning algorithm are adopted to accurately detect the target pedestrian;
in the step S1, a pedestrian monitoring video is obtained by using a commonly used monitoring camera.
Detecting the surveillance video obtained in step S1, detecting a moving area in the video, including a pedestrian, an automobile, and the like, specifically including:
step 2.1, selecting a Vibe moving target detection algorithm to detect a video to obtain a foreground area of each frame of picture;
step 2.2, performing relevant processing on the foreground area to obtain a moving target area including pedestrians, automobiles and the like;
step S3 is to perform pedestrian reduction on the target region obtained in step S2, and mainly use an improved DPM model, and the specific process includes:
and 3.1, LBP + PCA-HOG fusion characteristics are adopted to replace the traditional single HOG characteristics, so that the pedestrian is better described.
And 3.2, training the DPM model, and giving different weights to different parts, so that the different parts have stronger distinguishability and better detect the target pedestrian.
And 3.3, carrying out pedestrian fine inspection on the trained DPM in the moving target area to obtain a pedestrian rectangular frame.
In S4, a non-linear quantization method is used. Aiming at the characteristics of the HSV color space, the HSV color space is divided into 9 basic colors of black, gray, white, red, yellow, blue, green, cyan, purple and the like. In order to better divide the color regions to which the 9 colors respectively belong from the HSV space, a generative color table construction method is adopted, and the construction strategy is specifically shown in the table above. According to actual needs, the HSV space is divided into 9 colors and marked with different numerical values, and one code corresponds to one color.
After the pedestrian rectangular frame is determined in step S3, the upper and lower body images of the pedestrian are obtained according to the normal pedestrian ratio, and the information below the knee is not selected in order to reduce the disturbance. From top to bottom, 20% to 50% of the image is selected as the upper half area of the pedestrian and 55% to 75% is selected as the lower half area, each pixel point of the selected area is traversed, the main color of the section is judged according to step S4, the main color is stored in the pedestrian database together with the tracked pedestrian sequence, and the size of the pedestrian image is set to 128 pixels × 48 pixels (hereinafter, both are abbreviated as 128 × 48).
And step S6, processing the pedestrian image to be detected, extracting the main colors of the upper and lower bodies of the target pedestrian image according to the pedestrian partition strategy in the step S5 and the color name space characteristics in the step S4, and comparing the main colors with the color characteristic indexes in the pedestrian database to obtain the pedestrian sequence to be selected.
And step S7, fusing the middle-level color features with the low-level HOG, HSV and SILTP features to obtain more accurate pedestrian features, and then performing similarity calculation to obtain a sequence of the target pedestrian.
And system development is carried out by combining Opencv and Qt, and the system development comprises two parts, namely pedestrian detection and pedestrian re-identification. The pedestrian detection section implements the video pedestrian detection algorithm of steps S2 to S5 to obtain a pedestrian database. The pedestrian re-identification section includes pedestrian pre-identification, implementing the re-identification algorithm of steps S6 to S7. And meanwhile, simple semantic detection is set. And searching a pedestrian sequence by inputting the colors of the clothes of the upper and lower bodies of the pedestrian.
Fig. 1 is a general flow chart of pedestrian re-identification according to an embodiment of the present invention, which includes the following steps:
s1, obtaining a motion area in the detection video by using a Vibe algorithm to obtain a pedestrian motion area;
specifically, the method for achieving background modeling and completing moving target detection by using the Vibe algorithm mainly comprises the following steps:
(1) a training stage: background initialization is mainly performed. And extracting each pixel point in the single-frame image, and performing background modeling. The initialization of the background model is performed by establishing a background sample set through the first few frame sequences of the surveillance video. The Vibe algorithm can realize the initialization of the model by using a frame of image;
(2) a detection stage: foreground detection is mainly performed. And processing the corresponding video sequence according to the background model obtained in the step, and taking down the pixel value of the frame image and the background sample set of the object to calculate the distance. And calculating the distance between the background image and the image to be detected to obtain the motion foreground.
(3) Updating the model: due to the fact that the motion state of the target in the video changes at any time and the influence of the factors of illumination and wind blowing, the background model needs to be updated timely, and errors are reduced.
S2, carrying out pedestrian fine inspection by using the improved DPM model;
specifically, as shown in fig. 2, first, in the conventional DPM model, only the HOG features are used to represent the pedestrian features, and the HOG features do not perform well in the case of complex background, etc., which limits the performance of the DPM model. The design therefore uses PCA-HOG in combination with LBP features to express pedestrians. And connecting the feature vectors in series by adopting a direct fusion method. The HL feature corresponding to each image is expressed as
Figure BDA0002311569050000061
Due to the defects of the traditional DPM model, many missed detection situations occur when pedestrians are shielded. Since each component is given the same weight in the conventional DPM model, if a pedestrian is occluded, the information of the portion disappears. In this case, the blocked portion should be given a smaller weight, so the present invention performs parameter assignment using an adaptive weight, and the important portion is given a larger weight, and the improved component filter score formula is as follows.
Figure BDA0002311569050000071
Wherein wiRepresents a weight, FiRepresents the ith component filter, H represents the pyramid of the fused feature, piRepresenting a model of the ith part. diWhich represents the coefficient of the offset,
Figure BDA0002311569050000072
representing an offset penalty for the component. According to the formula
Figure BDA0002311569050000073
Performing weight adjustment, adjusting therein
Figure BDA0002311569050000074
To change the size of the weights. Through experiments, the method obtains
Figure BDA0002311569050000075
The effect is better when 1.5 is taken, Di,j(x, y) is the score calculated for each component filter. Then using the formula
Figure BDA0002311569050000076
The improved target response score of the component model after normalization (the final response score of a certain component at the target position) is as follows:
Figure BDA0002311569050000077
wherein F0Representing a root filter, and
Figure BDA0002311569050000078
multiplying to obtain corresponding scores, weighting each component filter by wiB represents the deviation value.
Experiments prove that the condition of missing detection due to pedestrian shielding can be well solved by setting a larger weight for the part of the upper body of the pedestrian which is not easy to be shielded, setting a smaller specific gravity for the crus, the feet and the like which are easy to be shielded, and improving the detection rate of the pedestrian. After the pedestrian region detection is carried out by using the Vibe algorithm, the DPM detection algorithm can achieve real time through the cascade detection acceleration and the self-carried cascade of the voc _ release5 version.
S3, dividing the color space into 9 common colors by adopting a new HSV space quantization algorithm;
specifically, the color quantization step is as follows:
(1) dividing HSV color space by using a bright color separation strategy, and dividing the HSV color space into a plurality of color discs V according to the size of V component valuesi(i＝1,2,…,m)；
(2) Continuously dividing the color discs obtained in the step 1, and dividing each V according to the size of the S component valueiThe color discs being divided into several color rings
Figure BDA0002311569050000079
(3) At VSijGenerating a plurality of color blocks according to the value of the hue H component in the space, fixing S, V values of the components, adjusting the size of the H component, and respectively determining the color label and the corresponding H interval of each block through artificial observation. In addition, the three colors of black, white and gray are tested by a controlled variable method, and when the brightness is more than or equal to V, the brightness is more than or equal to 0<40, no matter how the hue H and saturation S values change, the region appears black because the brightness is too low, when 40<V is less than or equal to 220 and is less than or equal to 0 and S<35, the area appears gray, 220<V is less than or equal to 255 and S is less than or equal to 0<The 25 area appears white. If the H, S, V value is other value, look-up table is performed to obtain the corresponding color, for example, V225->Red at 45, etc. After a large number of manual marks, a color comparison table as shown in the table is generated. Mapping of HSV space to 9 index colors can be obtained through a color look-up table.
Color table section example (V225-
Figure BDA0002311569050000081
I.e. a point is mapped to a 9-dimensional vector, with which the image can be encoded. And traversing the pixel points of the 9 colors as shown below, if the pixel points can be searched in the table, adding 1 to the corresponding color of the color histogram, if the pixel points are not in the table, belonging to a color transition region, calculating the probability of belonging to an adjacent color region according to the H component, accumulating the probability in the corresponding colors of the color histogram, and finally obtaining a 9-dimensional color vector, wherein the color vector with the largest value is the main color.
Figure BDA0002311569050000082
S4, performing key frame tracking on the pedestrian detected in the video, extracting CN color characteristics of the upper and lower bodies, and storing the CN color characteristics in a database; specifically, the method comprises the following two steps:
(1) and step S2, carrying out pedestrian fine inspection, extracting the color characteristics of the pedestrians by using the proposed pedestrian partition strategy after the pedestrians are detected, and storing the color characteristics in a pedestrian database.
(2) In order to improve the detection efficiency, the pedestrian detection is carried out once every 10 frames instead of frame-by-frame detection, and the pedestrian is tracked by a pedestrian tracking algorithm in the middle. The pedestrian is tracked by utilizing the front and rear frame correlation strategy, the pedestrian is identified by mainly utilizing the overlapping area of the rectangular frame between the two frames and the color of the pedestrian clothes in the rectangular frame, because the motion range of one pedestrian in the video is not too large, and meanwhile, the colors of the pedestrians in the front and rear frames are not changed greatly, the similarity is utilized for tracking, and the better tracking effect is achieved. Meanwhile, the false detection frame is removed to improve the tracking accuracy, sometimes two rectangular frames may appear on one pedestrian, the invention assumes that if 70% of the areas of the two rectangular frames are overlapped, the false detection is regarded as false detection, and the false detection frame is removed.
And S5, extracting the colors of the upper and lower half bodies of the pedestrian target to be detected by adopting a pedestrian pre-recognition strategy, and comparing the colors with the data in the pedestrian database to obtain a quasi-target pedestrian sequence.
According to fig. 3, the pedestrian pre-identification step is as follows:
1. the detected color characteristics of the upper and lower body clothes exist in the database and are marked as QA、QB。
2. Firstly, calculating the main color of the upper body of the pedestrian to be detected as SAWhen S isA＝QAAnd (3) when the main colors of the upper body regions of the two images are consistent, continuing to calculate the lower body region, and otherwise, executing the step (4).
3. Calculating the main color of the lower body region and recording as SB. Comparing with the dominant color of the template image if SB＝QBWhen the colors of the lower half regions of the two images are consistent, the images are set as quasi-target images.
4. And (5) inputting the next frame of image, and returning to the step 2 to re-identify the new image.
The pedestrian pre-recognition can remove non-target pedestrian images more quickly, and the detection efficiency can be improved well.
S6, accurately detecting the target pedestrian by adopting a new feature fusion algorithm and a simple metric learning algorithm;
specifically, the invention integrates various characteristics to solve the influence caused by pedestrian attitude change, illumination change and the like. The extracted features are fused in a direct serial connection mode. Let t1, t2, t3, t4 denote the extracted feature matrices, respectively. Then they are fused to be characterized as T ═ T1; t 2; t 3; t4 ]. The pedestrian re-identification step is as follows:
1. and extracting the overall characteristics. The resolution of the pedestrian image subjected to the re-recognition experiment is 48 × 128, and when the conventional HOG feature extraction is performed, the size of a cell is set to be 8, and one cell contains a feature vector of 9 dimensions. A block consists of four cells, so a block has a 9 x 4-36 dimensional feature vector. The number of blocks can be calculated to be 5 × 15 to 75 according to the resolution of the image, so that the total number of HOG feature vectors of an image is 75 × 36 to 2700. Extracting HSV space characteristics which have H, S, V three channels. Each channel is divided into 8 bins, resulting in an 8 × 8 × 8 ═ 512-dimensional color histogram. According to the characteristics of the HSV color space, only HSV color histogram features of one scale are extracted from the image.
2. And extracting local features. The method comprises the following steps:
(1) first, each pedestrian image is scaled to a size of 128 × 48, and is horizontally divided into 5 blocks of (1+ (48-16)/8) and 15 blocks of 5 × 3, respectively, for each block size of 16 × 48. Then, the 16 × 16 sub-windows are adopted to slide on the pedestrian images in sequence. The step size is fixed to 8 pixels for a total of 5 overlapping blocks.
(2) And extracting color features. When extracting CN features, we therefore selected 25 bins for each dimension of CN features to perform the experiment. We extract Color Namespace (CN) features of 9 dimensions for each block, respectively, where each dimension is divided into 25 bins, and finally obtain a 225-dimensional CN color feature histogram.
(3) SILTP texture features are extracted. In order to extract the texture features more accurately, the texture features are extracted from two scales, the number of sampling pixel points is 4, the proportion coefficient is 0.3, the radius of a circular neighborhood is respectively 3 and 5, namely the number of the sampling pixel points is respectively selected from
Figure BDA0002311569050000091
Figure BDA0002311569050000092
And
Figure BDA0002311569050000093
the two scales are used for extracting the texture features of the pedestrian image, and a 3 is obtained by calculating4X 2 ═ 162 dimensional texture histogram. Corresponding dimensions are added to obtain local features with dimensions 225+162 and 387, and then 9 histograms with dimensions 387 in the image block are combined into 1 histogram.
3. And sampling by three layers of pyramids. In actual monitoring, the pedestrian is different from the camera in distance, so that the image has scale change, and in order to accurately represent the image, the pedestrian image is subjected to two times of reduction sampling. The reduction was 2 and 4 times, respectively. This results in a total of three scale-sized histogram extraction and region partitioning. The original image size is 128 × 48, and the sizes of the original image are 64 × 24 and 32 × 12 after two times of reduction. In the HOG feature extraction, the dimension of the cell is changed from 8 × 8 to 4 × 4 and 2 × 2 after being reduced according to corresponding multiple change. The HOG feature is divided into three times of sampling, and the total number of the HOG features is 2700 multiplied by 3 which is 8100-dimensional HOG features. The HSV feature is 512 dimensions, after the features of SILTP, CN and the like are respectively extracted, the local feature dimension of each pedestrian image is calculated to be 387 x (15+4+1) ═ 7740 dimensions, and then feature fusion is carried out.
4. Compared with the method of selecting the XQDA metric learning algorithm for similarity calculation, in the normal use process, the extracted original feature dimension is large, the corresponding covariance matrix is large, the dimension can be reduced by using a PCA dimension reduction mode, but the dimension reduction method does not take the factor of distance metric learning into consideration, the effect is poor, and the XQDA metric learning method not only reduces the dimension of the feature, but also performs metric learning by using a fisher method. Meanwhile, the time complexity is low because the algorithm is based on statistical inference.
The above description is only a preferred embodiment of the present invention and does not specifically limit the scope of the present invention. Although the foregoing preferred embodiments have been described in some detail, it should be understood by those skilled in the art that various changes in detail or structure may be made therein without departing from the spirit and scope of the invention as defined by the appended claims.

Claims (7)

1. A pedestrian re-identification method is characterized by comprising the following steps:
s1: acquiring a pedestrian monitoring video by using monitoring equipment;
s2: performing background modeling by using a Vibe algorithm to obtain a moving target area including pedestrians and automobiles in a detection video;
s3: carrying out pedestrian fine inspection by using an improved DPM model;
s4: a new HSV space quantization algorithm is adopted to divide the color space into 9 colors;
s5: tracking the pedestrian detected in the video, extracting the CN color characteristics of the upper and lower bodies, and storing the CN color characteristics into a database;
s6: extracting the colors of the upper and lower half bodies of the pedestrian target to be detected by adopting a pedestrian pre-recognition strategy, and comparing the colors with the data in a pedestrian database to obtain a quasi-target pedestrian sequence;
s7: and a new feature fusion algorithm and a simple metric learning algorithm are adopted to accurately detect the target pedestrian.
2. The pedestrian re-identification method according to claim 1, characterized in that: s2, the background modeling is carried out by using the Vibe algorithm, and the specific steps of obtaining the moving target area in the detected video are as follows:
s2.1: in the training stage, a background sample set is established through a first frame image sequence of a monitoring video to initialize a background model;
s2.2: a detection stage, performing foreground detection, taking a pixel value of a next frame of image and a background sample set of an object to perform distance calculation, and calculating the distance between the background image and the image to be detected to obtain a moving foreground;
s2.3: and updating the model, namely updating the background model by combining a moving point counting strategy, and updating a point if the point is continuously detected as a moving point.
3. The pedestrian re-identification method according to claim 1, characterized in that: s3, the pedestrian fine inspection using the improved DPM model specifically comprises the following steps:
s3.1: the PCA-HOG is combined with the LBP characteristic to express the pedestrian, and the HL characteristic corresponding to each image is expressed as
Figure FDA0002311569040000011
Wherein the HOG represents the characteristic of the HOG,
Figure FDA0002311569040000012
represents the LBP signature;
s3.2: and (3) carrying out DPM model training, endowing different components with different weights, and meeting the requirement of the improved component model target response score:
Figure FDA0002311569040000013
wherein, score (p)i) Represents the ith component target response score, F0Representing a root filter, and
Figure FDA0002311569040000014
multiplying to obtain a score, the weight of each root part filter being wiB represents the value of the deviation, FiRepresents the ith component filter, H represents the pyramid of the LBP + HOG fusion feature, piModel representing the ith part, diWhich represents the coefficient of the offset,
Figure FDA0002311569040000015
represents the offset penalty for the part and n represents the total number of parts.
S3.3: and carrying out pedestrian fine inspection on the trained DPM in the moving target area to obtain a pedestrian rectangular frame.
4. The pedestrian re-identification method according to claim 1, characterized in that: s4, the new HSV space quantization algorithm is adopted to divide the color space into 9 colors, specifically:
s4.1: dividing HSV color space by using a bright color separation strategy, and dividing the HSV color space into a plurality of color discs V according to the size of V component valuesi，i＝1,2,…,m；
S4.2: dividing the color disc obtained in S4.1, and dividing V according to the size of S component valueiColor wheel classification into color wheel VSi j，j＝1,2,…,n；
S4.3 at VSijGenerating color blocks according to the value of the hue H component in the space, fixing S, V the value of the component, adjusting the size of the H component, and respectively determining the color label and the corresponding H interval of each block through artificial observation; in addition, the three colors of black, white and gray are tested by a controlled variable method, and when the lightness 0 is more than or equal to V and less than 40, no matter how the values of the hue H and the saturation S are changed, the area presents black, and when the value is 40<When V is less than or equal to 220 and S is less than or equal to 0 and less than 35, the area is gray, and when V is more than 220 and less than or equal to 255 and S is more than or equal to 0 and less than 25, the area is white; if the H, S, V value is other values, determining the corresponding color according to the H and S values, generating a color comparison table through manual marking, and obtaining the mapping from the HSV space to the 9 index colors through the color comparison table.
5. The pedestrian re-identification method according to claim 1, characterized in that: s5, the method includes the steps of tracking pedestrians detected in the video, extracting CN color features of the upper and lower bodies, and storing the CN color features into a database specifically:
and for the pedestrian rectangular frame obtained in the step S3, selecting 20% -50% of the area of the image from top to bottom as the upper half body area of the pedestrian, selecting 55% -75% of the area as the lower half body area of the pedestrian, traversing each pixel point of the selected area, judging the main color of the interval according to the step S4, storing the main color and the tracked pedestrian sequence into a pedestrian database, and setting the size of the pedestrian image to be 128 pixels multiplied by 48 pixels.
6. The pedestrian re-identification method according to claim 1, characterized in that: s6, the pedestrian pre-identification strategy is specifically as follows: accurate detection of target pedestrians by adopting new feature fusion algorithm and simple metric learning algorithm
S6.1: extracting the main color of the upper and lower half bodies of the target pedestrian image, and recording as QA、QB；
S6.2: calculating the main color of the upper body of the pedestrian to be detected as SAWhen S isA＝QAIf so, indicating that the main colors of the upper body areas of the two images are consistent, continuously executing S6.3 to calculate the lower body area, otherwise executing S6.4;
s6.3: calculating the main color of the lower body region and recording as SBComparing with the dominant color of the template image if SB＝QBThe color of the lower half area of the two images is consistent, and the images are set as quasi-target images.
S6.4: the next frame of image is input and the process returns to S6.2 to re-identify the new image.
And obtaining the quasi-target pedestrian sequence through the steps.
7. The pedestrian re-identification method according to claim 1, characterized in that: s7, the accurate detection of the target pedestrian by adopting the new feature fusion algorithm and the simple metric learning algorithm is specifically as follows:
s7.1: extracting overall characteristics: firstly, HOG feature extraction is extracted; extracting HSV spatial features which comprise H, S, V three channels, and extracting HSV color histogram features of one scale from the image;
s7.2: the local feature extraction specifically comprises the following steps:
(1) firstly, zooming each pedestrian image into a size of 128 × 48, horizontally dividing each pedestrian image into 5 blocks according to the size of each block being 16 × 48, wherein 5 × 3 is equal to 15 blocks of areas, then adopting 16 × 16 sub-windows to sequentially slide on the pedestrian images, and setting the step length as 8 pixels to obtain 5 overlapped blocks in total;
(2) extracting color features, selecting 25 bins of each dimension of the CN features, extracting color namespace features of 9 dimensions for each block, wherein each dimension is divided into 25 bins, and finally obtaining a 225-dimensional CN color feature histogram;
(3) extracting SILTP texture features, extracting texture features from two scales, selecting the number of sampling pixel points to be 4, selecting the proportion coefficient to be 0.3, and selecting the radius of a circular neighborhood to be 3 and 5 respectively, namely respectively selecting the sampling pixel points from the two scales
Figure FDA0002311569040000031
And
Figure FDA0002311569040000032
the two scales are used for extracting the texture features of the pedestrian image, and a 3 is obtained through calculation4X 2 ═ 162 dimensional texture histogram; adding corresponding dimensions to obtain local features with dimensions 225+162 being 387, and then combining 9 histograms with dimensions 387 in the image block into 1;
s7.3: three-layer pyramid sampling: carrying out two times of reduction sampling on a pedestrian image, respectively reducing the pedestrian image by 2 times and 4 times, wherein the size of an original image is 128 multiplied by 48, the original image is respectively changed into sizes of 64 multiplied by 24 and 32 multiplied by 12 after the two times of reduction, the cell size is changed into sizes of 4 multiplied by 4 and 2 multiplied by 2 after the cell size is reduced from 8 multiplied by 8 in HOG feature extraction, the HOG feature is sampled for three times, the total size is 2700 multiplied by 3 which is 8100 dimension HOG feature, the HSV feature is 512 dimension, after the features such as SILTP and CN are respectively extracted, the total size of the local feature dimension 7740 dimension of each pedestrian image is calculated, and then feature fusion is carried out;
s7.4: and carrying out similarity calculation by comparing and selecting an XQDA metric learning algorithm.
CN201911260981.2A 2019-12-10 2019-12-10 Pedestrian re-identification method Pending CN111046789A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201911260981.2A CN111046789A (en) 2019-12-10 2019-12-10 Pedestrian re-identification method

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201911260981.2A CN111046789A (en) 2019-12-10 2019-12-10 Pedestrian re-identification method

Publications (1)

Publication Number Publication Date
CN111046789A true CN111046789A (en) 2020-04-21

Family

ID=70235450

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201911260981.2A Pending CN111046789A (en) 2019-12-10 2019-12-10 Pedestrian re-identification method

Country Status (1)

Country Link
CN (1) CN111046789A (en)

Cited By (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111553214A (en) * 2020-04-20 2020-08-18 哈尔滨工程大学 Method and system for detecting smoking behavior of driver
CN111565300A (en) * 2020-05-22 2020-08-21 深圳市百川安防科技有限公司 Object-based video file processing method, device and system
CN112001251A (en) * 2020-07-22 2020-11-27 山东大学 Pedestrian re-identification method and system based on combination of human body analysis and clothing color
CN112800968A (en) * 2021-01-29 2021-05-14 江苏大学 Method for identifying identity of pig in drinking area based on feature histogram fusion of HOG blocks
CN112926522A (en) * 2021-03-30 2021-06-08 广东省科学院智能制造研究所 Behavior identification method based on skeleton attitude and space-time diagram convolutional network
CN113591697A (en) * 2021-07-30 2021-11-02 上海电科智能系统股份有限公司 Video pedestrian re-identification method based on triple pyramid model and migration fusion
CN113807189A (en) * 2021-08-20 2021-12-17 浙江大学自贡创新中心 Video pedestrian re-identification method based on human body part learning and posture reconstruction
CN114120260A (en) * 2020-09-01 2022-03-01 广州汽车集团股份有限公司 Method and system for identifying travelable area, computer device, and storage medium

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109033955A (en) * 2018-06-15 2018-12-18 中国科学院半导体研究所 A kind of face tracking method and system
CN109271932A (en) * 2018-09-17 2019-01-25 中国电子科技集团公司第二十八研究所 Pedestrian based on color-match recognition methods again

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109033955A (en) * 2018-06-15 2018-12-18 中国科学院半导体研究所 A kind of face tracking method and system
CN109271932A (en) * 2018-09-17 2019-01-25 中国电子科技集团公司第二十八研究所 Pedestrian based on color-match recognition methods again

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
何谐: "基于DSP优化的行人识别算法在智能监控中的研究与应用", 《中国优秀硕士学位论文全文数据库 信息科技辑》 *
王帅帅等: "基于特征融合的行人重识别算法", 《应用科技》 *
陈芝垚: "基于DPM的行人检测和行人特征提取算法研究", 《中国优秀硕士学位论文全文数据库 信息科技辑》 *

Cited By (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111553214A (en) * 2020-04-20 2020-08-18 哈尔滨工程大学 Method and system for detecting smoking behavior of driver
CN111565300A (en) * 2020-05-22 2020-08-21 深圳市百川安防科技有限公司 Object-based video file processing method, device and system
CN112001251A (en) * 2020-07-22 2020-11-27 山东大学 Pedestrian re-identification method and system based on combination of human body analysis and clothing color
CN114120260A (en) * 2020-09-01 2022-03-01 广州汽车集团股份有限公司 Method and system for identifying travelable area, computer device, and storage medium
CN114120260B (en) * 2020-09-01 2024-08-20 广州汽车集团股份有限公司 Method and system for identifying drivable area, computer equipment and storage medium
CN112800968A (en) * 2021-01-29 2021-05-14 江苏大学 Method for identifying identity of pig in drinking area based on feature histogram fusion of HOG blocks
CN112800968B (en) * 2021-01-29 2024-05-14 江苏大学 HOG blocking-based feature histogram fusion method for identifying identity of pigs in drinking area
CN112926522B (en) * 2021-03-30 2023-11-24 广东省科学院智能制造研究所 Behavior recognition method based on skeleton gesture and space-time diagram convolution network
CN112926522A (en) * 2021-03-30 2021-06-08 广东省科学院智能制造研究所 Behavior identification method based on skeleton attitude and space-time diagram convolutional network
CN113591697A (en) * 2021-07-30 2021-11-02 上海电科智能系统股份有限公司 Video pedestrian re-identification method based on triple pyramid model and migration fusion
CN113591697B (en) * 2021-07-30 2024-08-20 上海电科智能系统股份有限公司 Video pedestrian re-identification method based on triple pyramid model and migration fusion
CN113807189B (en) * 2021-08-20 2023-10-27 浙江大学自贡创新中心 Video pedestrian re-recognition method based on human body part learning and gesture reconstruction
CN113807189A (en) * 2021-08-20 2021-12-17 浙江大学自贡创新中心 Video pedestrian re-identification method based on human body part learning and posture reconstruction

Similar Documents

Publication Publication Date Title
CN111046789A (en) Pedestrian re-identification method
CN111767882B (en) Multi-mode pedestrian detection method based on improved YOLO model
CN109584248B (en) Infrared target instance segmentation method based on feature fusion and dense connection network
CN107622229B (en) Video vehicle re-identification method and system based on fusion features
CN107066559B (en) Three-dimensional model retrieval method based on deep learning
CN109684922B (en) Multi-model finished dish identification method based on convolutional neural network
CN113065558A (en) Lightweight small target detection method combined with attention mechanism
CN108520226B (en) Pedestrian re-identification method based on body decomposition and significance detection
CN111310862A (en) Deep neural network license plate positioning method based on image enhancement in complex environment
CN107273832B (en) License plate recognition method and system based on integral channel characteristics and convolutional neural network
CN102214309B (en) Special human body recognition method based on head and shoulder model
CN111260738A (en) Multi-scale target tracking method based on relevant filtering and self-adaptive feature fusion
CN110929593A (en) Real-time significance pedestrian detection method based on detail distinguishing and distinguishing
CN109902576B (en) Training method and application of head and shoulder image classifier
Hechri et al. Automatic detection and recognition of road sign for driver assistance system
CN110334703B (en) Ship detection and identification method in day and night image
CN111199556A (en) Indoor pedestrian detection and tracking method based on camera
CN111274964B (en) Detection method for analyzing water surface pollutants based on visual saliency of unmanned aerial vehicle
CN112818905B (en) Finite pixel vehicle target detection method based on attention and spatio-temporal information
CN104732534B (en) Well-marked target takes method and system in a kind of image
CN105678318A (en) Traffic label matching method and apparatus
Liu et al. Multi-type road marking recognition using adaboost detection and extreme learning machine classification
CN114049572A (en) Detection method for identifying small target
CN113850748A (en) Point cloud quality evaluation system and method
CN113792686B (en) Vehicle re-identification method based on visual representation of invariance across sensors

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
RJ01 Rejection of invention patent application after publication
RJ01 Rejection of invention patent application after publication

Application publication date: 20200421