CN110046669B - Pedestrian retrieval method based on sketch image half-coupling metric identification dictionary learning - Google Patents
Pedestrian retrieval method based on sketch image half-coupling metric identification dictionary learning Download PDFInfo
- Publication number
- CN110046669B CN110046669B CN201910325189.4A CN201910325189A CN110046669B CN 110046669 B CN110046669 B CN 110046669B CN 201910325189 A CN201910325189 A CN 201910325189A CN 110046669 B CN110046669 B CN 110046669B
- Authority
- CN
- China
- Prior art keywords
- learning
- image
- dictionary
- sketch
- image set
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
- G06F16/58—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
- G06F16/583—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/28—Determining representative reference patterns, e.g. by averaging or distorting; Generating dictionaries
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
Abstract
The invention belongs to the technical field of traffic pedestrian image processing, and discloses a pedestrian retrieval method and a system for learning a semi-coupling metric identification dictionary based on a sketch image, wherein a heterogeneous pedestrian retrieval database is established, then characteristic extraction is carried out, characteristics are extracted from an image, and different people are marked by corresponding colors; processing the extracted sample characteristics, introducing a dictionary learning method, and learning a dictionary pair of heterogeneous data; learning a mapping matrix from the sketch image set and the regular image set; and introducing a learning metric of a discrimination algorithm. The invention has the advantages of solving the problem of lack of heterogeneous pedestrian data sets in the field of heterogeneous pedestrian retrieval and providing half-coupling metric discriminative dictionary learning (SMD) for the first time2L) technique. The technique can learn the half-coupling mapping matrix from the heterogeneous sample and dictionary pairs, reducing the difference between heterogeneous data to some extent. The ideal searching effect is achieved on the new SINPID data set.
Description
Technical Field
The invention belongs to the technical field of traffic pedestrian image processing, and particularly relates to a pedestrian retrieval method based on sketch image semi-coupling metric identification dictionary learning.
Background
Currently, the closest prior art:
heterogeneous pedestrian retrieval between sketch images and ordinary images plays an important role in public safety and criminal investigation, and the purpose of Heterogeneous Pedestrian Retrieval (HPR) is to retrieve images of the same person from heterogeneous image sets for identification. Although pedestrian retrieval plays an important role in public safety and criminal investigation, research is not yet abundant, and no data set for pedestrian retrieval problem (SINPR) between a sketch image and a general image exists in the field of pedestrian recognition so far. It is therefore necessary to acquire pedestrian data Sets (SINPID) of a sketch image and a normal image.
Most of the existing pedestrian re-identification problems are mainly focused on the matching problem in a normal scene, and the problem is solved to a certain extent.
However, directly applying existing pedestrian re-identification methods to SINPR would limit their performance. The half-coupling matrix is an effective technology suitable for different data source applications, and can link the relationship between different data sources. Dictionary learning is an effective computer vision application and object representation technique. Meanwhile, the invention can learn a projection matrix to reduce the difference between heterogeneous samples and learn dictionary pairs for different data sources.
In summary, the problems of the prior art are as follows:
(1) directly applying existing pedestrian re-identification methods to SINPR would limit their performance. The existing pedestrian reconstruction method cannot be directly used because the previous method is based on a heterogeneous pedestrian retrieval method, and the purpose of the previous method is to retrieve images of the same person from a heterogeneous image set for recognition. And pedestrian retrieval between the sketch image and the ordinary image, which cannot be analyzed.
(2) In the prior art, a half-coupling mapping strategy is not adopted, the relationship between a sketch image and a common photo is connected, and the difference between heterogeneous samples is reduced.
(3) No combination of learning metric matrices can reveal intrinsic projections of heterogeneous data.
(4) For sketch images and common photos in complex scenes, the dictionary pair in the prior art does not have good applicability.
(5) The prior art does not combine discrimination constraint to make the same category compact and different categories separate, which is not beneficial to retrieval and classification.
To date, there is no data set for the pedestrian retrieval problem (SINPR) between a sketch image and a normal image in the field of pedestrian recognition. It is therefore necessary to acquire pedestrian data Sets (SINPID) of a sketch image and a normal image.
The difficulty of solving the technical problems is as follows:
according to the method, a data set of the sketch image and the common photo needs to be established, and in the prior art, a half-coupling mapping strategy is not adopted, so that the relationship between the sketch image and the common photo is connected, and the difference between heterogeneous samples is reduced.
A better dictionary pair needs to be established to satisfy the relationship between the sketch picture and the common picture in the complex scene.
For intrinsic projection of heterogeneous data, a learning metric matrix needs to be incorporated.
In order to make the retrieval and classification more effective, it is necessary to make the same category compact and separate different categories.
The significance of solving the technical problems is as follows:
in the technical field of pedestrian traffic image processing, heterogeneous pedestrian retrieval between a sketch image and a common image plays an important role in public safety and criminal investigation, and the purpose of Heterogeneous Pedestrian Retrieval (HPR) is to retrieve images of the same person from heterogeneous image sets for identification. Although pedestrian retrieval plays an important role in public safety and criminal investigation, research is not yet abundant, and no data set for pedestrian retrieval problem (SINPR) between a sketch image and a general image exists in the field of pedestrian recognition so far. It is therefore necessary to acquire pedestrian data Sets (SINPID) of a sketch image and a normal image.
However, directly applying existing pedestrian re-identification methods to SINPR would limit their performance. The half-coupling matrix is an effective technology suitable for different data source applications, and can link the relationship between different data sources. Dictionary learning is an effective computer vision application and object representation technique. Meanwhile, the invention can learn a projection matrix to reduce the difference between heterogeneous samples and learn dictionary pairs for different data sources.
Disclosure of Invention
Aiming at the problems in the prior art, the invention provides a pedestrian retrieval method based on sketch image semi-coupling metric identification dictionary learning. Currently, there is no data set for the pedestrian retrieval problem (SINPR) between a sketch image and a normal image in the field of pedestrian recognition. It is therefore necessary to acquire pedestrian data Sets (SINPID) of a sketch image and a normal image. The half-coupling matrix is an effective technology suitable for different data source applications, and can link the relationship between different data sources. Dictionary learning is an effective computer vision application and object representation technique. Meanwhile, the invention can learn a projection matrix to reduce the difference between heterogeneous samples and learn dictionary pairs for different data sources.
The invention is realized in such a way that a pedestrian retrieval method (SMD) based on the learning of the half-coupling metric identification dictionary of sketch images2L), comprising the following steps:
step one, shooting a photo by using a camera and establishing a database of the photo;
step two, feature extraction;
step three, learning a heterogeneous dictionary pair D by utilizing a dictionary learning technologySAnd DN；
Step four, learning a projection matrix P from the sketch image set and the common image set;
fifthly, learning a metric matrix W by using the idea of discriminant learning;
solving parameters;
and seventhly, re-identifying the pedestrians.
Further, in the first step, a camera is used for taking pictures and establishing a database of the camera, and the specific method is as follows:
data was first collected on a campus using 2 cameras to take pictures of a real scene, for a total of 400 pictures of 200 pedestrians. Then the data of one camera is used as a common image set, and the data of the other camera is processed to form a sketch image set. Therefore, the pedestrian data Set (SINPID) collected by the invention is composed of 2 parts, namely 1) a common image set; 2) a set of sketch images. Half of samples of sketch images and common images are randomly selected as training sets, and the rest samples are used as testing sets. And the invention performs segmentation processing on the image set, the common image is 4160 × 3120pixels, the invention manually segments each person from the original photograph, and finally each pedestrian image is 560 × 230 pixels. Wherein the common image set is a common RGB image shot by a camera, and the common image set is used as a gallery data set; the sketch image set is a water powder sketch image generated through computer software and manual assistance instead of generating sketch style images through artists and witnesses, and the group of sketch image sets are used as probe sets. Where the software that generates the Sketch set may use Sketch Guru, the type of Sketch image may be selected by the present invention. Because the witness cannot provide suspicious court information, but the witness can give a rough and comprehensive description, the gouache sketch image ignores some details of the person to some extent but the color and layout of the clothing is the same, i.e. there is little change in the colors of the coat and pants, which helps to identify a person from the gallery dataset. The SINPID dataset is available from https:// sites.
Further, in the second step, feature extraction, the invention extracts two types of features from the collected pedestrian data Set (SINPID) for evaluation, wherein the two types of features include a LOMO patch feature and a PCB depth feature. Because the LOMO can extract HSV color features and SILTP texture features from the image represented by the three-scale pyramid, the features are extracted by the patch, and have certain robustness to the change of different viewpoints; and the PCB may extract a deep convolution descriptor consisting of a number of part-level features that may take the entire image as input and obtain a feature vector for each image.
Further, in the third step, the dictionary learning technology is utilized to learn the heterogeneous dictionary pair DSAnd DN. The dictionary learning technique is introduced to obtain good data representation of images of different styles, so that dictionary pairs are learned separately for heterogeneous data. Let X be ═ X1，x2...xN]And Y ═ Y1，y2...yN]Respectively a sketch image set and a normal image set. X is formed by Rd*N,Y∈Rd* ND is the dimension of the image feature and N represents the total number of sample images. DSAnd DNRespectively represent elementDescribing dictionary pairs of images and common images, therefore learning an objective function of a heterogeneous dictionary pair using dictionary learning techniques can be defined as:
wherein the dictionary pair DSAnd DNRespectively as follows: dS∈Rd*m，DN∈Rd*mAnd m represents the number of elements in the dictionary pair. A ═ a1,a2...aN],B＝[b1,b2...bN]A denotes X in DSB represents Y in DNThe coding coefficient matrix of (2).
Further, in step four, the projection matrix P is learned from the sketch image and the normal image set. The method aims to establish the relationship between a sketch image set and a common image set and search a half-coupling projection matrix to reduce the difference between the sketch image set and the common image set to a certain extent. Suppose P ∈ Rd*dIs a half-coupled projection matrix. By minimizing the distance between the a and B coding coefficient matrices, a half-coupled projection matrix is obtained, thereby reducing the difference between samples. The half-coupling mapping matrix can thus be calculated as follows:
further, in step five, the metric matrix W is learned using the concept of the discriminative learning. The method aims to improve good feature representation capability and can make samples of the same class compact and samples of different classes separate by adding discrimination constraint terms. Metric learning and dictionary learning using discriminative ideas can be computed as follows:
where S denotes that the (i, j) elements belong to the same class, and D denotes that the (i, j) elements belong to different classes.
M＝WTW W∈Rd*d (5)
Furthermore, to prevent overfitting, the present invention regularizes the following: a parameter projection matrix P, coefficients A, B and a discrimination constraint term W. The regularization term can be expressed as:
combine the above equations (1), (2), (3), (6) to produce the SMD2The objective function of L can be rewritten as:
where λ is the regularization parameter balance factor.
Further, in step six, the parameters are solved. Equation (7) for DS,DNNeither P, W is a joint convex function, but if the other variables are fixed, it is a convex function solution for each variable. Therefore, the invention divides the formula (7) into 4 subproblem solutions, namely dictionary pair updating, representing coefficient updating and updating of the half-coupling projection matrix. The method specifically comprises the following steps:
1) first the invention fixes other parameters to update a and B. Updating a preserves the term where only a exists, and equation (7) can be rewritten as:
to solve equation (8), the present invention mayBy mixing alphaiIs set to 0 to solve. Alpha is alphaiCan be used for
Expressed as follows:
αi＝(DS TDS+PTP+λI+(1-β)WTW)-1
b is similar to the solution of a in that,
bi＝(DN TDN+(λ-1)I+(1-β)WTW)-1
2) update DSAnd DNUpdate DSEquation (7) can be rewritten as follows:
to solve equation (11), DSCan be obtained by the formula DS＝XAT(AAT+∧))-1(12) (ii) a Λ is a diagonal matrix, similar to equation (11), updating DNEquation (7) can be rewritten as follows:
3) updating P, fixing other parameters, the invention can rewrite the formula (7) as:
solving equation (14), which can be solved by setting the derivative of P to 0, P is solved as follows:
P＝BAT(AAT+λI))-1 (15)；
4) finally, W is updated, and other parameters are fixed, and equation (7) can be rewritten as follows:
the present invention may update W by a gradient descent algorithm,
where t denotes the number of iterations of the algorithm as t.
Further, in step seven, the pedestrian is re-identified. And randomly selecting half of sketch images and common images as training sets, and the rest of sketch images and common images as test sets. And the sketch image is used as a probe set, and the ordinary image is used as a gallery set. And giving a sketch image from the probe to search pedestrians. Inputting a sketch image set and a common image set, and solving parameters D according to the formulas (12), (15) and (18)S,DNP, W. Suppose F is a feature of the sketch image set, G is a feature of the general photo set,
the invention performs pedestrian retrieval in the following manner:
1) firstly, the formula (9) is used for solving the pixel image set dictionary DSSolving for D using the learned projection matrix PSThe corresponding coefficient matrix f is calculated as follows:
2) solving a common image set dictionary D using equation (10)NThen solve for DNThe corresponding coefficient matrix g is calculated as follows:
3) the coefficient matrix f corresponding to the sketch image set and the coefficient matrix g corresponding to the common image set are solved by the formula (19) and the formula (20), the sketch image retrieval method can retrieve the sketch image corresponding to the common image set, and can obtain the sketch image by calculating the distance between the two images, and the calculation method is as follows:
and (3) calculating a formula (21), solving the corresponding distance, and sequencing the distances, wherein the common picture with the minimum distance is the picture searched by using the sketch image.
Another object of the present invention is to provide a pedestrian retrieval control system based on half-coupling metric discriminative dictionary learning of sketch images.
The invention also aims to provide a traffic road pedestrian image retrieval terminal of the pedestrian retrieval method based on the half-coupling metric identification dictionary learning of the sketch image.
In summary, the advantages and positive effects of the invention are:
the invention solves the problem of lack of heterogeneous pedestrian data sets in the field of heterogeneous pedestrian retrieval and provides half-coupling metric discriminative dictionary learning (SMD) for the first time2L) technique. The technique can learn the half-coupling mapping matrix from the heterogeneous sample and dictionary pairs, reducing the difference between heterogeneous data to some extent. The ideal searching effect is achieved on the new SINPID data set.
In order to verify whether the algorithm has good superiority or not, a pedestrian retrieval algorithm based on sketch image half-coupling metric identification dictionary learning and 6 comparison algorithms of KISSME, XQDA, TDL and SLD are used2L, JDML were compared to a PCB. These 6 comparison algorithms include metric-based, dictionary-based learning, deep learning-based, pedestrian re-recognition-based algorithms. And finally, verifying the new data set SINPID as experimental data.
The evaluation index of retrieval and identification is accumulated matching characteristic curve CMC, CMC curve is a matching probability of top-k, each sample in the common image set is sequentially calculated to be a distance from the probe image set, then the samples are sequenced, rank is the value of top-k selected by the invention, the value of CMC matching rate is about 1, if the number of times of testing is more, the identification accuracy rate is better.
In order to verify the method SMD of the invention2L performance, experiments were performed on the new SINPID dataset, with the experimental results shown in table 1 and fig. 2.
TABLE 1 Top-r match ratio on SINPID dataset
Fig. 2 (a) shows the results of the experiment using the LOMO characteristics, and (b) shows the results of the experiment using the PCB characteristics. As can be seen from the experimental results in table 1: the algorithm of the present invention achieves a higher match rate in the comparison algorithm, e.g., SMD2Compared with a comparison algorithm XQDA with L having LOMO characteristics in a SINPID data set, the matching rate of Rank-1 is improved by 2.1% (-36.2% -34.1%). Meanwhile, the invention also utilizes the depth characteristics for evaluation, and shows that some results are lower than those of the LOMO characteristic method. The reason for this may be three-fold: 1) the SINPID data set had only 400 images of 200 people, which resulted in the PCB not having sufficient sample training. 2) Watercolor sketch images contain a smaller amount of information than ordinary photographs of another camera. 3) The network architecture is not suitable for heterogeneous pedestrian samples, but for two normal samples.
Through the above experiments, it can be seen that: method SMD of the invention2Most of the matching rate of L is better than other methods with LOMO and PCB depth features. There are mainly 3 aspects: 1) by adopting a half-coupling mapping strategy, the relationship between the sketch image and the common photo can be connected, and the difference between heterogeneous samples is reduced. 2) The learning metric matrix may reveal intrinsic projections of heterogeneous data. 3) The dictionary pair has better applicability to sketch images and common photos in complex scenes. 4) The discrimination constraint can make the same category compact and different categories separate, which is beneficial to retrieval and classificationAnd (5) performing tasks. From the above analysis, it can be seen that the new SINPID dataset is stable for different types of features, suitable for further evaluation of pedestrian retrieval, indicating SMD2The superiority of the L algorithm.
Drawings
Fig. 1 is a flowchart of a pedestrian retrieval method based on sketch image half-coupling metric identification dictionary learning according to an embodiment of the present invention.
Fig. 2 is a graph of the performance of various methods and features provided by embodiments of the present invention on a new SINPID data set.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is further described in detail with reference to the following embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
In the prior art, a half-coupling mapping strategy is not adopted, the relationship between a sketch image and a common photo is connected, and the difference between heterogeneous samples is reduced. No combination of learning metric matrices can reveal intrinsic projections of heterogeneous data. For sketch images and common photos in complex scenes, the dictionary pair in the prior art does not have good applicability. The prior art does not combine discrimination constraint to make the same category compact and different categories separate, which is not beneficial to retrieval and classification.
In order to solve the above technical problems, the present invention will be described in detail with reference to specific embodiments.
As shown in fig. 1, a pedestrian retrieval method based on sketch image half-coupling metric identification dictionary learning according to an embodiment of the present invention includes:
step one, taking a picture by using a camera and establishing a database of the camera: data was first collected on a campus using 2 cameras to take pictures of a real scene, for a total of 400 pictures of 200 pedestrians. Then the data of one camera is used as a common image set, and the data of the other camera is processed to form a sketch image set. Therefore, the pedestrian data Set (SINPID) collected by the invention is composed of 2 parts, namely 1) a common image set; 2) a set of sketch images. Half of samples of sketch images and common images are randomly selected as training sets, and the rest samples are used as testing sets. And the invention performs segmentation processing on the image set, the common image is 4160 × 3120pixels, the invention manually segments each person from the original photograph, and finally each pedestrian image is 560 × 230 pixels. Wherein the common image set is a common RGB image shot by a camera, and the common image set is used as a gallery data set; the sketch image set is a water powder sketch image generated through computer software and manual assistance instead of generating sketch style images through artists and witnesses, and the group of sketch image sets are used as probe sets. Where the software that generates the Sketch set may use Sketch Guru, the type of Sketch image may be selected by the present invention. Because the witness cannot provide suspicious court information, but the witness can give a rough and comprehensive description, the gouache sketch image ignores some details of the person to some extent but the color and layout of the clothing is the same, i.e. there is little change in the colors of the coat and pants, which helps to identify a person from the gallery dataset. The SINPID dataset is available from https:// sites.
And step two, feature extraction, namely extracting two types of features from the collected pedestrian data Set (SINPID) for evaluation, wherein the features comprise LOMO patch features and PCB depth features. Because the LOMO can extract HSV color features and SILTP texture features from the image represented by the three-scale pyramid, the features are extracted by the patch, and have certain robustness to the change of different viewpoints; and the PCB may extract a deep convolution descriptor consisting of a number of part-level features that may take the entire image as input and obtain a feature vector for each image.
Step three, learning a heterogeneous dictionary pair D by utilizing a dictionary learning technologySAnd DN. The dictionary learning technique is introduced to obtain good data representation of images of different styles, so that dictionary pairs are learned separately for heterogeneous data. Let X be ═ X1，x2...xN]And Y ═ Y1，y2...yN]Respectively a sketch image set and a normal image set. X is formed by Rd*N,Y∈Rd*ND is the dimension of the image feature and N represents the total number of sample images. DSAnd DNDictionary pairs representing a sketch image and a normal image, respectively, so learning the objective function of a heterogeneous dictionary pair using dictionary learning techniques can be defined as:
wherein the dictionary pair DSAnd DNRespectively as follows: dS∈Rd*m,DN∈Rd*mAnd m represents the number of elements in the dictionary pair. A ═ a1,a2...aN],B＝[b1,b2...bN]A denotes X in DSB represents Y in DNThe coding coefficient matrix of (2).
And step four, learning a projection matrix P from the sketch image and the common image set. The method aims to establish the relationship between a sketch image set and a common image set, find a half-coupling projection matrix,
reducing the difference between them to some extent. Suppose P ∈ Rd*dIs a half-coupled projection matrix. By minimizing the distance between the A and B coding coefficient matrixes, a half-coupling projection matrix is obtained, thereby reducing the number of samples
The difference between them. The half-coupling mapping matrix can thus be calculated as follows:
and step five, learning a metric matrix W by using the idea of the discriminant learning. The method aims to improve good feature representation capability and can make samples of the same class compact and samples of different classes separate by adding discrimination constraint terms. Metric learning and dictionary learning using discriminative ideas can be computed as follows:
where S denotes that the (i, j) elements belong to the same class, and D denotes that the (i, j) elements belong to different classes.
M＝WTW W∈Rd*d (5)；
Furthermore, to prevent overfitting, the present invention regularizes the following: a parameter projection matrix P, coefficients A, B and a discrimination constraint term W. The regularization term can be expressed as:
combine the above equations (1), (2), (3), (6) to produce the SMD2The objective function of L can be rewritten as:
where λ is the regularization parameter balance factor.
In step six, the parameters are solved. Equation (7) for DS,DNNeither P, W is a joint convex function, but if the other variables are fixed, it is a convex function solution for each variable. Therefore, the invention divides the formula (7) into 4 subproblem solutions, namely dictionary pair updating, representing coefficient updating and updating of the half-coupling projection matrix. The method specifically comprises the following steps:
1) first the invention fixes other parameters to update a and B. Updating a preserves the term where only a exists, and equation (7) can be rewritten as:
to solve equation (8), the present invention can be implemented by converting αiIs set to 0 to solve. Alpha is alphaiCan be expressed as follows:
αi＝(DS TDS+PTP+λI+(1-β)WTW)-1
b is similar to the solution of a in that,
bi＝(DN TDN+(λ-1)I+(1-β)WTW)-1
2) update DSAnd DNUpdate DSEquation (7) can be rewritten as follows:
to solve equation (11), DSCan be obtained by the formula DS＝XAT(AAT+∧))-1(12) (ii) a Λ is a diagonal matrix, similar to equation (11), updating DNEquation (7) can be rewritten as
The following were used:
3) updating P, fixing other parameters, the invention can rewrite the formula (7) as:
solving equation (14), which can be solved by setting the derivative of P to 0, P is solved as follows:
P＝BAT(AAT+λI))-1 (15)；
4) finally, W is updated, and other parameters are fixed, and equation (7) can be rewritten as follows:
the present invention may update W by a gradient descent algorithm,
where t denotes the number of iterations of the algorithm as t.
Further, in step seven, the pedestrian is re-identified. And randomly selecting half of sketch images and common images as training sets, and the rest of sketch images and common images as test sets. And the sketch image is used as a probe set, and the ordinary image is used as a gallery set. And giving a sketch image from the probe to search pedestrians. Inputting a sketch image set and a common image set, and solving parameters D according to the formulas (12), (15) and (18)S,DNP, W. Suppose F is a feature of the sketch image set, G is a feature of the general photo set,
the invention performs pedestrian retrieval in the following manner:
1) firstly, the formula (9) is used for solving the pixel image set dictionary DSSolving for D using the learned projection matrix PSThe corresponding coefficient matrix f is calculated as follows:
2) solving a common image set dictionary D using equation (10)NThen solve for DNThe corresponding coefficient matrix g is calculated as follows:
3) the coefficient matrix f corresponding to the sketch image set and the coefficient matrix g corresponding to the common image set are solved by the formula (19) and the formula (20), the sketch image retrieval method can retrieve the sketch image corresponding to the common image set, and can obtain the sketch image by calculating the distance between the two images, and the calculation method is as follows:
and (3) calculating a formula (21), solving the corresponding distance, and sequencing the distances, wherein the common picture with the minimum distance is the picture searched by using the sketch image.
The invention is further described below in connection with experiments.
In order to verify whether the algorithm has good superiority or not, a pedestrian retrieval algorithm based on sketch image half-coupling metric identification dictionary learning and 6 comparison algorithms of KISSME, XQDA, TDL and SLD are used2L, JDML were compared to a PCB. These 6 comparison algorithms include metric-based, dictionary-based learning, deep learning-based, pedestrian re-recognition-based algorithms. And finally, verifying the new data set SINPID as experimental data.
The evaluation index of retrieval and identification is accumulated matching characteristic curve CMC, CMC curve is a matching probability of top-k, each sample in the common image set is sequentially calculated to be a distance from the probe image set, then the samples are sequenced, rank is the value of top-k selected by the invention, the value of CMC matching rate is about 1, if the number of times of testing is more, the identification accuracy rate is better.
To verify the bookMethod of the invention SMD2L performance, experiments were performed on the new SINPID dataset, with the experimental results shown in table 1 and fig. 2.
TABLE 1 Top-r match ratio on SINPID dataset
Fig. 2 (a) shows the results of the experiment using the LOMO characteristics, and (b) shows the results of the experiment using the PCB characteristics. As can be seen from the experimental results in table 1: the algorithm of the present invention achieves a higher match rate in the comparison algorithm, e.g., SMD2Compared with a comparison algorithm XQDA with L having LOMO characteristics in a SINPID data set, the matching rate of Rank-1 is improved by 2.1% (-36.2% -34.1%). Meanwhile, the invention also utilizes the depth characteristics for evaluation, and shows that some results are lower than those of the LOMO characteristic method. The reason for this may be three-fold: 1) the SINPID data set had only 400 images of 200 people, which resulted in the PCB not having sufficient sample training. 2) Watercolor sketch images contain a smaller amount of information than ordinary photographs of another camera. 3) The network architecture is not suitable for heterogeneous pedestrian samples, but for two normal samples.
Through the above experiments, it can be seen that: method SMD of the invention2Most of the matching rate of L is better than other methods with LOMO and PCB depth features. There are mainly 3 aspects: 1) by adopting a half-coupling mapping strategy, the relationship between the sketch image and the common photo can be connected, and the difference between heterogeneous samples is reduced. 2) The learning metric matrix may reveal intrinsic projections of heterogeneous data. 3) The dictionary pair has better applicability to sketch images and common photos in complex scenes. 4) Discrimination constraints enable the same category to be compact and different categories to be separated, and retrieval and classification tasks are facilitated. From the above analysis, it can be seen that the new SINPID dataset is stable for different types of features, suitable for further evaluation of pedestrian retrieval, indicating SMD2L is calculatedThe superiority of the method.
The above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the invention, and any modifications, equivalents and improvements made within the spirit and principle of the present invention are intended to be included within the scope of the present invention.
Claims (5)
1. A pedestrian retrieval method based on sketch image semi-coupling metric identification dictionary learning is characterized by comprising the following steps:
establishing a heterogeneous pedestrian retrieval database SINPID comprising a sketch image set and a regular image set; extracting features, namely extracting the features from the images and marking the images with different corresponding colors;
carrying out heterogeneous data processing on the extracted sample characteristics; a dictionary learning method is introduced to learn a dictionary pair of heterogeneous data, and a mapping matrix is learned from a sketch image set and a regular image set;
and introducing an identification algorithm learning metric to carry out pedestrian retrieval based on the sketch image semi-coupling metric identification dictionary learning;
the pedestrian retrieval calculation method based on the sketch image semi-coupling metric identification dictionary learning comprises the following steps:
step one, shooting a photo by using a camera and establishing a database of the photo;
step two, feature extraction;
step three, learning a heterogeneous dictionary pair D by utilizing a dictionary learning technologySAnd DN；
Step four, learning a projection matrix P from the sketch image set and the common image set;
fifthly, learning a metric matrix W by using the idea of discriminant learning;
solving parameters;
seventhly, identifying the pedestrians again;
in step three, X ═ X1，x2...xN]And Y ═ Y1，y2...yN]Are sketches respectivelyAn image set and a common image set; x is formed by Rd*N,Y∈Rd*N(ii) a d is the dimension of the image feature, N represents the total number of sample images; dSAnd DNThe dictionary pair respectively represents the sketch image and the common image, and the target function of the heterogeneous dictionary pair is defined as follows by utilizing a dictionary learning technology to learn:
wherein the dictionary pair DSAnd DNRespectively as follows: dS∈Rd*m,DN∈Rd*mM denotes the number of elements in the dictionary pair; a ═ a1,a2...aN],B＝[b1,b2...bN]A represents X in DSB represents Y in DNA matrix of coding coefficients of (c);
in step four, P is equal to Rd*dIs a half-coupled projection matrix; obtaining a half-coupling projection matrix by minimizing the distance between the A and B coding coefficient matrixes; the half-coupling mapping matrix is calculated as follows:
in the fifth step, the metric learning and the dictionary learning are calculated by utilizing the identification thought according to the following modes:
wherein S indicates that the (i, j) elements belong to the same class, and D indicates that the (i, j) elements belong to different classes;
M＝WTW W∈Rd*d
(5)；
a parameter projection matrix P, coefficients A and B and an identification constraint term W; the regularization term is expressed as:
combining the above formulas (1), (2), (3), (6); then SMD2The objective function of L is rewritten as:
wherein λ is a regularization parameter balance factor;
in the sixth step, the dictionary pair divided by the formula (7) is updated, and the subproblems of representing coefficient updating and half-coupling projection matrix updating comprise:
1) fixing parameters of formula (8) except for parameters A and B and updating parameters A and B; updating and keeping the terms of A and the parameters A only exist, and rewriting the formula (7) as follows:
by mixing alphaiThe derivative of (a) is set to 0 for solution; alpha is alphaiExpressed as follows:
αi＝(DS TDS+PTP+λI+(1-β)WTW)-1
updating B the entry for which only parameter B exists and keeping it,
bi＝(DN TDN+(λ-1)I+(1-β)WTW)-1
2) update DSAnd DNUpdate DSEquation (7) is rewritten as follows:
DScan be obtained by the formula DS＝XAT(AAT+∧))-1 (12)；
Λ is a diagonal matrix, update DNRewrite equation (7) as follows:
3) updating P, fixing other parameters, and rewriting equation (7) as:
solving by setting the derivative of P to 0, P is solved as follows:
P＝BAT(AAT+λI))-1 (15)；
4) finally, W is updated, other parameters are fixed, and formula (7) is rewritten as follows:
w is updated by a gradient descent algorithm,
wherein t represents the iteration number of the algorithm as t;
in the seventh step, the sketch image set and the common image set are input, and the parameters D are solved according to the formulas (12), (15) and (18)S,DNP, W; f is the characteristic of the sketch image set, G is the characteristic of the common photo set, and the method specifically comprises the following steps:
solving a pixilated image set dictionary D using equation (9)SSolving for D using the learned projection matrix PSThe corresponding coefficient matrix f is calculated as follows:
solving a common image set dictionary D using equation (10)NThen solve for DNThe corresponding coefficient matrix g is calculated as follows:
the coefficient matrix f corresponding to the sketch image set and the coefficient matrix g corresponding to the common image set are solved through a formula (19) and a formula (20), the sketch image corresponding to the common image set is searched, and the distance between the two images is calculated to obtain the sketch image, wherein the calculation mode is as follows:
and (3) calculating a formula (21), solving the corresponding distance, and sequencing the distances, wherein the common picture with the minimum distance is the picture searched by using the sketch image.
2. The sketch image-based pedestrian retrieval method for identification dictionary learning by using half-coupling metrics as claimed in claim 1, wherein the step one specifically comprises:
shooting pictures in a real scene by using a plurality of cameras to collect data, and collecting data of a plurality of pictures in total; then, taking the data of one camera as a common image set, and processing the data of the other camera to form a sketch image set;
randomly selecting half samples of the sketch images and the common images as a training set, and using the rest samples as a test set; and performs a segmentation process on the image set.
3. The pedestrian retrieval method based on learning of the sketch image semi-coupling metric identification dictionary of claim 1, wherein in the second step, two types of features extracted from the collected pedestrian data set SINPID are evaluated, wherein the two types of features comprise LOMO patch features and PCB depth features; LOMO patch characteristics are extracted by patch; the PCB depth feature is a depth convolution descriptor composed of a plurality of part-level features, takes the whole image as input, and obtains a feature vector of each image.
4. A pedestrian retrieval control system for sketch image based semi-coupling metric identification dictionary learning implementing the pedestrian retrieval method for sketch image based semi-coupling metric identification dictionary learning according to claim 1.
5. A traffic road pedestrian image retrieval terminal implementing the pedestrian retrieval method based on the sketch image half-coupling metric identification dictionary learning of claim 1.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910325189.4A CN110046669B (en) | 2019-04-22 | 2019-04-22 | Pedestrian retrieval method based on sketch image half-coupling metric identification dictionary learning |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910325189.4A CN110046669B (en) | 2019-04-22 | 2019-04-22 | Pedestrian retrieval method based on sketch image half-coupling metric identification dictionary learning |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110046669A CN110046669A (en) | 2019-07-23 |
| CN110046669B true CN110046669B (en) | 2019-12-24 |
Family
ID=67278414
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910325189.4A Active CN110046669B (en) | 2019-04-22 | 2019-04-22 | Pedestrian retrieval method based on sketch image half-coupling metric identification dictionary learning |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110046669B (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113240715A (en) * | 2021-05-29 | 2021-08-10 | 刘文平 | Multiple feature-driven similarity measurement learning figure retrieval method |
| CN113553947B (en) * | 2021-07-23 | 2022-05-17 | 湖南大学 | Method and device for generating and describing multi-mode pedestrian re-recognition and electronic equipment |
| CN116052095B (en) * | 2023-03-31 | 2023-06-16 | 松立控股集团股份有限公司 | Vehicle re-identification method for smart city panoramic video monitoring |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7567715B1 (en) * | 2004-05-12 | 2009-07-28 | The Regents Of The University Of California | System and method for representing and encoding images |
| CN108027836A (en) * | 2015-09-24 | 2018-05-11 | 高通股份有限公司 | With the system and method for the query image access images of capture |
| CN108932518A (en) * | 2018-06-22 | 2018-12-04 | 大连理工大学 | A kind of feature extraction of shoes watermark image and search method of view-based access control model bag of words |
| CN109284668A (en) * | 2018-07-27 | 2019-01-29 | 昆明理工大学 | A kind of pedestrian's weight recognizer based on apart from regularization projection and dictionary learning |
Family Cites Families (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20120162244A1 (en) * | 2010-12-28 | 2012-06-28 | Microsoft Corporation | Image search color sketch filtering |
| CN104239359B (en) * | 2013-06-24 | 2017-09-01 | 富士通株式会社 | Based on multi-modal image labeling device and method |
| CN105550649B (en) * | 2015-12-09 | 2019-03-08 | 武汉工程大学 | Extremely low resolution ratio face identification method and system based on unity couping local constraint representation |
| JP6851193B2 (en) * | 2016-12-20 | 2021-03-31 | 東芝テック株式会社 | Information processing equipment and programs |
| CN106919919B (en) * | 2017-02-28 | 2019-08-06 | 西安电子科技大学 | A kind of SAR target discrimination method based on multiple features fusion word packet model |
| CN108647690B (en) * | 2017-10-17 | 2021-12-31 | 南京工程学院 | Non-constrained face image dimension reduction method based on discrimination sparse preserving projection |
| CN109117860B (en) * | 2018-06-27 | 2022-10-25 | 南京邮电大学 | Image classification method based on subspace projection and dictionary learning |
| CN109064403B (en) * | 2018-08-10 | 2022-11-01 | 安徽师范大学 | Fingerprint image super-resolution method based on classification coupling dictionary sparse representation |
| CN109409201B (en) * | 2018-09-05 | 2021-06-18 | 昆明理工大学 | Pedestrian re-recognition method based on shared and special dictionary pair joint learning |
-
2019
- 2019-04-22 CN CN201910325189.4A patent/CN110046669B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7567715B1 (en) * | 2004-05-12 | 2009-07-28 | The Regents Of The University Of California | System and method for representing and encoding images |
| CN108027836A (en) * | 2015-09-24 | 2018-05-11 | 高通股份有限公司 | With the system and method for the query image access images of capture |
| CN108932518A (en) * | 2018-06-22 | 2018-12-04 | 大连理工大学 | A kind of feature extraction of shoes watermark image and search method of view-based access control model bag of words |
| CN109284668A (en) * | 2018-07-27 | 2019-01-29 | 昆明理工大学 | A kind of pedestrian's weight recognizer based on apart from regularization projection and dictionary learning |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110046669A (en) | 2019-07-23 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108229468B (en) | Vehicle appearance feature recognition and vehicle retrieval method and device, storage medium and electronic equipment | |
| Laffont et al. | Transient attributes for high-level understanding and editing of outdoor scenes | |
| CN106778604B (en) | Pedestrian re-identification method based on matching convolutional neural network | |
| CN113065558A (en) | Lightweight small target detection method combined with attention mechanism | |
| CN108520226B (en) | Pedestrian re-identification method based on body decomposition and significance detection | |
| CN109543602B (en) | Pedestrian re-identification method based on multi-view image feature decomposition | |
| CN110909605B (en) | Cross-modal pedestrian re-identification method based on contrast correlation | |
| CN110046669B (en) | Pedestrian retrieval method based on sketch image half-coupling metric identification dictionary learning | |
| Zhang et al. | Detecting and extracting the photo composites using planar homography and graph cut | |
| CN109961051A (en) | A kind of pedestrian's recognition methods again extracted based on cluster and blocking characteristic | |
| CN108280411A (en) | A kind of pedestrian's searching method with spatial alternation ability | |
| CN111738048B (en) | Pedestrian re-identification method | |
| CN107977948B (en) | Salient map fusion method facing community image | |
| CN113095371B (en) | Feature point matching method and system for three-dimensional reconstruction | |
| CN111046789A (en) | Pedestrian re-identification method | |
| Liao et al. | Unsupervised foggy scene understanding via self spatial-temporal label diffusion | |
| CN114239754B (en) | Pedestrian attribute identification method and system based on attribute feature learning decoupling | |
| Zhou et al. | Attention transfer network for nature image matting | |
| Kınlı et al. | Modeling the lighting in scenes as style for auto white-balance correction | |
| Di Martino et al. | Rethinking shape from shading for spoofing detection | |
| CN115620049A (en) | Method for detecting disguised target based on polarized image clues and application thereof | |
| CN112417961B (en) | Sea surface target detection method based on scene prior knowledge | |
| Oh et al. | Visual adversarial attacks and defenses | |
| CN111091492B (en) | Face image illumination migration method based on convolutional neural network | |
| CN114445916A (en) | Living body detection method, terminal device and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |