CN110046669A - Half Coupling Metric based on sketch image identifies the pedestrian retrieval method of dictionary learning - Google Patents
Half Coupling Metric based on sketch image identifies the pedestrian retrieval method of dictionary learning Download PDFInfo
- Publication number
- CN110046669A CN110046669A CN201910325189.4A CN201910325189A CN110046669A CN 110046669 A CN110046669 A CN 110046669A CN 201910325189 A CN201910325189 A CN 201910325189A CN 110046669 A CN110046669 A CN 110046669A
- Authority
- CN
- China
- Prior art keywords
- image
- sketch
- dictionary
- pedestrian
- half coupling
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
- G06F16/58—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
- G06F16/583—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/28—Determining representative reference patterns, e.g. by averaging or distorting; Generating dictionaries
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
Abstract
The invention belongs to traffic pedestrian image processing technology fields, disclose the pedestrian retrieval method and system of a kind of half Coupling Metric identification dictionary learning based on sketch image, establish the isomery pedestrian retrieval database of oneself, then feature extraction is carried out, feature is extracted from image and marks different people with corresponding color；The sample characteristics extracted are handled, dictionary learning method is introduced, learns the dictionary pair of isomeric data；From sketch map image set and regular image focusing study mapping matrix；Introduce identification algorithm study measurement.The advantage of the invention is that solving the problems, such as that isomery pedestrian's data set lacks and be put forward for the first time half Coupling Metric identification dictionary learning (SMD in isomery pedestrian retrieval field2L) technology.The technology can learn half coupled maps matrix from isomery sample and dictionary centering, reduce the difference between isomeric data to a certain extent.Ideal retrieval effectiveness is achieved on new SINPID data set.
Description
Technical field
The invention belongs to traffic pedestrian image processing technology field more particularly to a kind of half degrees of coupling based on sketch image
Amount identifies the pedestrian retrieval method of dictionary learning.
Background technique
Currently, the immediate prior art:
Isomery pedestrian retrieval between sketch image and normal image plays in terms of public safety and criminal investigation focuses on
It acts on, the purpose of isomery pedestrian retrieval (HPR) is exactly to retrieve the image of the same person from foreign peoples's image set to identify.
Although pedestrian retrieval plays a very important role in public safety and criminal investigation, it is still rare to study, up to the present,
Pedestrian identifies that field has not been used in the data set of the pedestrian retrieval problem (SINPR) between sketch map image and normal image.
It is therefore desirable to acquire pedestrian's data set (SINPID) of sketch image and normal image.
Current most of pedestrians identify the matching problem that problem is laid particular emphasis on mostly under normal scene again, and in certain journey
It is solved this problem in that on degree.
However, existing pedestrian again recognition methods is directly applied to the performance that SINPR will will limit them.Half coupling moment
Battle array is a kind of effective technology suitable for different data sources application, it can get up the relationship between different data sources.
And dictionary learning is that a kind of effective computer vision is applied and object presentation technology.The present invention can learn a projection simultaneously
Matrix to reduce the difference between isomery sample, and learns dictionary pair to different data sources.
In conclusion problem of the existing technology is:
(1) existing pedestrian again recognition methods is directly applied to the performance that SINPR will will limit them.It cannot directly use
Existing pedestrian's weighing method, because of method of the pervious method based on isomery pedestrian retrieval, its purpose is from foreign peoples's image set
In retrieve the image of the same person and identify.And the pedestrian retrieval between unanalyzable sketch map image and normal image
Problem.
(2) in the prior art, half coupled maps strategy is not used, the relationship of sketch image and common photo is connected
Come, reduces the difference between isomery sample.
(3) being not bound with study metric matrix can reveal that the inherent projection of isomeric data.
(4) for the sketch image and common photo under complex scene, prior art dictionary is preferable applicable to not having
Property.
(5) prior art be not bound with differentiation constraint keep same category compact and different classes of separation, be unfavorable for retrieval with
Classification.
Up to the present, identify that field has not been used in the pedestrian retrieval between sketch map image and normal image in pedestrian
The data set of problem (SINPR).It is therefore desirable to acquire pedestrian's data set (SINPID) of sketch image and normal image.
Solve the difficulty of above-mentioned technical problem:
The present invention needs to establish the data set of vegetarian noodles image and common photo, in the prior art, is not reflected using half coupling
Strategy is penetrated, the relationship of sketch image and common photo is connected, reduces the difference between isomery sample.
Need to establish preferable dictionary pair, to meet the relationship of the sketch picture under complex scene and common photo.
Inherence projection for isomeric data, needs associative learning metric matrix.
In order to enable the effect of retrieval and classification is more preferable, need to keep same category compact and different classes of separation.
Solve the meaning of above-mentioned technical problem:
In traffic pedestrian image processing technology field, the isomery pedestrian retrieval between sketch image and normal image is public
It is played an important role in terms of safety and criminal investigation, the purpose of isomery pedestrian retrieval (HPR) is exactly to examine from foreign peoples's image set
The image that rope goes out the same person identifies.Although pedestrian retrieval plays a very important role in public safety and criminal investigation,
Be study it is still rare, up to the present, pedestrian identify field have not been used between sketch map image and normal image
The data set of pedestrian retrieval problem (SINPR).It is therefore desirable to acquire pedestrian's data set of sketch image and normal image
(SINPID)。
However, existing pedestrian again recognition methods is directly applied to the performance that SINPR will will limit them.Half coupling moment
Battle array is a kind of effective technology suitable for different data sources application, it can get up the relationship between different data sources.
And dictionary learning is that a kind of effective computer vision is applied and object presentation technology.The present invention can learn a projection simultaneously
Matrix to reduce the difference between isomery sample, and learns dictionary pair to different data sources.
Summary of the invention
In view of the problems of the existing technology, the present invention provides a kind of, and half Coupling Metric based on sketch image identifies word
The pedestrian retrieval method of allusion quotation study.Currently, identifying that field has not been used between sketch map image and normal image in pedestrian
The data set of pedestrian retrieval problem (SINPR).It is therefore desirable to acquire pedestrian's data set of sketch image and normal image
(SINPID).Half coupling matrix be it is a kind of suitable for different data sources application effective technology, it can by different data sources it
Between relationship get up.And dictionary learning is that a kind of effective computer vision is applied and object presentation technology.This hair simultaneously
It is bright to learn a projection matrix, to reduce the difference between isomery sample, and dictionary pair is learnt to different data sources.
The invention is realized in this way a kind of half Coupling Metric based on sketch image identifies the pedestrian retrieval of dictionary learning
Method (SMD2L), comprising the following steps:
Step 1: shooting photo with camera and establishing the database of oneself；
Step 2: feature extraction；
Step 3: using dictionary learning technological learning isomery dictionary to DSAnd DN；
Step 4: learning projection matrix P from sketch map image set and normal image collection；
Step 5: learning metric matrix W using the thought for identifying study；
Step 6: parametric solution；
Step 7: pedestrian identifies again.
Further, in step 1, photo is shot with camera and establishes the database of oneself, specific practice is:
The photo under real scene is shot to collect data with 2 cameras first in campus, has collected 200 in total
A pedestrian totally 400 picture datas.Then using the data of one of camera as normal image collection, another camera
Data by processing formed sketch map image set.Therefore pedestrian's data set (SINPID) that the present invention is collected into is by 2 parts
Form 1) normal image collection；2) sketch map image set.The present invention has randomly selected the sketch image of half and the sample of normal image
As training set, remaining is as test set.And the present invention is split processing to image set, and normal image is 4160*
3120pixels, the present invention are partitioned into everyone from original photo by hand, and last each pedestrian image is 560*
230pixels.Wherein normal image collection is the common RGB image shot with camera, using this group of normal image collection as gallery
Data set；Sketch map image set be pigment sketch collection image generated by computer software and human assistance rather than by artist and
Witness generates the image of sketch style, using this group of sketch map image set as probe collection.Here the software for generating sketch collection can be with
Using Sketch Guru, the type present invention for generating sketch image can choose pigment sketch type.Because witness cannot provide
Suspicious appears in court information, but witness can provide rough and comprehensive description, and the image of pigment sketch is ignored to a certain extent
Some details of people but the color of clothes and layout are all that the color of identical i.e. housing and trousers has almost no change, this
Facilitate from gallery data set to identify a people.SINPID data set can be from https: //sites.google.com/
Site/SINPID2018 is obtained.
Further, in step 2, feature extraction, the present invention extracts collected pedestrian's data set (SINPID)
Two kinds of feature is evaluated out, including LOMO patch feature and PCB depth characteristic.Because LOMO can be from three rulers
It spends in the image of pyramid representation and extracts hsv color feature and SILTP textural characteristics, this feature is extracted by patch
, there is certain robustness to the variation of different points of view；And PCB can extract a kind of depth being made of multiple portions grade feature
Convolution descriptor is spent, this feature can be using whole image as input, and obtains the feature vector of each image.
Further, in step 3, using dictionary learning technological learning isomery dictionary to DSAnd DN.Introduce dictionary learning
The purpose of technology is that the good data of different-style image in order to obtain indicates, therefore learns dictionary pair respectively to isomeric data.
Assuming that X=[x1, x2...xN] and Y=[y1, y2...yN] it is sketch map image set and normal image collection respectively.X∈Rd*N,Y∈Rd* NIt .d is the dimension of characteristics of image, N indicates the total quantity of sample image. DSAnd DNRespectively indicate sketch image and normal image
Dictionary pair, therefore can be with using the objective function of dictionary learning technological learning isomery dictionary pair is defined as:
Wherein dictionary is to DSAnd DNIt is respectively as follows: DS∈Rd*m, DN∈Rd*m, what m was indicated is the quantity of the element of dictionary centering.A
=[a1,a2...aN], B=[b1,b2...bN] .A indicate be X in DSOn code coefficient matrix, B indicate be Y in DNOn
Code coefficient matrix.
Further, in step 4, from sketch image and normal image focusing study projection matrix P.Its purpose be for
Sketch map image set and normal image collection relationship between the two are established, one and half coupling projection matrixes are found, to a certain degree
The upper difference reduced between them.Assuming that P ∈ Rd*dIt is half coupling projection matrix.By minimizing A and B code coefficient matrix
Distance obtains half coupling projection matrix, to reduce the difference between sample.Therefore half coupled maps matrix can be by with lower section
Formula calculates:
Further, in step 5, learn metric matrix W using the thought for identifying study.Its purpose is to improve
Good character representation ability can make of a sort sample compact in addition identifying bound term, inhomogeneous sample separation.
It can be calculated in the following manner using thought progress metric learning and dictionary learning is identified:
What wherein S was indicated is that (i, j) element belongs to same class, and what D was indicated is that (i, j) element belongs to inhomogeneity.
M=WTW W∈Rd*d (5)
In addition, over-fitting in order to prevent, below regularization of the present invention: parameter projection matrix P, coefficient A, B and identifying about
Beam item W.Regularization term can indicate are as follows:
In conjunction with above-mentioned formula (1), (2), (3), (6) then SMD2The objective function of L can be rewritten are as follows:
Wherein λ is the regularization parameter balance factor.
Further, in step 6, parametric solution.Formula (7) is for DS,DN, P, W are not joint convex functions, but such as
It is that convex function solves for each variable in the case that its dependent variable of fruit is fixed.Therefore formula (7) is divided into 4 by the present invention
A sub- problem solving, i.e. dictionary indicate the update of coefficient update and half coupling projection matrix to update.It specifically includes:
1) present invention fixes other parameters to update A and B first.It updates A to retain the item for only existing A, formula (7)
It can rewrite are as follows:
For solution formula (8), the present invention can be by by αiDerivative be set as 0 to solve.αiIt can
It is expressed as:
αi=(DS TDS+PTP+λI+(1-β)WTW)-1
B is similar with the solution of A,
bi=(DN TDN+(λ-1)I+(1-β)WTW)-1
2) D is updatedSAnd DN, update DSFormula (7) can be rewritten as follows:
For solution formula (11), DSCalculating can be obtained by following formula: DS=XAT(AAT+∧))-1(12)；
∧ is a diagonal matrix, similar to formula (11), updates DNFormula (7) can be rewritten as follows:
3) P, fixed other parameters are updated, the present invention can rewrite formula (7) are as follows:
Solution formula (14) can be solved by setting 0 for the derivative of P, and the solution of P is as follows:
P=BAT(AAT+λI))-1(15)；
4) final updating W, fixed other parameters, formula (7) can be rewritten as follows:
The present invention can update W by gradient descent algorithm,
Wherein t indicate be algorithm the number of iterations be t.
Further, in step 7, pedestrian identifies again.The sketch image and normal image for having randomly selected half are as instruction
Practice collection, remaining is as test set.And using sketch image as probe collection, normal image is as gallery collection.Provide a width
From the sketch image of probe, pedestrian retrieval is carried out.Sketch map image set and normal image collection are inputted, according to above-mentioned formula (12),
(15), (18) solve parameter DS,DN,P,W.Vacation lets f be the feature of sketch map image set, and G is the feature of common photo collection,
The present invention executes pedestrian retrieval in the following way:
1) sketch map image set dictionary D is solved first with formula (9)S, using the projection matrix P learnt, to solve DSIt is right
The coefficient matrix f answered, calculation are as follows:
2) normal image collection dictionary D is solved using formula (10)N, then solve DNCorresponding coefficient matrix g, calculation
It is as follows:
3) coefficient matrix f and normal image collection corresponding to sketch map image set are solved by formula (19) and formula (20)
Corresponding coefficient matrix g, the present invention, which can be retrieved, concentrates corresponding sketch image in normal image, can be by calculating two
The distance between a image obtains, and calculation is as follows:
Calculation formula (21) solves corresponding distance, then adjusts the distance and be ranked up, wherein apart from the smallest common photograph
Piece is exactly the picture being retrieved using sketch image.
Another object of the present invention is to provide a kind of, and half Coupling Metric based on sketch image identifies the row of dictionary learning
People retrieves control system.
Another object of the present invention is to provide a kind of, and half Coupling Metric based on sketch image identifies the row of dictionary learning
The traffic route pedestrian image searching terminal of people's search method.
In conclusion advantages of the present invention and good effect are as follows:
The present invention solves the problems, such as that isomery pedestrian's data set lacks and is put forward for the first time in isomery pedestrian retrieval field
Half Coupling Metric identifies dictionary learning (SMD2L) technology.The technology can be reflected from isomery sample and half coupling of dictionary centering study
Matrix is penetrated, reduces the difference between isomeric data to a certain extent.Ideal inspection is achieved on new SINPID data set
Suo Xiaoguo.
To verify whether this algorithm has good superiority, half Coupling Metric based on sketch image is identified into dictionary learning
Pedestrian retrieval algorithm and 6 kinds of comparison algorithms KISSME, XQDA, TDL, SLD2L, JDML and PCB are compared.This 6 kinds comparisons
Algorithm includes being based on dictionary learning based on measurement, is based on deep learning, the algorithm identified again based on pedestrian.Finally in the present invention
New data set SINPID verified as experimental data.
The evaluation index of retrieval is the matching probability that accumulative matching properties curve CMC, CMC curve is a kind of top-k,
Each sample that normal image is concentrated successively is taken into a distance with the calculation of its probe image collection, is then ranked up, rank is
The value of the value of top-k selected by the present invention, CMC matching rate tends to 1, if test number it is more, identification it is accurate
Rate is better.
In order to verify the method for the present invention SMD2The performance of L is tested in new SINPID data set, and experimental result is table 1
And Fig. 2.
The matching rate of top-r of the table 1 on SINPID data set
What (a) was indicated in Fig. 2 be using the experimental result of LOMO feature, (b) in indicate is reality using PCB feature
Test result.From the experimental result in table 1 it can be seen that algorithm of the invention obtains higher matching rate in comparison algorithm,
For example, SMD2L has the comparison algorithm XQDA of LOMO feature to compare in SINPID data set, and the matching rate of Rank-1 improves
2.1% (=36.2%-34.1%).The present invention also utilizes depth characteristic and assesses simultaneously, it will be seen that some results is lower than
The method of LOMO feature.Its reason may have three aspects: 1) SINPID data set only has the image of 400 200 people, this makes
PCB does not pass through enough sample trainings.2) watercolor sketch formula image is than information that the common photo of another video camera includes
Amount is few.3) network architecture is not suitable for isomery pedestrian sample, and is suitable for two kinds of normal samples.
By being tested above it can be seen that the method for the present invention SMD2Most of matching rates of L are better than other with LOMO
The method of feature and PCB depth characteristic.Mainly have 3 aspects: 1) using half coupled maps strategy, can by sketch image with it is general
The relationship of logical photo connects, and reduces the difference between isomery sample.2) study metric matrix can reveal that isomeric data
Inherence projection.3) for the sketch image and common photo under complex scene, dictionary is to preferable applicability.4) differentiate about
Beam can make the compact and different classes of separation of same category, be conducive to retrieval and classification task.As seen from the above analysis, newly
SINPID data set be to different types of feature it is stable, suitable for the further evaluation of pedestrian retrieval, show SMD2L is calculated
The superiority of method.
Detailed description of the invention
Fig. 1 is the pedestrian retrieval that half Coupling Metric provided in an embodiment of the present invention based on sketch image identifies dictionary learning
Method flow diagram.
Fig. 2 is the performance map of distinct methods provided in an embodiment of the present invention and feature on new SINPID data set.
Specific embodiment
In order to make the objectives, technical solutions, and advantages of the present invention clearer, with reference to embodiments, to the present invention
It is further elaborated.It should be appreciated that the specific embodiments described herein are merely illustrative of the present invention, it is not used to
Limit the present invention.
In the prior art, half coupled maps strategy is not used, the relationship of sketch image and common photo is connected,
Reduce the difference between isomery sample.Being not bound with study metric matrix can reveal that the inherent projection of isomeric data.For multiple
Sketch image and common photo under miscellaneous scene, prior art dictionary is to without preferable applicability.The prior art is not tied
It closes and differentiates that constraint keeps same category compact and different classes of separation, be unfavorable for retrieving and classify.
In order to solve the above technical problems, below with reference to concrete scheme, the present invention is described in detail.
As shown in Figure 1, half Coupling Metric provided in an embodiment of the present invention based on sketch image identifies the row of dictionary learning
People's search method includes:
Step 1: shooting photo with camera and establishing the database of oneself: true with 2 camera shootings first in campus
Photo under real field scape collects data, has collected 200 pedestrians totally 400 picture datas in total.Then one of them is taken the photograph
As the data of head are as normal image collection, the data of another camera form sketch map image set by processing.Therefore this hair
The bright pedestrian's data set (SINPID) being collected into forms 1 by 2 parts) normal image collection；2) sketch map image set.The present invention with
Machine has chosen the sketch image of half and the sample of normal image as training set, remaining is as test set.And the present invention
Processing is split to image set, normal image is 4160*3120pixels, and the present invention is partitioned into often by hand from original photo
One people, last each pedestrian image is 560*230pixels.Wherein normal image collection is the common RGB shot with camera
Image, using this group of normal image collection as gallery data set；Sketch map image set is to generate water by computer software and human assistance
Powder sketch collection image rather than the image that sketch style is generated by artist and witness, using this group of sketch map image set as spy
Needle collection.Here Sketch Guru can be used in the software for generating sketch collection, and the type present invention for generating sketch image can choose
Pigment sketch type.Because witness cannot provide suspicious information of appearing in court, but witness can provide rough and comprehensive description, water
The image of powder sketch has ignored some details of people to a certain extent but the color of clothes and layout are all identical i.e. outer
The color of set and trousers has almost no change, this facilitates from gallery data set to identify a people.SINPID data set can
To be obtained from https: //sites.google.com/site/SINPID2018.
Step 2: feature extraction, the present invention extracts collected pedestrian's data set (SINPID) two kinds of
Feature is evaluated, including LOMO patch feature and PCB depth characteristic.Because LOMO can be from three scale pyramid tables
Hsv color feature and SILTP textural characteristics are extracted in the image shown, this feature is extracted by patch, to different views
The variation of point has certain robustness；And PCB can extract a kind of depth convolution description being made of multiple portions grade feature
Symbol, this feature can be using whole image as input, and obtains the feature vector of each image.
Step 3: using dictionary learning technological learning isomery dictionary to DSAnd DN.The purpose for introducing dictionary learning technology is
The good data of different-style image indicates in order to obtain, therefore learns dictionary pair respectively to isomeric data.Assuming that X=[x1,
x2...xN] and Y=[y1, y2...yN] it is sketch map image set and normal image collection respectively. X∈Rd*N,Y∈Rd*NIt .d is image spy
The dimension of sign, N indicate the total quantity of sample image.DSAnd DNRespectively indicate the dictionary pair of sketch image and normal image, therefore benefit
It can be with the objective function of dictionary learning technological learning isomery dictionary pair is defined as:
Wherein dictionary is to DSAnd DNIt is respectively as follows: DS∈Rd*m,DN∈Rd*m, what m was indicated is the quantity of the element of dictionary centering.A
=[a1,a2...aN], B=[b1,b2...bN] .A indicate be X in DSOn code coefficient matrix, B indicate be Y in DNOn
Code coefficient matrix.
Step 4: from sketch image and normal image focusing study projection matrix P.Its purpose is to establish sketch map
Image set and normal image collection relationship between the two, find one and half coupling projection matrixes,
The difference between them is reduced to a certain extent.Assuming that P ∈ Rd*dIt is half coupling projection matrix.By minimizing A
With the distance of B code coefficient matrix, obtain half coupling projection matrix, thus reduce sample it
Between difference.Therefore half coupled maps matrix can calculate in the following manner:
Step 5: learning metric matrix W using the thought for identifying study.Its purpose is to improve good mark sheet
Show ability, in addition identifying bound term, of a sort sample can be made compact, inhomogeneous sample separation.Utilize identification thought
Carrying out metric learning and dictionary learning can calculate in the following manner:
What wherein S was indicated is that (i, j) element belongs to same class, and what D was indicated is that (i, j) element belongs to inhomogeneity.
M=WTW W∈Rd*d(5)；
In addition, over-fitting in order to prevent, below regularization of the present invention: parameter projection matrix P, coefficient A, B and identifying about
Beam item W.Regularization term can indicate are as follows:
In conjunction with above-mentioned formula (1), (2), (3), (6) then SMD2The objective function of L can be rewritten are as follows:
Wherein λ is the regularization parameter balance factor.
In step 6, parametric solution.Formula (7) is for DS,DN, P, W are not joint convex functions, but if other
It is that convex function solves for each variable in the case that variable is fixed.Therefore formula (7) is divided into 4 sons and asked by the present invention
Topic solves, i.e., dictionary indicates the update of coefficient update and half coupling projection matrix to update.It specifically includes:
1) present invention fixes other parameters to update A and B first.It updates A to retain the item for only existing A, formula (7)
It can rewrite are as follows:
For solution formula (8), the present invention can be by by αiDerivative be set as 0 to solve.αiIt is represented by as follows:
αi=(DS TDS+PTP+λI+(1-β)WTW)-1
B is similar with the solution of A,
bi=(DN TDN+(λ-1)I+(1-β)WTW)-1
2) D is updatedSAnd DN, update DSFormula (7) can be rewritten as follows:
For solution formula (11), DSCalculating can be obtained by following formula: DS=XAT(AAT+∧))-1(12)；
∧ is a diagonal matrix, similar to formula (11), updates DNFormula (7) can be rewritten as
It is as follows:
3) P, fixed other parameters are updated, the present invention can rewrite formula (7) are as follows:
Solution formula (14) can be solved by setting 0 for the derivative of P, and the solution of P is as follows:
P=BAT(AAT+λI))-1(15)；
4) final updating W, fixed other parameters, formula (7) can be rewritten as follows:
The present invention can update W by gradient descent algorithm,
Wherein t indicate be algorithm the number of iterations be t.
Further, in step 7, pedestrian identifies again.The sketch image and normal image for having randomly selected half are as instruction
Practice collection, remaining is as test set.And using sketch image as probe collection, normal image is as gallery collection.Provide a width
From the sketch image of probe, pedestrian retrieval is carried out.Sketch map image set and normal image collection are inputted, according to above-mentioned formula (12),
(15), (18) solve parameter DS,DN,P,W.Vacation lets f be the feature of sketch map image set, and G is the feature of common photo collection,
The present invention executes pedestrian retrieval in the following way:
1) sketch map image set dictionary D is solved first with formula (9)S, using the projection matrix P learnt, to solve DSIt is right
The coefficient matrix f answered, calculation are as follows:
2) normal image collection dictionary D is solved using formula (10)N, then solve DNCorresponding coefficient matrix g, calculation
It is as follows:
3) coefficient matrix f and normal image collection corresponding to sketch map image set are solved by formula (19) and formula (20)
Corresponding coefficient matrix g, the present invention, which can be retrieved, concentrates corresponding sketch image in normal image, can be by calculating two
The distance between a image obtains, and calculation is as follows:
Calculation formula (21) solves corresponding distance, then adjusts the distance and be ranked up, wherein apart from the smallest common photograph
Piece is exactly the picture being retrieved using sketch image.
The present invention is described further below with reference to experiment.
To verify whether this algorithm has good superiority, half Coupling Metric based on sketch image is identified into dictionary learning
Pedestrian retrieval algorithm and 6 kinds of comparison algorithms KISSME, XQDA, TDL, SLD2L, JDML and PCB are compared.This 6 kinds comparisons
Algorithm includes being based on dictionary learning based on measurement, is based on deep learning, the algorithm identified again based on pedestrian.Finally in the present invention
New data set SINPID verified as experimental data.
The evaluation index of retrieval is the matching probability that accumulative matching properties curve CMC, CMC curve is a kind of top-k,
Each sample that normal image is concentrated successively is taken into a distance with the calculation of its probe image collection, is then ranked up, rank is
The value of the value of top-k selected by the present invention, CMC matching rate tends to 1, if test number it is more, identification it is accurate
Rate is better.
In order to verify the method for the present invention SMD2The performance of L is tested in new SINPID data set, and experimental result is table 1
And Fig. 2.
The matching rate of top-r of the table 1 on SINPID data set
What (a) was indicated in Fig. 2 be using the experimental result of LOMO feature, (b) in indicate is reality using PCB feature
Test result.From the experimental result in table 1 it can be seen that algorithm of the invention obtains higher matching rate in comparison algorithm,
For example, SMD2L has the comparison algorithm XQDA of LOMO feature to compare in SINPID data set, and the matching rate of Rank-1 improves
2.1% (=36.2%-34.1%).The present invention also utilizes depth characteristic and assesses simultaneously, it will be seen that some results is lower than
The method of LOMO feature.Its reason may have three aspects: 1) SINPID data set only has the image of 400 200 people, this makes
PCB does not pass through enough sample trainings.2) watercolor sketch formula image is than information that the common photo of another video camera includes
Amount is few.3) network architecture is not suitable for isomery pedestrian sample, and is suitable for two kinds of normal samples.
By being tested above it can be seen that the method for the present invention SMD2Most of matching rates of L are better than other with LOMO
The method of feature and PCB depth characteristic.Mainly have 3 aspects: 1) using half coupled maps strategy, can by sketch image with it is general
The relationship of logical photo connects, and reduces the difference between isomery sample.2) study metric matrix can reveal that isomeric data
Inherence projection.3) for the sketch image and common photo under complex scene, dictionary is to preferable applicability.4) differentiate about
Beam can make the compact and different classes of separation of same category, be conducive to retrieval and classification task.As seen from the above analysis, newly
SINPID data set be to different types of feature it is stable, suitable for the further evaluation of pedestrian retrieval, show SMD2L is calculated
The superiority of method.
The foregoing is merely illustrative of the preferred embodiments of the present invention, is not intended to limit the invention, all in essence of the invention
Made any modifications, equivalent replacements, and improvements etc., should all be included in the protection scope of the present invention within mind and principle.
Claims (10)
1. the pedestrian retrieval method that a kind of half Coupling Metric based on sketch image identifies dictionary learning, which is characterized in that described
The pedestrian retrieval method that half Coupling Metric based on sketch image identifies dictionary learning includes:
Establish isomery pedestrian retrieval database SINPID, including sketch map image set and regular image set；Feature extraction is carried out, from figure
Feature is extracted as in, and different image labelings is carried out with corresponding color；
Data processing to isomery is carried out to the sample characteristics extracted；Introduce the word of dictionary learning method study isomeric data
Allusion quotation pair, from sketch map image set and regular image focusing study mapping matrix；
And identification algorithm study measurement is introduced, carry out pedestrian's inspection that half Coupling Metric based on sketch image identifies dictionary learning
Rope.
2. half Coupling Metric based on sketch image identifies the pedestrian retrieval method of dictionary learning as described in claim 1,
It is characterized in that, it includes following step that the pedestrian retrieval that half Coupling Metric based on sketch image identifies dictionary learning, which calculates method,
It is rapid:
Step 1: shooting photo with camera and establishing the database of oneself；
Step 2: feature extraction；
Step 3: using dictionary learning technological learning isomery dictionary to DSAnd DN；
Step 4: learning projection matrix P from sketch map image set and normal image collection；
Step 5: learning metric matrix W using the thought for identifying study；
Step 6: parametric solution；
Step 7: pedestrian identifies again.
3. half Coupling Metric based on sketch image identifies the pedestrian retrieval method of dictionary learning as claimed in claim 2,
It is characterized in that, is specifically included in step 1:
The photo under real scene is shot with multiple cameras to collect data, collects multiple pictures data in total；Then by one
The data of camera are as normal image collection, and in addition the data of a camera form sketch map image set by processing；
Sketch map image set randomly selects the sketch image of half and the sample of normal image as training set, and remaining be used as is surveyed
Examination collection；And processing is split to image set.
4. half Coupling Metric based on sketch image identifies the pedestrian retrieval method of dictionary learning, step as claimed in claim 2
In rapid two, two kinds of feature is extracted to collected pedestrian's data set SINPID and is evaluated, including LOMO patch
Feature and PCB depth characteristic；LOMO patch feature is extracted by patch；PCB depth characteristic is to be made of multiple portions grade feature
Depth convolution descriptor and obtain the feature vector of each image using whole image as input.
5. half Coupling Metric based on sketch image identifies the pedestrian retrieval method of dictionary learning, step as claimed in claim 2
In rapid three, X=[x1, x2...xN] and Y=[y1, y2...yN] it is sketch map image set and normal image collection respectively；X∈Rd*N,Y∈
Rd*N；D is the dimension of characteristics of image, and N indicates the total quantity of sample image；DSAnd DNRespectively indicate sketch image and normal image
Dictionary pair utilizes the objective function of dictionary learning technological learning isomery dictionary pair is defined as:
Wherein dictionary is to DSAnd DNIt is respectively as follows: DS∈Rd*m,DN∈Rd*m, what m was indicated is the quantity of the element of dictionary centering.A=
[a1,a2...aN], B=[b1,b2...bN] .A indicate be X in DSOn code coefficient matrix, B indicate be Y in DNOn volume
Code coefficient matrix.
6. half Coupling Metric based on sketch image identifies the pedestrian retrieval method of dictionary learning, step as claimed in claim 2
In rapid four, P ∈ Rd*dIt is half coupling projection matrix；By minimizing the distance of A and B code coefficient matrix, half coupling projection is obtained
Matrix；Half coupled maps matrix calculates in the following manner:
In step 5, using identifying, thought carries out metric learning and dictionary learning calculates in the following manner:
What wherein S was indicated is that (i, j) element belongs to same class, and what D was indicated is that (i, j) element belongs to inhomogeneity；
M=WTW W∈Rd*d(5)；
Parameter projection matrix P, coefficient A, B and identification bound term W；Regularization term indicates are as follows:
In conjunction with above-mentioned formula (1), (2), (3), (6)；Then SMD2The objective function of L is rewritten are as follows:
Wherein λ is the regularization parameter balance factor.
7. half Coupling Metric based on sketch image identifies the pedestrian retrieval method of dictionary learning, step as claimed in claim 2
The dictionary for dividing formula (7) in rapid six indicates that the update subproblem of coefficient update and half coupling projection matrix includes: to update
1) parameter and undated parameter A and B of the set formula (8) in addition to parameter A and B；A is updated to only existing the item of parameter A simultaneously
Retained, formula (7) is rewritten are as follows:
By by αiDerivative be set as 0 solution；αiIt is expressed as:
αi=(DS TDS+PTP+λI+(1-β)WTW)-1
B is updated to the item for only existing parameter B and to retain,
bi=(DN TDN+(λ-1)I+(1-β)WTW)-1
2) D is updatedSAnd DN, update DSFormula (7) is rewritten as follows:
DSCalculating can be obtained by following formula: DS=XAT(AAT+∧))-1(12)；
∧ is a diagonal matrix, updates DNFormula (7) is rewritten as follows:
3) P is updated, fixed other parameters rewrite formula (7) are as follows:
By setting 0 solution for the derivative of P, the solution of P is as follows:
P=BAT(AAT+λI))-1(15)；
4) final updating W, fixed other parameters, formula (7) is rewritten as follows:
W is updated by gradient descent algorithm,
Wherein t indicate be algorithm the number of iterations be t；
8. half Coupling Metric based on sketch image identifies the pedestrian retrieval method of dictionary learning, step as claimed in claim 2
In rapid seven, sketch map image set and normal image collection are inputted, according to formula (12), (15), (18) solve parameter DS,DN,P,W；F is
The feature of sketch map image set, G are the features of common photo collection, are specifically included:
Sketch map image set dictionary D is solved using formula (9)S, using the projection matrix P learnt, to solve DSCorresponding coefficient square
Battle array f, calculation are as follows:
Normal image collection dictionary D is solved using formula (10)N, then solve DNCorresponding coefficient matrix g, calculation are as follows:
It is solved corresponding to coefficient matrix f and normal image collection corresponding to sketch map image set by formula (19) and formula (20)
Coefficient matrix g, retrieve and concentrate corresponding sketch image in normal image, obtained by calculating the distance between two images
, calculation is as follows:
Calculation formula (21) solves corresponding distance, then adjusts the distance and be ranked up, and is exactly benefit apart from the smallest common photo
The picture being retrieved with sketch image.
9. a kind of pedestrian retrieval side for implementing half Coupling Metric described in claim 1 based on sketch image and identifying dictionary learning
Half Coupling Metric based on sketch image of method identifies the pedestrian retrieval control system of dictionary learning.
10. the pedestrian retrieval side that kind of implementation half Coupling Metric described in claim 1 based on sketch image identifies dictionary learning
The traffic route pedestrian image searching terminal of method.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910325189.4A CN110046669B (en) | 2019-04-22 | 2019-04-22 | Pedestrian retrieval method based on sketch image half-coupling metric identification dictionary learning |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910325189.4A CN110046669B (en) | 2019-04-22 | 2019-04-22 | Pedestrian retrieval method based on sketch image half-coupling metric identification dictionary learning |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110046669A true CN110046669A (en) | 2019-07-23 |
| CN110046669B CN110046669B (en) | 2019-12-24 |
Family
ID=67278414
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910325189.4A Active CN110046669B (en) | 2019-04-22 | 2019-04-22 | Pedestrian retrieval method based on sketch image half-coupling metric identification dictionary learning |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110046669B (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113240715A (en) * | 2021-05-29 | 2021-08-10 | 刘文平 | Multiple feature-driven similarity measurement learning figure retrieval method |
| CN113553947A (en) * | 2021-07-23 | 2021-10-26 | 湖南大学 | Method and device for re-identifying pedestrian by embedding multi-mode into generation description and electronic equipment |
| CN116052095A (en) * | 2023-03-31 | 2023-05-02 | 松立控股集团股份有限公司 | Vehicle re-identification method for smart city panoramic video monitoring |
Citations (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7567715B1 (en) * | 2004-05-12 | 2009-07-28 | The Regents Of The University Of California | System and method for representing and encoding images |
| US20120162244A1 (en) * | 2010-12-28 | 2012-06-28 | Microsoft Corporation | Image search color sketch filtering |
| US20140379730A1 (en) * | 2013-06-24 | 2014-12-25 | Fujitsu Limited | Multimodality-based image tagging apparatus and method |
| CN105550649A (en) * | 2015-12-09 | 2016-05-04 | 武汉工程大学 | Extremely low resolution human face recognition method and system based on unity coupling local constraint expression |
| CN106919919A (en) * | 2017-02-28 | 2017-07-04 | 西安电子科技大学 | A kind of SAR target discrimination methods based on multiple features fusion word bag model |
| CN108027836A (en) * | 2015-09-24 | 2018-05-11 | 高通股份有限公司 | With the system and method for the query image access images of capture |
| US20180174313A1 (en) * | 2016-12-20 | 2018-06-21 | Toshiba Tec Kabushiki Kaisha | Information processing apparatus and information processing method |
| CN108647690A (en) * | 2017-10-17 | 2018-10-12 | 南京工程学院 | The sparse holding projecting method of differentiation for unconstrained recognition of face |
| CN108932518A (en) * | 2018-06-22 | 2018-12-04 | 大连理工大学 | A kind of feature extraction of shoes watermark image and search method of view-based access control model bag of words |
| CN109064403A (en) * | 2018-08-10 | 2018-12-21 | 安徽师范大学 | Fingerprint image super-resolution method based on classification coupling dictionary rarefaction representation |
| CN109117860A (en) * | 2018-06-27 | 2019-01-01 | 南京邮电大学 | A kind of image classification method based on subspace projection and dictionary learning |
| CN109284668A (en) * | 2018-07-27 | 2019-01-29 | 昆明理工大学 | A kind of pedestrian's weight recognizer based on apart from regularization projection and dictionary learning |
| CN109409201A (en) * | 2018-09-05 | 2019-03-01 | 昆明理工大学 | A kind of pedestrian's recognition methods again based on shared and peculiar dictionary to combination learning |
-
2019
- 2019-04-22 CN CN201910325189.4A patent/CN110046669B/en active Active
Patent Citations (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7567715B1 (en) * | 2004-05-12 | 2009-07-28 | The Regents Of The University Of California | System and method for representing and encoding images |
| US20120162244A1 (en) * | 2010-12-28 | 2012-06-28 | Microsoft Corporation | Image search color sketch filtering |
| US20140379730A1 (en) * | 2013-06-24 | 2014-12-25 | Fujitsu Limited | Multimodality-based image tagging apparatus and method |
| CN108027836A (en) * | 2015-09-24 | 2018-05-11 | 高通股份有限公司 | With the system and method for the query image access images of capture |
| CN105550649A (en) * | 2015-12-09 | 2016-05-04 | 武汉工程大学 | Extremely low resolution human face recognition method and system based on unity coupling local constraint expression |
| US20180174313A1 (en) * | 2016-12-20 | 2018-06-21 | Toshiba Tec Kabushiki Kaisha | Information processing apparatus and information processing method |
| CN106919919A (en) * | 2017-02-28 | 2017-07-04 | 西安电子科技大学 | A kind of SAR target discrimination methods based on multiple features fusion word bag model |
| CN108647690A (en) * | 2017-10-17 | 2018-10-12 | 南京工程学院 | The sparse holding projecting method of differentiation for unconstrained recognition of face |
| CN108932518A (en) * | 2018-06-22 | 2018-12-04 | 大连理工大学 | A kind of feature extraction of shoes watermark image and search method of view-based access control model bag of words |
| CN109117860A (en) * | 2018-06-27 | 2019-01-01 | 南京邮电大学 | A kind of image classification method based on subspace projection and dictionary learning |
| CN109284668A (en) * | 2018-07-27 | 2019-01-29 | 昆明理工大学 | A kind of pedestrian's weight recognizer based on apart from regularization projection and dictionary learning |
| CN109064403A (en) * | 2018-08-10 | 2018-12-21 | 安徽师范大学 | Fingerprint image super-resolution method based on classification coupling dictionary rarefaction representation |
| CN109409201A (en) * | 2018-09-05 | 2019-03-01 | 昆明理工大学 | A kind of pedestrian's recognition methods again based on shared and peculiar dictionary to combination learning |
Non-Patent Citations (2)
| Title |
|---|
| MA FEI ETC: "Multi-View based Coupled Dictionary Learning for Person Re-identification", 《2017 INTERNATIONAL CONFERENCE ON SECURITY, PATTERN ANALYSIS,》 * |
| XIAO-YUAN JING ETC: "Multi-spectrallow-rankstructureddictionarylearning", 《PATTERNRECOGNITION》 * |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113240715A (en) * | 2021-05-29 | 2021-08-10 | 刘文平 | Multiple feature-driven similarity measurement learning figure retrieval method |
| CN113553947A (en) * | 2021-07-23 | 2021-10-26 | 湖南大学 | Method and device for re-identifying pedestrian by embedding multi-mode into generation description and electronic equipment |
| CN116052095A (en) * | 2023-03-31 | 2023-05-02 | 松立控股集团股份有限公司 | Vehicle re-identification method for smart city panoramic video monitoring |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110046669B (en) | 2019-12-24 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109948425B (en) | Pedestrian searching method and device for structure-aware self-attention and online instance aggregation matching | |
| You et al. | Relative CNN-RNN: Learning relative atmospheric visibility from images | |
| Mouine et al. | A shape-based approach for leaf classification using multiscaletriangular representation | |
| CN106446150B (en) | A kind of method and device of vehicle precise search | |
| CN105809146B (en) | A kind of image scene recognition methods and device | |
| CN111104867A (en) | Recognition model training and vehicle heavy recognition method and device based on component segmentation | |
| CN110046669A (en) | Half Coupling Metric based on sketch image identifies the pedestrian retrieval method of dictionary learning | |
| CN107506703A (en) | A kind of pedestrian's recognition methods again for learning and reordering based on unsupervised Local Metric | |
| CN109344851B (en) | Image classification display method and device, analysis instrument and storage medium | |
| CN109299707A (en) | A kind of unsupervised pedestrian recognition methods again based on fuzzy depth cluster | |
| CN108764269A (en) | A kind of cross datasets pedestrian recognition methods again based on space-time restriction incremental learning | |
| CN109558833A (en) | A kind of face recognition algorithms evaluating method and device | |
| Clarke et al. | Perceptual similarity: A texture challenge. | |
| CN114758362B (en) | Clothing changing pedestrian re-identification method based on semantic perception attention and visual shielding | |
| CN112288761A (en) | Abnormal heating power equipment detection method and device and readable storage medium | |
| CN112507853A (en) | Cross-mode pedestrian re-identification method based on mutual attention mechanism | |
| CN116897012A (en) | Method and device for identifying physique of traditional Chinese medicine, electronic equipment, storage medium and program | |
| Burghardt et al. | Automated visual recognition of individual African penguins | |
| CN109670423A (en) | A kind of image identification system based on deep learning, method and medium | |
| Zhang et al. | Deep feature learning with attributes for cross-modality person re-identification | |
| CN114579794A (en) | Multi-scale fusion landmark image retrieval method and system based on feature consistency suggestion | |
| CN113220915B (en) | Remote sensing image retrieval method and device based on residual attention | |
| CN109492120A (en) | Model training method, search method, device, electronic equipment and storage medium | |
| CN113705310A (en) | Feature learning method, target object identification method and corresponding device | |
| CN109241315A (en) | A kind of fast face search method based on deep learning |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |