CN106776950B - On-site shoe-print trace pattern image retrieval method based on expert experience guidance - Google Patents
On-site shoe-print trace pattern image retrieval method based on expert experience guidance Download PDFInfo
- Publication number
- CN106776950B CN106776950B CN201611097257.9A CN201611097257A CN106776950B CN 106776950 B CN106776950 B CN 106776950B CN 201611097257 A CN201611097257 A CN 201611097257A CN 106776950 B CN106776950 B CN 106776950B
- Authority
- CN
- China
- Prior art keywords
- image
- images
- similarity
- shoe
- library
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
- 238000000034 method Methods 0.000 title claims abstract description 47
- 239000011159 matrix material Substances 0.000 claims abstract description 25
- 238000004364 calculation method Methods 0.000 claims abstract description 9
- 230000003595 spectral effect Effects 0.000 claims description 38
- 238000001228 spectrum Methods 0.000 claims description 18
- 230000006870 function Effects 0.000 claims description 17
- 238000012549 training Methods 0.000 claims description 16
- 238000012163 sequencing technique Methods 0.000 claims description 9
- 238000011840 criminal investigation Methods 0.000 claims description 6
- 230000008569 process Effects 0.000 claims description 5
- 239000004576 sand Substances 0.000 claims description 4
- 238000012935 Averaging Methods 0.000 claims description 3
- 238000007477 logistic regression Methods 0.000 claims description 3
- 239000000126 substance Substances 0.000 claims description 3
- 238000010276 construction Methods 0.000 claims 1
- 238000002474 experimental method Methods 0.000 abstract description 9
- 230000009467 reduction Effects 0.000 abstract description 2
- 230000009466 transformation Effects 0.000 description 5
- 239000000284 extract Substances 0.000 description 2
- 238000012545 processing Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000003672 processing method Methods 0.000 description 1
- 230000035939 shock Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
- G06F16/58—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
- G06F16/583—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content
Landscapes
- Engineering & Computer Science (AREA)
- Library & Information Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
- Image Analysis (AREA)
Abstract
The invention relates to a spot shoe print trace pattern image retrieval method based on expert experience guidance, which utilizes expert experience to give out corresponding expert opinion similarity scores for the same spot image as an inquiry image, and utilizes the scores to guide retrieval, thereby improving retrieval precision; the expert opinion score prediction model is constructed, so that the reduction of the retrieval precision caused by subjective reasons of people is effectively avoided; by introducing a manifold learning method and considering the relation between every two images in the library, the retrieval precision is further improved; the invention introduces the coefficient matrix, and effectively solves the problem that the calculation ordering score tends to zero in the traditional manifold learning retrieval. Experiments prove that the retrieval precision of the method in the retrieval experiment of the on-site shoe printing pattern image is obviously superior to that of the prior art, and the retrieval performance is improved by using a small amount of same on-site shoe printing pattern image samples. Therefore, the method can be widely applied to the field of on-site shoe printing pattern image retrieval.
Description
Technical Field
The invention relates to an image retrieval method, in particular to an on-site shoe-print trace pattern image retrieval method based on expert experience guidance. Relating to the calculation of a patent classification number G06; calculating; the counting G06F electric digital data processing G06F17/00 is particularly suitable for the retrieval of specific functional digital computing devices or data processing methods G06F17/30 information; and its database structure.
Background
In criminal cases, sole trace patterns play a significant role in the detection of cases. Among them, the most challenging task is to compare the low-quality images of sole traces left in the crime scene with the sole traces in the database to find the best matching image. The on-site sole trace pattern image retrieval method aims to help an identification expert to accurately retrieve the most matched sole trace pattern image, so that the solution solving efficiency is improved.
At present, shoe printing pattern image retrieval algorithms at home and abroad are mainly divided into three categories: a retrieval method based on shoe printing patterns, local areas and interest points. However, these methods search the artificially synthesized clear and complete shoe mark images, and in practical applications, the shoe mark images acquired in the field in a multidimensional manner are required to be searched, and in most environments, the shoe mark images acquired in the field have the characteristics of low image quality, incomplete images or unclear images, and the like, so that the above-mentioned several types of shoe mark searching methods have many limitations in practical applications. In 2015, Xinnian Wang and the like propose an automatic retrieval method for on-site shoe-print pattern images based on wavelet-Fourier transform, which divides the shoe-print pattern images into a sole part and a heel part to respectively extract features, and the specific method comprises the following steps: respectively performing wavelet transformation on the sole and the heel, performing Fourier transformation on the images after the wavelet transformation, performing polar coordinate transformation on the amplitude values of the images, performing Fourier transformation to obtain the spectral characteristics of the images, and matching the spectral characteristics. The accuracy of the match was 87.5%.
Although the prior art has achieved certain achievements for shoe mark retrieval, the existing method cannot achieve good retrieval results for a large number of low-quality on-site shoe mark images, and the existing on-site shoe mark image retrieval method only considers the relation between a query image and an image in each library and does not consider the relation between every two images in the library. In the process of implementing the invention, the inventor finds that the following problems exist in the prior art:
1) the existing method extracts low-level features and matches the low-level features according to the features, and does not consider semantic gap between high-level semantic concepts and the low-level features, so that the retrieval precision is not high; 2) the existing method does not utilize other shoe mark images extracted on the same site as the query image, which is not beneficial to further improving the retrieval performance; 3) the prior method does not introduce the experience of criminal investigation personnel, and easily causes the inconsistency of objective retrieval results and subjective results of the criminal investigation personnel.
In view of the above, the present invention is particularly proposed.
Disclosure of Invention
In view of the above problems, the present invention provides an on-site shoe-print trace image retrieval method based on expert experience guidance, which includes the following steps:
calculating the similarity between the on-site shoe print trace image to be inquired and the sample images in the image library, and sequencing the images in the image library from large to small according to the similarity score; calculating the similarity score between any two image samples in the image library, and constructing a manifold regularization item;
dividing the pattern similarity degree of the two shoe mark images into a plurality of similar grades and grades corresponding to the grades by an expert according to experience, marking the corresponding similar grades for the sample images of the query graph in the same field according to the grades to be used as the subjective opinion similarity score of the expert,
searching k nearest neighbor images of sample images collected from the same site as the query site shoe mark image in an image library; giving the similarity grade of the subjective opinions of the k nearest neighbor images and the corresponding scores of the grades, and expressing the similarity scores of the subjective opinions of the k nearest neighbor images and sample images acquired from the same scene for inquiring the scene shoe mark images as follows:
wherein o isiIs an expert subjective opinion similarity score, w (q), representing the similarity of the query image and the ith image in the image set collected from the same scene as the query imagej,ui) The similarity score between the jth image and the k nearest neighbor images in the image set acquired from the same scene as the query image is calculated as follows:
wherein A ismnThe elements in the frequency spectrum characteristic of the jth shoe-print pattern image in the image set Q collected from the same scene with the query image,
represents the mean value of the spectral feature A, BmnFor the elements in the spectral features of the k nearest neighbor images in the bin,
represents the average value of the spectral feature B;
-constructing an expert subjective opinion similarity score ysAnd (3) prediction model:
the similarity scores of the predicted expert opinions of the sample images collected from the same site as the shoe mark image of the query site and the k nearest neighbor images are expressed as a four-parameter logistic regression function about the similarity of the query image and the image;
-the parameters of the prediction model are obtained by learning, and a training sample library T { (x) is constructed and used(i),y(i)) And i is 1,2, …, M, learning the parameters of the prediction model by using a least square method, and obtaining the optimized parameters as follows:
the images in the training sample library T are randomly divided into LTGroups, each group consists of a reference image and the rest n (i) shoe mark images;
making a plurality of criminal investigation experts respectively give expert opinion similarity scores to each group of images in the training sample library T according to the similarity degree of the images and the group of reference images, averaging the expert opinion similarity scores, and recording as y(i)(ii) a Calculating the similarity score x of each group of reference images and the rest images(i)The similarity score calculation formula is as follows:
wherein A ismnFor training elements in the frequency spectrum characteristics of the ith shoe printing pattern image in the sample library,
representing spectral featuresAverage value of A, BmnTo train the elements in the spectral features of the set of reference images in the sample library,
represents the average value of the spectral feature B;
-using a formula for sample images taken from the same site as the image of the shoe mark in the query site, and for k nearest neighbor images thereof
Fusing the expert subjective opinion similarity score and the predictive expert opinion similarity score, wherein asAnd apRepresents a weight coefficient, and as+ap1 is ═ 1; for other images in the image library
Constructing and solving an objective function, calculating the ranking score of each image sample, giving a ranking result, and completing image retrieval; the objective function is as follows:
wherein f is*Calculate the ranking score, u, for the bestiRepresenting the ith image, f, in the image library UiRanking score for ith image in image library U α, gamma is weight coefficient, α<β，0<α,β,γ<1, R is coefficient matrix, is diagonal matrix, Ri,iRepresents the ith row and ith column element in R; w represents an adjacency matrix, wherein elements correspond to similarity scores of any two sample images in the image library U; c is a diagonal matrix, CiiFor the ith element on its diagonal, CiiRepresenting the sum of the similarity of the ith image and the remaining images,
qifor the ith image in the image set Q, which is acquired from the same site as the query image, Q1To query an image, Nk(qi) Is qiK nearest neighbor images;
and scoring the similarity of the opinions of the experts.
In a preferred embodiment, the image library,
As a preferred embodiment, the specific process of calculating the similarity between the on-site shoe mark image to be queried and the sample image in the image library is as follows:
extracting binary image frequency spectrum characteristics from the sample images in the library to obtain the similarity between the sample images and the query images;
-sorting the images of the shoe mark patterns in the sample image library according to the obtained similarity, so that each image in the sample image library obtains a sorting index Ind (u)i)；
-constructing a coefficient matrix R of the first term of the objective function, R being a diagonal matrix, R having the i-th row and i-th column elements RiiHas a value of
Where K denotes the total number of images in the library of field shoe mark sample images, K ═ N + N.
As a preferred embodiment, the similarity between the two image samples and constructing the manifold regularization term specifically includes the following steps:
-calculating a similarity score w (U) of the binary image spectral feature between two image samples in the image library Ui,uj) According to the following calculation formula:
wherein A ismnThe elements in the frequency spectrum characteristic of the ith shoe printing pattern image in the image library U,
represents the mean value of the spectral feature A, BmnThe elements in the frequency spectrum characteristic of the jth shoe printing pattern image in the image library U,
represents the average value of the spectral feature B; from the similarity score, a similarity matrix W (i, j) ═ W (u) can be obtainedi,uj)；
-constructing a laplacian manifold regularization term by similarity score, wherein
As a preferred implementation mode, the expert empirically divides the pattern similarity degree of two shoe mark images into a plurality of similar grades and grades corresponding to the grades, according to the grades, the sample images of the query graph in the same scene are marked with corresponding similar grades to be used as the subjective opinion similarity score of the expert,
-finding k nearest neighbor images in the image library from the same on-site sample image as the query on-site shoe mark image; giving the similarity grade of the subjective opinions of the k nearest neighbor images and the corresponding scores of the grades, and expressing the similarity scores of the subjective opinions of the k nearest neighbor images and sample images acquired from the same scene for inquiring the scene shoe mark images as follows:
wherein the content of the first and second substances,
representing the query image and a set of images acquired from the same site as the query image, qiFor the image in the image set, Nk(qj) For its k nearest neighbor images, oiFor its expert subjective opinion score, w (q)j,ui) Is qjSimilarity to its k nearest neighbor images.
Wherein o isiIs an expert subjective opinion similarity score, w (q), representing the similarity of the query image and the ith image in the image set collected from the same scene as the query imagej,ui) The similarity score between the jth image and the k nearest neighbor images in the image set acquired from the same scene as the query image is calculated as follows:
wherein A ismnIs an element in the frequency spectrum characteristic of the jth shoe printing pattern image in the image set collected from the same site as the query image,
represents the mean value of the spectral feature A, BmnFor the elements in the spectral features of the k nearest neighbor images in the bin,
represents the average value of the spectral feature B;
as a preferred embodiment, the solution process of converting the objective function into a matrix form is as follows:
f is optimized so that Q (f) is minimized, and the derivative of Q (f) over f can be written as:
make it equal to zero, i.e.
To obtain
Then
And sorting the image samples in the library according to the calculated sorting score, and outputting the sorted result.
Due to the adoption of the technical scheme, the invention has the following advantages:
(1) the innovation is as follows: 1) a new search method based on expert experience guidance is provided, the opinion similarity score is given by using the expert experience, the search is guided, and the search precision is improved;
2) an expert opinion similarity score prediction model is constructed, and the model is trained by using opinion similarity scores of a plurality of experts, so that the reduction of retrieval precision caused by subjective factors of people is effectively avoided;
3) the retrieval performance is improved by using a small amount of same on-site shoe printing pattern image samples and k nearest neighbor samples thereof;
4) different from the traditional manifold learning method, the method only considers the sample images collected from the same site as the query site shoe mark images and the guiding function of k nearest neighbor images on retrieval, and effectively avoids the calculation and ordering scores of other images in the library from tending to a very small value;
5) and through manifold regularization learning and the relation between every two images in the reference library, the retrieval precision is further improved.
(2) Effectiveness: experiments prove that the retrieval precision of the method in the retrieval experiment of the on-site shoe printing pattern image is obviously superior to that of the prior art, and the retrieval performance is improved by using a small amount of same on-site shoe printing pattern image samples.
Drawings
FIG. 1 is a flow chart of an on-site shoe print trace image retrieval method according to an embodiment of the present invention;
FIG. 2 is a schematic view of a field acquired image of a shoe mark trace;
FIG. 3 is a schematic diagram of an image of a shoe mark image acquired on site after binarization processing.
Detailed Description
The present invention will be described in detail with reference to the drawings and examples, but the present invention is not limited thereto.
On-site shoe print sample image library
Wherein
Representing a set of images of shoe mark marks to be retrieved collected from different sites,
representing the query image and a set of images acquired from the same site as the query image, where q1Is a query image;
as shown in fig. 1, a method for searching an image of an on-site shoe mark pattern based on expert experience guidance comprises the following steps:
step 101 of calculating the distance between the query graph and the image sample in the image librarySimilarity of charactersAnd sequencing the image samples in turn;
wherein the steps specifically include:
(1) extracting binary image spectral features from sample images in a library, wherein the feature extraction Algorithm is Wang XN, Sun HH, Yu Q, Zhang C (2015) Automatic geographic shock recent Algorithm for Real CrimeSenes. ACCV, Singapore, pp 399-;
obtaining the similarity between the sample image and the query image according to a formula
Wherein A ismnThe elements in the frequency spectrum characteristic of the ith shoe printing pattern image in the library,
represents the mean value of the spectral feature A, BmnTo query for elements in the spectral features of an image,
represents the average value of the spectral feature B;
(2) sequencing the shoe-print trace pattern images in the sample image library according to the similarity, so that each image in the sample image library obtains a sequencing index Ind (u)i)；
(3) A coefficient matrix R of the first term of the objective function is constructed,
r is a diagonal matrix, and the ith row and ith column element R in RiiHas a value of
Wherein K represents the total number of images in the image library of shoe mark samples, and K is N + N
Because the current mainstream only considers the similarity between the query graph and the sample images in the library and does not consider that the correlation between the sample images is also one of the important factors influencing the recall ratio, the recall ratio of the existing algorithm is not ideal, so the method is taken as a preferred embodiment,
102, calculating the similarity between any two image samples in an image library, and constructing a manifold regularization item;
wherein the steps specifically include:
(1) calculating similarity score of binary image spectrum characteristics between two image samples in image library Uw(ui,uj) According to a calculation formula
Wherein A ismnThe elements in the frequency spectrum characteristic of the ith shoe printing pattern image in the image library U,
represents the mean value of the spectral feature A, BmnThe elements in the frequency spectrum characteristic of the jth shoe printing pattern image in the image library U,
represents the average of the spectral feature B. From the similarity score, a similarity matrix W (i, j) ═ W (u) can be obtainedi,uj)；
(2) Constructing a Laplace manifold regularization term by similarity score, wherein
103, dividing similarity degrees of field image samples which are the same as the query image into five grades by criminal investigation experts according to the similarity of the field image samples and the query image, and distributing corresponding expert subjective opinion similarity scores to each similarity grade;
dividing sample images collected from the same site as the shoe print trace image of the query site in the image sample library into five grades according to the similarity degree with the shoe print pattern image of the query site, wherein the five grades are the same, approximately the same, similar, approximately similar and different, and distributing corresponding expert subjective opinion similarity scores to each similarity grade, wherein the expert subjective opinion similarity scores are 1,0.8,0.6,0.4 and 0.2 respectively;
step 104, searching k nearest neighbor images of sample images collected from the same site in the database as the site shoe mark image query, and giving the k nearest neighbor images the same expert subjective opinion similarity scores;
wherein the steps specifically include:
searching k nearest neighbor images of the sample images collected from the same site in the image library and the image of the shoe mark in the query site, wherein the numerical value of k is 5 in the embodiment; giving k nearest neighbor images, namely k images with the maximum similarity degree, the expert subjective opinion similarity grade and the grade corresponding score, and expressing the expert subjective opinion similarity scores of the k nearest neighbor images and sample images acquired from the same scene for inquiring the scene shoe mark images as follows:
wherein the content of the first and second substances,
representing the query image and a set of images acquired from the same site as the query image, qiFor the image in the image set, Nk(qj) For its k nearest neighbor images, oiFor its expert subjective opinion score, w (q)j,ui) Is qjSimilarity to its k nearest neighbor images.
Wherein A ismnThe elements in the frequency spectrum characteristic of the jth shoe-print pattern image in the image set Q collected from the same scene with the query image,
represents the mean value of the spectral feature A, BmnFor the elements in the spectral features of the k nearest neighbor images in the bin,
represents the average value of the spectral feature B;
thus, sample images collected from the same site as the site shoe mark image query and having similar expert subjective opinions of k nearest neighbor images can be obtainedDegree score ys；
105, constructing and training an expert opinion similarity score prediction model, and predicting the expert opinion similarity scores of the sample images collected from the same site as the on-site shoe mark image query and the k nearest neighbor images of the sample images;
wherein the steps specifically include:
1) constructing an expert opinion similarity score prediction model:
the similarity scores of the predicted expert opinions of the sample images collected from the same site as the shoe mark image of the query site and the k nearest neighbor images are expressed as a four-parameter logistic regression function about the similarity of the query image and the image;
2) the parameters of the prediction model are obtained through learning, and a training sample library T { (x) is constructed and used(i),y(i)) And i is 1,2, …, M, learning the parameters of the prediction model by using a least square method, and obtaining the optimized parameters as follows:
obtaining optimal parameters by least square method
λ1λ2λ3λ4The initial values are respectively lambda1The initial value of (A) is the maximum value max (y), lambda of the mean expert opinion score of the images in all training sample libraries2The initial value of (a) is the minimum value min (y), lambda of the mean expert opinion score of the images in all training sample libraries3The initial value of (a) is the mean (y), lambda, of the mean expert opinion scores of the images in all training sample libraries4Is 1.
The images in the training sample library T are randomly divided into LTGroups of one reference pictureAnd the rest n (i) shoe mark images;
making a plurality of criminal investigation experts respectively give expert opinion similarity scores to each group of images in the training sample library T according to the similarity degree of the images and the group of reference images, averaging the expert opinion similarity scores, and recording as y(i)(ii) a Calculating the similarity score x of each group of reference images and the rest images(i)The similarity score calculation formula is as follows:
wherein A ismnFor training elements in the frequency spectrum characteristics of the ith shoe printing pattern image in the sample library,
represents the mean value of the spectral feature A, BmnTo train the elements in the spectral features of the set of reference images in the sample library,
represents the average value of the spectral feature B;
step 106, fusing the expert subjective opinion similarity score and the forecast expert opinion similarity score to obtain an expert opinion similarity score;
for the sample image collected from the same scene as the scene of inquiring the shoe mark image and the k nearest neighbor images, the similarity score y of the expert subjective opinion can be obtained by using the step 103 and the step 104sThe similarity score y of the predicted expert opinions can be obtained in step 105pFor the sample image collected from the same site as the on-site shoe mark image to be inquired and k nearest neighbor images thereof, a formula is utilized
Fusing the expert subjective opinion similarity score and the predictive expert opinion similarity score, wherein asAnd apRepresents a weight coefficient, and as+ap1 is ═ 1; for the figureOther images in the image library
And 107, constructing and solving an objective function, calculating the sequencing score of each image sample, and feeding the sequenced result back to the user.
The objective function constructed in the step is as follows:
wherein f is*Calculate the ranking score, u, for the bestiRepresenting the ith image, f, in the image library UiRanking score for ith image in image library U α, gamma is weight coefficient, α<β，0<α,β,γ<1, R is coefficient matrix, is diagonal matrix, Ri,iRepresents the ith row and ith column element in R; w represents an adjacency matrix, wherein elements correspond to similarity scores of any two sample images in the image library U; c is a diagonal matrix, CiiFor the ith element on its diagonal, CiiRepresenting the sum of the similarity of the ith image and the remaining images,
qifor the ith image in the image set Q, which is acquired from the same site as the query image, Q1To query an image, Nk(qi) Is qiK nearest neighbor images;
and scoring the similarity of the opinions of the experts.
To facilitate solving the objective function, the objective function can be written in a matrix form:
f is optimized so that Q (f) is minimized, and the derivative of Q (f) over f can be written as:
make it equal to zero, i.e.
Can obtain the product
Then
And sequencing the image samples in the library according to the calculated sequencing score, and feeding back the sequenced result to the user.
The effectiveness of the on-site shoe-print trace pattern image retrieval method based on expert experience guidance provided by the embodiment of the invention is verified by a specific experiment, which specifically comprises the following steps:
two shoe print image databases were used for the experiments, respectively MUES-SR10KS2S shoe print image search database (Wang XN, Sun HH, Yu Q, Zhang C (2015) Automatic shoes printing Retrieval Algorithm for real documents Scenex. ACCV, Singapore, pp 399-. MUES-SR10KS2S contained 10096 shoe print images collected from the case scene, including 72 query images, as shown in FIGS. 2 and 3. The CSFID-170 database contains 170 shoe print query images taken from the case site and 1175 image samples taken from the suspect. The experimental result is measured by Recall @ K%, namely the Recall ratio of the first K% of the sequencing result. In the experiment, the method performs the experiment on the MUES-SR10KS2S shoe print image retrieval database, and obtains the following experiment results:
comparison with the method proposed by Adam Kortylewski on the CSFID-170 database gave the following experimental results:
as can be seen from the table, compared with the result of the comparison method, the scheme provided by the method can obviously improve the retrieval precision.
The embodiments in the above embodiments can be further combined or replaced, and the embodiments are only used for describing the preferred embodiments of the present invention, and do not limit the concept and scope of the present invention, and various changes and modifications made to the technical solution of the present invention by those skilled in the art without departing from the design idea of the present invention belong to the protection scope of the present invention.
Claims (6)
1. An on-site shoe-print trace pattern image retrieval method based on expert experience guidance is characterized by comprising the following steps:
calculating the similarity between the on-site shoe print trace image to be inquired and the sample images in the image library, and sequencing the images in the image library from large to small according to the similarity score;
calculating a similarity score between any two image samples in the image library to construct a manifold regularization term;
dividing the pattern similarity degree of the two shoe print trace images into a plurality of similar grades and grades corresponding to the grades by an expert according to experience, and marking the corresponding similar grades for sample images of the query graph in the same field according to the grades to be used as the scores of the subjective opinion similarity of the expert;
searching k nearest neighbor images of sample images collected from the same site as the query site shoe mark image in an image library; giving the similarity grade of the subjective opinions of the k nearest neighbor images and the corresponding scores of the grades, and expressing the similarity scores of the subjective opinions of the k nearest neighbor images and sample images acquired from the same scene for inquiring the scene shoe mark images as follows:
wherein o isiIs an expert subjective opinion similarity score, w (q), representing the similarity of the query image and the ith image in the image set collected from the same scene as the query imagej,ui) The similarity score between the jth image and the k nearest neighbor images in the image set acquired from the same scene as the query image is calculated as follows:
wherein A ismnThe elements in the frequency spectrum characteristic of the jth shoe-print pattern image in the image set Q collected from the same scene with the query image,
represents the mean value of the spectral feature A, BmnFor the elements in the spectral features of the k nearest neighbor images in the bin,
represents the average value of the spectral feature B;
constructing an expert subjective opinion similarity score prediction model:
the similarity scores of the predicted expert opinions of the sample images collected from the same site as the shoe mark image of the query site and the k nearest neighbor images are expressed as a four-parameter logistic regression function about the similarity of the query image and the image;
the parameters of the prediction model are obtained through learning, and a training sample library T { (x) is constructed and used(i),y(i)) And i is 1,2, …, M, learning the parameters of the prediction model by using a least square method, and obtaining the optimized parameters as follows:
the images in the training sample library T are randomly divided into LTGroups, each group consists of a reference image and the rest n (i) shoe mark images;
making a plurality of criminal investigation experts respectively give expert opinion similarity scores to each group of images in the training sample library T according to the similarity degree of the images and the group of reference images, averaging the expert opinion similarity scores, and recording as y(i)(ii) a Calculating the similarity score x of each group of reference images and the rest images(i)The similarity score calculation formula is as follows:
wherein A ism′n′For training elements in the frequency spectrum characteristics of the ith shoe printing pattern image in the sample library,
represents the mean value of the spectral feature A, Bm′n′To train the elements in the spectral features of the set of reference images in the sample library,
represents the average value of the spectral feature B;
-using a formula for sample images acquired from the same site as the image of the shoe mark in the query site and for k nearest neighbor images thereof
Subjective opinion similarity scoring and predicting experts for expertsFusing opinion similarity scores, wherein asAnd apRepresents a weight coefficient, and as+ap1 is ═ 1; for other images in the image library
Constructing and solving an objective function, calculating the ranking score of each image sample, giving a ranking result, and completing image retrieval; the objective function is as follows:
wherein f is*Calculate the ranking score, u, for the bestiRepresenting the ith image, f, in the image library UiRanking score for ith image in image library U α, gamma is weight coefficient, α<β，0<α,β,γ<1, R is coefficient matrix, is diagonal matrix, Ri,iRepresents the ith row and ith column element in R; w denotes an adjacency matrix, WijRepresenting the ith row and ith column elements in W, wherein the elements correspond to the similarity scores of any two sample images in the image library U; c is a diagonal matrix, CiiFor the ith element on its diagonal, CiiRepresenting the sum of the similarity of the ith image and the remaining images,
qifor the ith image in the image set Q, collected from the same site as the query image, Nk(qi) Is qiK nearest neighbor images;
and scoring the similarity of the opinions of the experts.
2. The expert experience based on guided on-site shoe imprint pattern image retrieval method of claim 1, further characterized in that said image library,
wherein
Representing a set of images of shoe mark marks to be retrieved collected from different sites,
representing the query image and a set of images acquired from the same site as the query image, where q1To query the image.
3. The expert experience guidance-based on-site shoe print trace image retrieval method according to claim 1, further characterized in that the specific process of calculating the similarity between the on-site shoe print trace image to be inquired and the sample image in the image library is as follows:
extracting binary image frequency spectrum characteristics from a sample image in an image library to obtain the similarity between the sample image and a query image;
-sorting the images of the shoe imprints in the library of sample images according to the obtained similarities, so that each image in the library of sample images obtains a sorting index Ind (u)i)；
-constructing a coefficient matrix R of the first term of the objective function, R being a diagonal matrix, R having the elements R in the ith row and ith columniiHas a value of
And K represents the total number of images in the shoe mark sample image library, and K is N '+ N, N' and N respectively represent the number of sample images in the shoe mark image set D to be retrieved, which is acquired from different sites, and the image set Q, which is acquired from the same site with the query image.
4. The expert experience guidance-based on-site shoe print trace image retrieval method according to claim 1, further characterized in that the similarity between the two image samples and the construction of the manifold regularization term specifically comprise the steps of:
similarity score w (u) of the binary image spectral features between two image samplesi,uj) According to the following calculation formula:
wherein A ism″′n″′The elements in the frequency spectrum characteristic of the ith shoe printing pattern image in the image library U,
represents the mean value of the spectral feature A, Bm″′n″′The elements in the frequency spectrum characteristic of the jth shoe printing pattern image in the image library U,
represents the average value of the spectral feature B; from the similarity score, a similarity matrix W (i, j) ═ W (u) can be obtainedi,uj)；
-constructing a Laplace manifold regularization term by similarity score, wherein
5. The expert experience guided field shoe print trace image based retrieval method of claim 1 further characterized by:
dividing the pattern similarity degree of the two shoe mark images into a plurality of similar grades and grades corresponding to the grades by an expert according to experience, and marking the corresponding similar grades on the sample images of the query graph in the same field according to the grades to be used as the scores of the subjective opinion similarity of the expert;
searching k nearest neighbor images of sample images collected from the same site as the query site shoe mark image in an image library; giving the similarity grade of the subjective opinions of the k nearest neighbor images and the corresponding scores of the grades, and expressing the similarity scores of the subjective opinions of the k nearest neighbor images and sample images acquired from the same scene for inquiring the scene shoe mark images as follows:
wherein the content of the first and second substances,
representing the query image and a set of images acquired from the same site as the query image, qiFor the image in the image set, Nk(qj) For its k nearest neighbor images, oiFor its expert subjective opinion score, w (q)j,ui) Is qjThe similarity with its k nearest neighbor images is calculated as follows:
wherein A ismnThe elements in the frequency spectrum characteristic of the jth shoe-print pattern image in the image set Q collected from the same scene with the query image,
represents the mean value of the spectral feature A, BmnFor the elements in the spectral features of the k nearest neighbor images in the bin,
represents the average value of the spectral feature B;
6. the expert experience guidance-based on-site shoe imprint pattern image retrieval method according to claim 1, further characterized in that the solution process of converting the objective function into a matrix form is as follows:
f is optimized so that Q (f) is minimized, and the derivative of Q (f) over f can be written as:
make it equal to zero, i.e.
To obtain
Then
And sorting the image samples in the library according to the calculated sorting score, and outputting the sorted result.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201611097257.9A CN106776950B (en) | 2016-12-02 | 2016-12-02 | On-site shoe-print trace pattern image retrieval method based on expert experience guidance |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201611097257.9A CN106776950B (en) | 2016-12-02 | 2016-12-02 | On-site shoe-print trace pattern image retrieval method based on expert experience guidance |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN106776950A CN106776950A (en) | 2017-05-31 |
| CN106776950B true CN106776950B (en) | 2020-05-05 |
Family
ID=58883018
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201611097257.9A Expired - Fee Related CN106776950B (en) | 2016-12-02 | 2016-12-02 | On-site shoe-print trace pattern image retrieval method based on expert experience guidance |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106776950B (en) |
Families Citing this family (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107436943B (en) * | 2017-07-31 | 2020-05-29 | 大连海事大学 | On-site sole pattern retrieval method combined with subjective significance |
| CN110188222B (en) * | 2019-06-03 | 2022-06-14 | 大连海事大学 | Shoe print retrieval method based on local semantic filter and bridging similarity |
| CN110287370B (en) * | 2019-06-26 | 2022-03-04 | 中国人民公安大学 | Crime suspect tracking method and device based on-site shoe printing and storage medium |
| CN112966716B (en) * | 2021-02-03 | 2023-10-27 | 大连海事大学 | Sketch-guided shoe print image retrieval method |
| CN112800267B (en) * | 2021-02-03 | 2024-06-11 | 大连海事大学 | Fine-granularity shoe print image retrieval method |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101464951A (en) * | 2007-12-21 | 2009-06-24 | 北大方正集团有限公司 | Image recognition method and system |
| US7702247B1 (en) * | 2005-03-04 | 2010-04-20 | John Michael Weldon | Latent image developer system and method |
| CN105023027A (en) * | 2014-04-18 | 2015-11-04 | 大连恒锐科技股份有限公司 | Sole trace pattern image retrieval method based on multi-feedback mechanism |
-
2016
- 2016-12-02 CN CN201611097257.9A patent/CN106776950B/en not_active Expired - Fee Related
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7702247B1 (en) * | 2005-03-04 | 2010-04-20 | John Michael Weldon | Latent image developer system and method |
| CN101464951A (en) * | 2007-12-21 | 2009-06-24 | 北大方正集团有限公司 | Image recognition method and system |
| CN105023027A (en) * | 2014-04-18 | 2015-11-04 | 大连恒锐科技股份有限公司 | Sole trace pattern image retrieval method based on multi-feedback mechanism |
Non-Patent Citations (1)
| Title |
|---|
| A manifold ranking based method using hybrid features;Xinnian Wang et al;《Multimed Tools Appl》;Springer Science+Business Media New York;20161031;全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN106776950A (en) | 2017-05-31 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106776950B (en) | On-site shoe-print trace pattern image retrieval method based on expert experience guidance | |
| CN107515895B (en) | Visual target retrieval method and system based on target detection | |
| CN101551809B (en) | Search method of SAR images classified based on Gauss hybrid model | |
| CN110851645B (en) | Image retrieval method based on similarity maintenance under deep metric learning | |
| CN102855268B (en) | Image ranking method and system based on attribute correlation | |
| Wiggers et al. | Image retrieval and pattern spotting using siamese neural network | |
| CN103473327A (en) | Image retrieval method and image retrieval system | |
| CN103810299A (en) | Image retrieval method on basis of multi-feature fusion | |
| CN102521366A (en) | Image retrieval method integrating classification with hash partitioning and image retrieval system utilizing same | |
| CN102663447B (en) | Cross-media searching method based on discrimination correlation analysis | |
| CN107895303B (en) | Personalized recommendation method based on OCEAN model | |
| CN106844739B (en) | Remote sensing image change information retrieval method based on neural network collaborative training | |
| CN102890700A (en) | Method for retrieving similar video clips based on sports competition videos | |
| CN107122411A (en) | A kind of collaborative filtering recommending method based on discrete multi views Hash | |
| CN107291895B (en) | Quick hierarchical document query method | |
| Rian et al. | Content-based image retrieval using convolutional neural networks | |
| Do et al. | Plant identification using score-based fusion of multi-organ images | |
| CN110866134A (en) | Image retrieval-oriented distribution consistency keeping metric learning method | |
| Chen et al. | Symmetric binary tree based co-occurrence texture pattern mining for fine-grained plant leaf image retrieval | |
| CN108319959A (en) | A kind of corps diseases image-recognizing method compressed based on characteristics of image with retrieval | |
| Yang et al. | Cross-batch reference learning for deep classification and retrieval | |
| Sadique et al. | Content-based image retrieval using color layout descriptor, gray-level co-occurrence matrix and k-nearest neighbors | |
| López-Cifuentes et al. | Attention-based knowledge distillation in scene recognition: the impact of a dct-driven loss | |
| CN111078859A (en) | Author recommendation method based on reference times | |
| Fu et al. | Effective logo retrieval with adaptive local feature selection |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20200505 |