CN110188694A - It is a kind of that shoeprints recognition sequence method is worn based on pressure characteristic - Google Patents
It is a kind of that shoeprints recognition sequence method is worn based on pressure characteristic Download PDFInfo
- Publication number
- CN110188694A CN110188694A CN201910464874.5A CN201910464874A CN110188694A CN 110188694 A CN110188694 A CN 110188694A CN 201910464874 A CN201910464874 A CN 201910464874A CN 110188694 A CN110188694 A CN 110188694A
- Authority
- CN
- China
- Prior art keywords
- footprint
- foot
- image
- pressure
- similarity
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 46
- 238000012549 training Methods 0.000 claims abstract description 17
- 108010084652 homeobox protein PITX1 Proteins 0.000 claims abstract description 8
- 238000009825 accumulation Methods 0.000 claims abstract description 3
- 238000010586 diagram Methods 0.000 claims description 26
- 230000004927 fusion Effects 0.000 claims description 10
- 238000005259 measurement Methods 0.000 claims description 6
- 238000000605 extraction Methods 0.000 claims description 4
- 235000002566 Capsicum Nutrition 0.000 claims description 3
- 239000006002 Pepper Substances 0.000 claims description 3
- 235000016761 Piper aduncum Nutrition 0.000 claims description 3
- 235000017804 Piper guineense Nutrition 0.000 claims description 3
- 235000008184 Piper nigrum Nutrition 0.000 claims description 3
- 230000002159 abnormal effect Effects 0.000 claims description 3
- 238000004364 calculation method Methods 0.000 claims description 3
- 238000006243 chemical reaction Methods 0.000 claims description 3
- 238000005260 corrosion Methods 0.000 claims description 3
- 230000007797 corrosion Effects 0.000 claims description 3
- 230000002708 enhancing effect Effects 0.000 claims description 3
- 238000001914 filtration Methods 0.000 claims description 3
- 239000011159 matrix material Substances 0.000 claims description 3
- 238000000691 measurement method Methods 0.000 claims description 3
- 238000010606 normalization Methods 0.000 claims description 3
- 238000007781 pre-processing Methods 0.000 claims description 3
- 238000007639 printing Methods 0.000 claims description 3
- 150000003839 salts Chemical class 0.000 claims description 3
- 230000011218 segmentation Effects 0.000 claims description 3
- 238000012935 Averaging Methods 0.000 claims 1
- 244000203593 Piper nigrum Species 0.000 claims 1
- 238000010276 construction Methods 0.000 claims 1
- 210000002683 foot Anatomy 0.000 description 103
- 238000010998 test method Methods 0.000 description 4
- 238000007689 inspection Methods 0.000 description 3
- 241000722363 Piper Species 0.000 description 2
- 238000004458 analytical method Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 238000012360 testing method Methods 0.000 description 2
- 101100136092 Drosophila melanogaster peng gene Proteins 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 210000004744 fore-foot Anatomy 0.000 description 1
- 210000000548 hind-foot Anatomy 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/25—Fusion techniques
- G06F18/254—Fusion techniques of classification results, e.g. of results related to same input data
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Physics & Mathematics (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Bioinformatics & Computational Biology (AREA)
- General Engineering & Computer Science (AREA)
- Artificial Intelligence (AREA)
- Life Sciences & Earth Sciences (AREA)
- Human Computer Interaction (AREA)
- Multimedia (AREA)
- Measurement Of The Respiration, Hearing Ability, Form, And Blood Characteristics Of Living Organisms (AREA)
- Image Analysis (AREA)
Abstract
What the present invention provided a kind of combination pressure feature wears shoeprints recognition sequence method, is divided into off-line training process and online recognition.In off-line training, single piece of shoes print of the pressure footprint sequence image to be trained is extracted to pressure image, extract the centroid motion angle based on front and back foot, the footprint pressure energy spirogram four-tuple of the building pressure footprint sequence image to be trained finally obtains the property data base that the five-tuple feature representation of footprint is constituted.In online recognition, the data in footprint five-tuple feature to be identified and pre-stored property data base are calculated into similarity score, the identification to shoeprints sequence is worn is completed by ranking.This method is according to the minimum unit that can reflect people's walking habits, footprint pressure energy spirogram is constructed using two pieces of footprints, reduce error accumulation, improve accuracy of identification, it is merged by footprint pressure energy spirogram four-tuple Similarity-Weighted, matching result can be made more stable, what the present invention studied is to wear shoeprints sequence, and applicability is more extensive.
Description
Technical Field
The invention relates to the technical field of footprint identification, in particular to a method for identifying shoe wearing footprint sequences based on pressure characteristics.
Background
Feature extraction and feature matching are currently the main steps in biometric identification based on footprint sequences. The feature extraction includes a stride feature and a pressure feature. Acquisition of footprint stride characteristics[1,2]The method mainly comprises the following steps: 1) obtaining a connecting line of centroids of two adjacent footprints on the same side as a walking line; 2) calculating the distance between corresponding points of the front and rear adjacent left and right footprints (usually selecting the edge point of the heel) as the step length; 3) the distance between the edge point of the heel of the footprint and the opposite side walking line is step width; 3) the included angle between the footprint central line and the walking line in the advancing direction is a step angle.
The commonly used method for extracting pressure characteristics is to extract the footprint centroid pressure trace (cop) in the walking process[5]。
The adopted matching method mainly comprises the following steps: (1) and (4) a U test method: general purpose document[3]Six indexes of stride left by 20 suspects and criminals are counted through experiments, the absolute value of the mean value and the mean value difference of the samples is calculated, and then the absolute value is compared with a calculated mean value difference upper limit table, 18 persons can be excluded, and finally the extreme difference test method of criminals (2) is determined by combining other case solving means: the method comprises the steps of firstly, measuring normal step length, step width and step angle data in a footprint sequence of a field perpetrator, and calculating the mean value of each index. And secondly, measuring normal step length, step width and step angle data in the footprint sequence of the suspect, and calculating the mean value of each characteristic index, wherein the leaving condition of the footprint sequence of the suspect is close to the site as much as possible. And thirdly, respectively calculating the absolute value of the average difference of the indexes corresponding to the stride characteristics of each suspect and the site. The fourth step is to examine the suspect and the suspectAnd each field corresponds to an index, if the absolute value of the mean difference of the characteristic test indexes is less than or equal to the range value of the index, the judgment is positive: if the difference is greater than the range of the index, the result is negative. After six indexes of the suspect and the field stride are checked, if one index is not received, the result is judged to be negative; only when the six indexes are all received, the conclusion can be drawn that the stride characteristic of the suspect is consistent with the stride characteristic of the field perpetrator. (3) Membership degree test method[3]: "degree of membership" is a concept in fuzzy mathematics. It represents the closeness of an element to a fuzzy set. Before a case is not checked, the footprint characteristic of the perpetrator is a fuzzy set, and the corresponding domain of interest is all suspicious objects. Each suspect is to some extent affiliated with the fuzzy set of "footprint characteristics of perpetrators", and the difference of the closeness of each element to a fuzzy set can be represented by a data between 0 and 1, which is called the affiliation.
Reference documents:
[1] shandong political academy of academic department, a method for quantitatively testing comprehensive indexes by using stride characteristics, China, applied for published patent invention 201710902633.5.2017
[2] Design and implementation of an intelligent stride characteristic analysis and inspection system [ J ]. scientific and technical and engineering, 2014,14(3):64-69.
[3] Yuan Sheng, Wang Yang, China footprint inspection technology, and its prospect, academic newspaper of Yunnan police college, 2011(2): 119-.
[4] Peng, five stars, et al, chengzhang and research on a three-dimensional footprint analysis and inspection system [ D ]. university of kunming, 2016.
[5] The application of the U test method in the identification of steps [ J ] mathematical statistics and management, 1982(2):17-21.
[6]Zhou B,Singh M S,Doda Set al.The carpet knows:Identifying peoplein a smart environment from a single step.IEEE International Conference onPervasive Computing and Communications Workshops,2017.
Disclosure of Invention
In accordance with the above-mentioned technical problem, there is provided a method for identifying a sequence of footmarks of shoes worn based on pressure characteristics, comprising: an off-line training process and an on-line identification process; the off-line process comprises at least the following steps:
step S11: extracting a single shoe print relative pressure image in a centralized footprint image sequence to be trained;
step S12: calculating a front foot mass center deviation angle I and a rear foot mass center deviation angle I according to the single shoe print relative pressure image extracted in the S11;
step S13: constructing a footprint pressure energy diagram quadruplet I of the footprint sequence set to be trained;
step S14: and constructing a footprint sequence quintuple feature database by the front and back foot mass center offset angle I extracted in the S12 and the footprint energy quadruple I constructed in the S13.
Further, the online identification process comprises at least the following steps:
step S21: extracting a single shoe print relative pressure image of the online recognized footprint image sequence;
step S22: calculating a front and rear foot mass center offset angle II according to the single shoe print relative pressure image extracted in the S21 and identified on line;
step S23: constructing a footprint pressure energy graph quadruple II of the online recognized footprint sequence images;
step S24: according to the extracted front and rear foot centroid deviation angle II and pressure energy quadruple II groups identified on line, calculating similarity of the footmark sequence data to be identified and the footmark sequence data stored in the off-line training process respectively in a normalized cross-correlation measurement mode and a cosine distance measurement mode, fusing the calculated similarity scores, sorting the fused similarity scores from large to small, and taking the similarity score with the maximum similarity as the category of the footmarks to be retrieved.
Furthermore, the single shoe print relative pressure image extraction step comprises the following steps:
step S111: preprocessing an original footprint sequence; removing salt and pepper noise and plaque noise from the online recognized/to-be-trained footprint sequence image through median filtering, and enhancing the denoised image to obtain a footprint sequence image Fi(x, y) where i ∈ [1, N)]N is the total number of people in the database;
step S112: extracting the single shoe print image; and binarizing the preprocessed footprint sequence images, solving segmentation errors caused by shoe printing pattern fracture through closed operation of expansion and corrosion, calculating horizontal projection of the obtained images, and segmenting a single footprint by utilizing the interval between the footprint sequences. Marking the divided single footprints with connected domains and calculating the number and the area of the connected domains, deleting the images with the area and the number larger than the threshold value as abnormal data to obtain the jth single shoe print image L of the ith individualij(x, y) where i ∈ [1, N)],j∈[1,M]Wherein N represents the total number of people in the database, and M represents the number of single shoe print images obtained by dividing each person.
Step S113: calculating a relative pressure image of the single shoe print image extracted in step S112; scanning the single shoe print image line by line to find a pixel value l of a non-zero pointij(x, y), wherein x, y respectively represent abscissa and ordinate of single piece of shoe print image pixel, and calculate the mean value to the pixel value point that obtains:
making a difference l between the pixel value and the average value of each non-zero pointnew(x,y)＝lij(x,y)-lmeanTo obtainSingle shoe print relative pressure image Rij(x,y)。
Furthermore, the method for calculating the offset angle based on the centroid of the front foot and the centroid of the rear foot comprises the following steps: setting the deviation angle of the mass centers of the front foot and the rear foot extracted from a single relative pressure distribution map as theta, distributing the deviation angle on the sole and the heel according to the main pressure area of a person, and obtaining the following data by a mass center coordinate formula:
separately determine the sole centroid (plot _ x)sole,plot_ysole) And the center of mass of the heel (plot _ x)heel,plot_yheel) The inverse tangent value of the slope of the connecting line of the two centroids is the relative centroid deviation angle:
calculating the centroid offset angle theta of each footprint sequence through the centroid offset angles of the front foot and the rear foot (theta)1,θ2) Wherein theta1Is the offset angle theta of the front and rear foot mass centers of the left foot of the footprint sequence2And the front foot mass center offset angle and the rear foot mass center offset angle of the right foot of the footprint sequence are obtained.
Further, the footprint pressure energy map quadruple is constructed by: finding the minimum external rectangle for the obtained single relative pressure distribution image, and respectively obtaining the coordinate values (x) of the top left corners of the external rectangles of the two adjacent single footprints1,y1) And (x)2,y2) If x1>x2The first picture is the right foot, otherwise the second picture is the right foot.
Furthermore, the obtained single relative pressure image is spliced up and down according to the walking habit of peopleThe number of spliced single feet is 2, the formed reference is splicing with step length and splicing without step length, the precedence relations are left and right and left, thus the ith person can respectively form spliced images with step length with left foot as referenceLeft foot-based spliced image without step lengthSpliced image with step length taking right foot as referenceSpliced image without step length with right foot as referenceWherein i ∈ [1, N ]],j∈[1,N]Wherein N is the total number of people in the database, and M is the number of each type of feature map of each person.
Further, performing scale normalization on the spliced image, respectively traversing the image under each tuple, and finding the maximum size of the image under each tuple as the standard size S of the tupleQ1,SQ2,SQ3,SQ4And solving a circumscribed rectangle of the footprint area of each image, and normalizing the circumscribed rectangle to the standard size under the corresponding tuple by a zero padding method.
Further, the normalized images are respectively added and averaged to obtain a footprint pressure energy map, and a four-tuple of the footprint pressure energy map of each person is constructedWherein,represents the kth element footprint pressure energy map of the ith individual, and m represents the number of pictures per tuple for each individual.
Further, for the condition that the obtained footprint pressure energy map has low contrast, the image is enhanced through gamma conversionWherein,representing a footprint pressure energy map, gamma representing an enhancement factor; γ is obtained by training and is typically taken to be 1.3.
Still further, the five-tuple of the footprint is: obtaining quintuple expression of the footprint sequence according to the relative centroid deviation angle and the four-tuple splicing of the footprint pressure energy diagram
Further, during the online identification, the footprint identification process based on the footprint quintuple is as follows:
step S241: calculating the similarity of the centroid deviation angles of the front and rear feet; calculating Euclidean distance of the centroid offset angle of the single relative pressure footprint image identified on line and the centroid offset angle characteristic of the single relative pressure image set in the database:
wherein, theta01Representing an on-line identification of the front and rear foot centroid offset angle, θ, of a person's left footi1Representing the anterior-posterior foot centroid offset angle, θ, of the left foot of the ith individual in the database02Representing the on-line identification of the offset angle, theta, of the front and rear foot centroids of the right foot of a personi2Representing the anterior-posterior foot centroid offset angle of the ith individual's right foot in the database;
for fusion with the similarity score, normalizing the resulting distance to obtain a normalized similarity:
where k represents a weighting coefficient obtained by training, and is typically 0.06.
Through a measurement method of the front and back foot mass center offset angle, a similarity matrix D ' ═ D ' based on the front and back foot offset angle can be obtained 'i},i∈[1,N]。
Step S242: calculating the similarity of the four-tuple of the footprint pressure energy diagram; calculating similarity scores of the footmark pressure energy graph with the step length and taking the left foot as the reference to be identified and the footmark pressure energy graph with the step length and taking the left foot as the reference in the data set to obtain a similarity score S1＝{si},i∈[1,N]Wherein N represents the total number of people in the pool, siRepresenting the similarity score of the sample to be identified and the ith sample in the library;
similarly, calculating the similarity score S between the footmark pressure energy graph of the step-removing step length based on the left foot to be identified and the footmark pressure energy graph of the step-removing step length based on the left foot in the database2＝{si},i∈[1,N]Similarity score S between the footmark pressure distribution diagram of the step-removing based on the right foot and the footmark pressure energy diagram of the step-removing based on the right foot in the database3＝{si},i∈[1,N]Similarity score S between footprint pressure energy map with step length based on right foot and footprint pressure energy map with step length based on right foot in database4＝{si},i∈[1,N]；
Performing weighted fusion on the similarity scores of the four characteristics, and determining weighting coefficients through training, wherein the weighting coefficients are generally 0.3,0.3,0.2 and 0.2 to obtain the fused similarity score:
S＝0.3S1+0.3S2+0.2S3+0.2S4。
further, the similarity score is calculated according to the normalized cross-correlation coefficient of the footprint pressure energy map in the online identification stage and each sample footprint pressure energy map in the data set.
Step S243: correcting the similarity score of the footprint pressure energy diagram by using the similarity of the centroid offset angles of the front foot and the rear foot;
weighting by using the normalized distance of the centroid offset angle of the front foot and the rear foot, and then correcting the similarity score to obtain a final similarity score S ' ═ S + D ', wherein S represents the similarity score of the four-tuple of the footprint energy diagram after weighted fusion, and D ' represents the normalized similarity of the centroid offset angle of the front foot and the rear foot; and according to the final scoring sorting result, finding the label corresponding to the maximum similarity as the identification result.
Compared with the prior art, the invention has the following advantages:
(1) the invention fully considers the mass center offset angle of the sole and the heel of the pressure footprint sequence image; and the Euclidean distance is calculated through the deviation angles of the mass centers of the front foot and the rear foot of the left foot and the right foot, then the similarity of the footprint energy diagram is weighted, the error of the footprint pressure energy diagram is corrected, and a more stable similarity score is obtained.
(2) The invention fully considers that the minimum unit reflecting the walking habits of people and errors brought by splicing a plurality of footprints are superposed according to the increase of the number of the footprints, so that images formed by two feet are used for splicing to obtain a footprint pressure energy diagram with stability, and the allowable fluctuation range of each person during walking can be reflected by weighting and fusing the shoe prints at different time points;
(3) the invention fully considers the difference of information reflected by different splicing modes, forms four footprint pressure energy graphs containing different characteristics, namely a footprint pressure energy graph with step length based on the left foot, a foot-length-removing footprint pressure energy graph with step length based on the right foot and a foot-length-removing footprint pressure energy graph with step length based on the right foot, obtains matching scores through similarity calculation, and performs weighting fusion to ensure higher matching precision.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings needed to be used in the description of the embodiments or the prior art will be briefly introduced below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to these drawings without creative efforts.
FIG. 1 is a general flow diagram of the present invention.
FIG. 2 is a foot-to-foot footprint pressure energy map based on the right foot of the present invention.
FIG. 3 is a graph of foot pressure energy without step size based on the right foot of the present invention.
FIG. 4 is a plot of foot-to-foot pressure energy with step size based on the left foot of the present invention.
FIG. 5 is a graph of the foot pressure energy without step size based on the left foot of the present invention.
FIG. 6 is a schematic view of the offset angle of the centroid of the forefoot and hindfoot of the present invention.
Detailed Description
In order to make the technical solutions of the present invention better understood, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
It should be noted that the terms "first," "second," and the like in the description and claims of the present invention and in the drawings described above are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used is interchangeable under appropriate circumstances such that the embodiments of the invention described herein are capable of operation in sequences other than those illustrated or described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
As shown in fig. 1 to 6, the present invention includes a method for identifying sequences of footmarks of shoes worn based on pressure characteristics, comprising: an off-line training process and an on-line recognition process.
As a preferred embodiment, the off-line process comprises at least the following steps:
step S11: and extracting the relative pressure image of the single shoe print in the sequence set of the footprint images to be trained.
In this embodiment, the step of extracting the relative pressure image of the single shoe print comprises:
step S111: preprocessing an original footprint sequence; removing salt and pepper noise and plaque noise from the online recognized/to-be-trained footprint sequence image through median filtering, and enhancing the denoised image to obtain a footprint sequence image Fi(x, y) where i ∈ [1, N)]And N is the total number of people in the database.
Step S112: extracting the single shoe print image; and binarizing the preprocessed footprint sequence images, solving segmentation errors caused by shoe printing pattern fracture through closed operation of expansion and corrosion, calculating horizontal projection of the obtained images, and segmenting a single footprint by utilizing the interval between the footprint sequences. Marking the divided single footprints with connected domainsCalculating the number and the area of the connected domains, deleting the abnormal data by using the images of which the area and the number are larger than the threshold value to obtain the jth single shoe print image L of the ith individualij(x, y) where i ∈ [1, N)],j∈[1,M]Wherein N represents the total number of people in the database, and M represents the number of single shoe print images obtained by dividing each person;
step S113: calculating a relative pressure image of the single shoe print image extracted in step S112; scanning the single shoe print image line by line to find a pixel value l of a non-zero pointij(x, y), wherein x, y respectively represent abscissa and ordinate of single piece of shoe print image pixel, and calculate the mean value to the pixel value point that obtains:
making a difference l between the pixel value and the average value of each non-zero pointnew(x,y)＝lij(x,y)-lmeanObtaining the relative pressure image R of the single shoe printij(x,y)。
Further, as a preferred embodiment, step S12: and calculating the mass center offset angle I of the front foot and the rear foot according to the relative pressure image of the single shoe print extracted in the step S11.
The calculation method based on the centroid deviation angle of the front foot and the rear foot comprises the following steps:
setting the deviation angle of the mass centers of the front foot and the rear foot extracted from a single relative pressure distribution map as theta, distributing the deviation angle on the sole and the heel according to the main pressure area of a person, and obtaining the following data by a mass center coordinate formula:
separately determining the sole centroid (plot \ u)xsole,plot_ysole) And the center of mass of the heel (plot _ x)heel,plot_yheel) The inverse tangent value of the slope of the connecting line of the two centroids is the relative centroid deviation angle:
calculating the centroid offset angle theta of each footprint sequence through the centroid offset angles of the front foot and the rear foot (theta)1,θ2) Wherein theta1Is the offset angle theta of the front and rear foot mass centers of the left foot of the footprint sequence2And the front foot mass center offset angle and the rear foot mass center offset angle of the right foot of the footprint sequence are obtained.
In the present embodiment, step S13: and constructing a footprint pressure energy diagram quadruplet I of the footprint sequence set to be trained. As a preferred embodiment, the footprint pressure energy map quadruple is constructed by:
finding the minimum external rectangle for the obtained single relative pressure distribution image, and respectively obtaining the coordinate values (x) of the top left corners of the external rectangles of the two adjacent single footprints1,y1) And (x)2,y2) If x1>x2The first picture is the right foot, otherwise the second picture is the right foot;
the obtained single relative pressure image is spliced up and down, the number of spliced single feet is 2 according to the minimum unit of the walking habit of people and the principle of reducing error accumulation, the formed reference is splicing with the step length and splicing without the step length, the precedence relationship is left and right and left, and therefore the ith person can respectively form the spliced image with the step length with the left foot as the referenceLeft foot-based spliced image without step lengthSpliced image with step length taking right foot as referenceSpliced image without step length with right foot as referenceWherein i ∈ [1, N ]],j∈[1,N]Wherein N is the total number of people in the database, and M is the number of each type of characteristic graph of each person;
carrying out scale normalization on the spliced images, respectively traversing the images under each tuple, and finding the maximum size of the images under each tuple as the standard size S of the tupleQ1,SQ2,SQ3,SQ4And solving a circumscribed rectangle of the footprint area of each image, and normalizing the circumscribed rectangle to the standard size under the corresponding tuple by a zero padding method.
As a preferred embodiment, the normalized images are respectively added and averaged to obtain a footprint pressure energy map, and a four-tuple of the footprint pressure energy map of each person is constructedWherein,represents the kth element footprint pressure energy map of the ith individual, and m represents the number of pictures per tuple for each individual.
For the condition that the obtained footprint pressure energy map has low contrast, the image is enhanced through gamma conversionWherein,representing a footprint pressure energy map, gamma representing an enhancement factor; γ is obtained by training and is typically taken to be 1.3.
Step S14: the front and rear foot centroid deviation angle I extracted by the step S12 and the one constructed by the step S13And (4) constructing a footprint sequence quintuple feature database by the footprint energy quadruplet I. Obtaining quintuple expression of the footprint sequence according to the relative centroid deviation angle and the four-tuple splicing of the footprint pressure energy diagram
In this embodiment, the online identification process at least includes the following steps:
step S21: extracting a single shoe print relative pressure image of the online recognized footprint image sequence;
step S22: calculating a front and rear foot mass center offset angle II according to the single shoe print relative pressure image which is extracted in the step S21 and is identified on line;
step S23: constructing a footprint pressure energy graph quadruple II of the online recognized footprint sequence images;
step S24: according to the extracted front and rear foot centroid deviation angle II and pressure energy quadruple II groups identified on line, calculating similarity of the footmark sequence data to be identified and the footmark sequence data stored in the off-line training process respectively in a normalized cross-correlation measurement mode and a cosine distance measurement mode, fusing the calculated similarity scores, sorting the fused similarity scores from large to small, and taking the similarity score with the maximum similarity as the category of the footmarks to be retrieved.
In a preferred embodiment, in the online identification, the footprint identification process based on the footprint quintuple is as follows:
s241: calculating the similarity of the centroid deviation angles of the front and rear feet; calculating Euclidean distance of the centroid offset angle of the single relative pressure footprint image identified on line and the centroid offset angle characteristic of the single relative pressure image set in the database:
wherein, theta01Representing an on-line identification of the front and rear foot centroid offset angle, θ, of a person's left footi1Representing the anterior-posterior foot centroid offset angle, θ, of the left foot of the ith individual in the database02Representing the on-line identification of the offset angle, theta, of the front and rear foot centroids of the right foot of a personi2Representing the anterior-posterior foot centroid offset angle of the ith individual's right foot in the database;
for fusion with the similarity score, normalizing the resulting distance to obtain a normalized similarity:
where k represents a weighting coefficient obtained by training, typically 0.06; through a measurement method of the front and back foot mass center offset angle, a similarity matrix D ' ═ D ' based on the front and back foot offset angle can be obtained 'i},i∈[1,N]。
S242: calculating the similarity of the four-tuple of the footprint pressure energy diagram; calculating similarity scores of the footmark pressure energy graph with the step length and taking the left foot as the reference to be identified and the footmark pressure energy graph with the step length and taking the left foot as the reference in the data set to obtain a similarity score S1＝{si},i∈[1,N]Wherein N represents the total number of people in the pool, siRepresenting the similarity score of the sample to be identified and the ith sample in the library;
similarly, calculating the similarity score S between the footmark pressure energy graph of the step-removing step length based on the left foot to be identified and the footmark pressure energy graph of the step-removing step length based on the left foot in the database2＝{si},i∈[1,N]Similarity score S between the footmark pressure distribution diagram of the step-removing based on the right foot and the footmark pressure energy diagram of the step-removing based on the right foot in the database3＝{si},i∈[1,N]Similarity between footprint pressure energy map with step length based on right foot and footprint pressure energy map with step length based on right foot in databaseScore S4＝{si},i∈[1,N]；
Performing weighted fusion on the similarity scores of the four characteristics, and determining weighting coefficients through training, wherein the weighting coefficients are generally 0.3,0.3,0.2 and 0.2 to obtain the fused similarity score:
S＝0.3S1+0.3S2+0.2S3+0.2S4；
calculating the similarity score according to the normalized cross-correlation coefficient of the footprint pressure energy graph in the online identification stage and each sample footprint pressure energy graph in the data set;
s243: correcting the similarity score of the footprint pressure energy diagram by using the similarity of the centroid offset angles of the front foot and the rear foot;
weighting by using the normalized distance of the centroid offset angle of the front foot and the rear foot, and then correcting the similarity score to obtain a final similarity score S ' ═ S + D ', wherein S represents the similarity score of the four-tuple of the footprint energy diagram after weighted fusion, and D ' represents the normalized similarity of the centroid offset angle of the front foot and the rear foot; and according to the final scoring sorting result, finding the label corresponding to the maximum similarity as the identification result.
The above-mentioned serial numbers of the embodiments of the present invention are merely for description and do not represent the merits of the embodiments.
In the above embodiments of the present invention, the descriptions of the respective embodiments have respective emphasis, and for parts that are not described in detail in a certain embodiment, reference may be made to related descriptions of other embodiments.
In the embodiments provided in the present application, it should be understood that the disclosed technology can be implemented in other ways. The above-described embodiments of the apparatus are merely illustrative, and for example, the division of the unit may be a logical function division, and there may be another division manner in actual implementation.
Finally, it should be noted that: the above embodiments are only used to illustrate the technical solution of the present invention, and not to limit the same; while the invention has been described in detail and with reference to the foregoing embodiments, it will be understood by those skilled in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some or all of the technical features may be equivalently replaced; and the modifications or the substitutions do not make the essence of the corresponding technical solutions depart from the scope of the technical solutions of the embodiments of the present invention.
Claims (6)
1. A method for recognizing sequences of shoe wearing footprints based on pressure characteristics is characterized by comprising the following steps: an off-line training process and an on-line identification process;
the off-line process comprises at least the following steps:
s11: extracting a single shoe print relative pressure image in a centralized footprint image sequence to be trained;
s12: calculating a front foot mass center deviation angle I and a rear foot mass center deviation angle I according to the single shoe print relative pressure image extracted in the S11;
s13: constructing a footprint pressure energy diagram quadruplet I of the footprint sequence set to be trained;
s14: constructing a footprint sequence quintuple feature database by the front and rear foot mass center offset angle I extracted in the S12 and the footprint energy quadruple I constructed in the S13;
the online identification process comprises at least the following steps:
s21: extracting a single shoe print relative pressure image of the online recognized footprint image sequence;
s22: calculating a front and rear foot mass center offset angle II according to the single shoe print relative pressure image extracted in the S21 and identified on line;
s23: constructing a footprint pressure energy graph quadruple II of the online recognized footprint sequence images;
s24: according to the extracted front and rear foot centroid deviation angle II and pressure energy quadruple II groups identified on line, calculating similarity of the footmark sequence data to be identified and the footmark sequence data stored in the off-line training process respectively in a normalized cross-correlation measurement mode and a cosine distance measurement mode, fusing the calculated similarity scores, sorting the fused similarity scores from large to small, and taking the similarity score with the maximum similarity as the category of the footmarks to be retrieved.
2. The method of identifying sequences of shoe wearing footprints based on pressure characteristics as claimed in claim 1, further characterized by:
the single shoe print relative pressure image extraction step comprises the following steps:
s111: preprocessing an original footprint sequence; removing salt and pepper noise and plaque noise from the online recognized/to-be-trained footprint sequence image through median filtering, and enhancing the denoised image to obtain a footprint sequence image Fi(x, y) where i ∈ [1, N)]N is the total number of people in the database;
s112: extracting the single shoe print image; binarizing the preprocessed footprint sequence images, solving segmentation errors caused by shoe printing pattern fracture through closed operation of expansion and corrosion, calculating horizontal projection of the obtained images, and segmenting a single footprint by utilizing the interval between the footprint sequences;
marking the divided single footprints with connected domains and calculating the number and the area of the connected domains, deleting the images with the area and the number larger than the threshold value as abnormal data to obtain the jth single shoe print image L of the ith individualij(x, y) where i ∈ [1, N)],j∈[1,M]Wherein N represents the total number of people in the database, and M represents the number of single shoe print images obtained by dividing each person;
s113: calculating a relative pressure image of the single shoe print image extracted in step S112; scanning the single shoe print image line by line to find a pixel value l of a non-zero pointij(x, y), wherein x, y respectively represent abscissa and ordinate of single piece of shoe print image pixel, and calculate the mean value to the pixel value point that obtains:
making a difference l between the pixel value and the average value of each non-zero pointnew(x,y)＝lij(x,y)-lmeanObtaining the relative pressure image R of the single shoe printij(x,y)。
3. The method of identifying sequences of shoe wearing footprints based on pressure characteristics as claimed in claim 1, further characterized by: the calculation method based on the centroid deviation angle of the front foot and the rear foot comprises the following steps:
setting the deviation angle of the mass centers of the front foot and the rear foot extracted from a single relative pressure distribution map as theta, distributing the deviation angle on the sole and the heel according to the main pressure area of a person, and obtaining the following data by a mass center coordinate formula:
separately determine the sole centroid (plot _ x)sole,plot_ysole) And the center of mass of the heel (plot _ x)heel,plot_yheel)，The inverse tangent value of the slope of the two centroid connecting lines is the relative centroid deviation angle:
calculating the centroid offset angle theta of each footprint sequence through the centroid offset angles of the front foot and the rear foot (theta)1,θ2) Wherein theta1Is the offset angle theta of the front and rear foot mass centers of the left foot of the footprint sequence2And the front foot mass center offset angle and the rear foot mass center offset angle of the right foot of the footprint sequence are obtained.
4. The method of identifying sequences of shoe wearing footprints based on pressure characteristics as claimed in claim 1, further characterized by: the construction of the footprint pressure energy diagram quadruple is as follows:
finding the minimum external rectangle for the obtained single relative pressure distribution image, and respectively obtaining the coordinate values (x) of the top left corners of the external rectangles of the two adjacent single footprints1,y1) And (x)2,y2) If x1>x2The first picture is the right foot, otherwise the second picture is the right foot;
the obtained single relative pressure image is spliced up and down, the number of spliced single feet is 2 according to the minimum unit of the walking habit of people and the principle of reducing error accumulation, the formed reference is splicing with the step length and splicing without the step length, the precedence relationship is left and right and left, and therefore the ith person can respectively form the spliced image with the step length with the left foot as the referenceLeft foot-based spliced image without step lengthSpliced image with step length taking right foot as referenceAssembly without step length based on right footReceiving imagesWherein i ∈ [1, N ]],j∈[1,N]Wherein N is the total number of people in the database, and M is the number of each type of characteristic graph of each person;
carrying out scale normalization on the spliced images, respectively traversing the images under each tuple, and finding the maximum size of the images under each tuple as the standard size S of the tupleQ1,SQ2,SQ3,SQ4Calculating a circumscribed rectangle of a footprint area of each image, and normalizing the circumscribed rectangle to a standard size under a corresponding tuple through a zero padding method;
respectively adding the normalized images to obtain a footprint pressure energy map by averaging, and constructing the four-tuple of the footprint pressure energy map of each personWherein,a kth element footprint pressure energy map representing the ith individual, m representing the number of pictures per tuple for each individual;
for the condition that the obtained footprint pressure energy map has low contrast, the image is enhanced through gamma conversionWherein,representing a footprint pressure energy map, gamma representing an enhancement factor; γ is obtained by training and is typically taken to be 1.3.
5. The method of identifying sequences of shoe wearing footprints based on pressure characteristics as claimed in claim 1, further characterized by: the five-element group of the footprint is as follows:
according to the relative centroid deviation angle and the footprint pressure energy diagramQuintuple expression of tuple stitching to obtain footprint sequence
6. The method of identifying sequences of shoe wearing footprints based on pressure characteristics as claimed in claim 1, further characterized by:
during the online identification, the footprint identification process based on the footprint quintuple comprises the following steps:
s241: calculating the similarity of the centroid deviation angles of the front and rear feet; calculating Euclidean distance of the centroid offset angle of the single relative pressure footprint image identified on line and the centroid offset angle characteristic of the single relative pressure image set in the database:
wherein, theta01Representing an on-line identification of the front and rear foot centroid offset angle, θ, of a person's left footi1Representing the anterior-posterior foot centroid offset angle, θ, of the left foot of the ith individual in the database02Representing the on-line identification of the offset angle, theta, of the front and rear foot centroids of the right foot of a personi2Representing the anterior-posterior foot centroid offset angle of the ith individual's right foot in the database;
for fusion with the similarity score, normalizing the resulting distance to obtain a normalized similarity:
where k represents a weighting coefficient obtained by training, typically 0.06; through a measurement method of the front and back foot mass center offset angle, a similarity matrix D ' ═ D ' based on the front and back foot offset angle can be obtained 'i},i∈[1,N]。
S242: calculating the similarity of the four-tuple of the footprint pressure energy diagram; left foot referenced footprint pressure energy map to be identified and left foot referenced footprint pressure in data setCalculating similarity score by using the force energy diagram to obtain similarity score S1＝{si},i∈[1,N]Wherein N represents the total number of people in the pool, siRepresenting the similarity score of the sample to be identified and the ith sample in the library;
similarly, calculating the similarity score S between the footmark pressure energy graph of the step-removing step length based on the left foot to be identified and the footmark pressure energy graph of the step-removing step length based on the left foot in the database2＝{si},i∈[1,N]Similarity score S between the footmark pressure distribution diagram of the step-removing based on the right foot and the footmark pressure energy diagram of the step-removing based on the right foot in the database3＝{si},i∈[1,N]Similarity score S between footprint pressure energy map with step length based on right foot and footprint pressure energy map with step length based on right foot in database4＝{si},i∈[1,N]；
Performing weighted fusion on the similarity scores of the four characteristics, and determining weighting coefficients through training, wherein the weighting coefficients are generally 0.3,0.3,0.2 and 0.2 to obtain the fused similarity score:
S＝0.3S1+0.3S2+0.2S3+0.2S4；
calculating the similarity score according to the normalized cross-correlation coefficient of the footprint pressure energy graph in the online identification stage and each sample footprint pressure energy graph in the data set;
s243: correcting the similarity score of the footprint pressure energy diagram by using the similarity of the centroid offset angles of the front foot and the rear foot;
weighting by using the normalized distance of the centroid offset angle of the front foot and the rear foot, and then correcting the similarity score to obtain a final similarity score S ' ═ S + D ', wherein S represents the similarity score of the four-tuple of the footprint energy diagram after weighted fusion, and D ' represents the normalized similarity of the centroid offset angle of the front foot and the rear foot; and according to the final scoring sorting result, finding the label corresponding to the maximum similarity as the identification result.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910464874.5A CN110188694B (en) | 2019-05-30 | 2019-05-30 | Method for identifying shoe wearing footprint sequence based on pressure characteristics |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910464874.5A CN110188694B (en) | 2019-05-30 | 2019-05-30 | Method for identifying shoe wearing footprint sequence based on pressure characteristics |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110188694A true CN110188694A (en) | 2019-08-30 |
| CN110188694B CN110188694B (en) | 2022-10-21 |
Family
ID=67719039
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910464874.5A Active CN110188694B (en) | 2019-05-30 | 2019-05-30 | Method for identifying shoe wearing footprint sequence based on pressure characteristics |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110188694B (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110781852A (en) * | 2019-10-31 | 2020-02-11 | 大连海事大学 | Shoe wearing sequence footprint identification method based on multi-resolution feature fusion |
| CN111402202A (en) * | 2020-02-24 | 2020-07-10 | 安徽大学 | Similarity evaluation method for plantar pressure images |
| CN112102362A (en) * | 2020-09-14 | 2020-12-18 | 北京数衍科技有限公司 | Pedestrian step track determination method and device |
| CN112633338A (en) * | 2020-12-14 | 2021-04-09 | 辽宁师范大学 | Suspect gender prediction method based on shoe imprints |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2015084868A (en) * | 2013-10-29 | 2015-05-07 | 花王株式会社 | Walking feature display method |
| CN106887019A (en) * | 2017-02-23 | 2017-06-23 | 大连海事大学 | A kind of footprint Pressure Distribution method for expressing |
| CN109325546A (en) * | 2018-10-19 | 2019-02-12 | 大连海事大学 | A kind of combination footwork feature at time footprint recognition method |
-
2019
- 2019-05-30 CN CN201910464874.5A patent/CN110188694B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2015084868A (en) * | 2013-10-29 | 2015-05-07 | 花王株式会社 | Walking feature display method |
| CN106887019A (en) * | 2017-02-23 | 2017-06-23 | 大连海事大学 | A kind of footprint Pressure Distribution method for expressing |
| CN109325546A (en) * | 2018-10-19 | 2019-02-12 | 大连海事大学 | A kind of combination footwork feature at time footprint recognition method |
Non-Patent Citations (1)
| Title |
|---|
| 管燕等: "基于稳定特征的鞋印图像识别方法研究", 《计算机工程与应用》 * |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110781852A (en) * | 2019-10-31 | 2020-02-11 | 大连海事大学 | Shoe wearing sequence footprint identification method based on multi-resolution feature fusion |
| CN110781852B (en) * | 2019-10-31 | 2023-10-27 | 大连海事大学 | Footwear sequence footprint identification method based on multi-resolution feature fusion |
| CN111402202A (en) * | 2020-02-24 | 2020-07-10 | 安徽大学 | Similarity evaluation method for plantar pressure images |
| CN112102362A (en) * | 2020-09-14 | 2020-12-18 | 北京数衍科技有限公司 | Pedestrian step track determination method and device |
| CN112633338A (en) * | 2020-12-14 | 2021-04-09 | 辽宁师范大学 | Suspect gender prediction method based on shoe imprints |
| CN112633338B (en) * | 2020-12-14 | 2024-07-12 | 辽宁师范大学 | Suspected person gender prediction method based on shoe mark |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110188694B (en) | 2022-10-21 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110188694B (en) | Method for identifying shoe wearing footprint sequence based on pressure characteristics | |
| CN108038476B (en) | A kind of facial expression recognition feature extracting method based on edge detection and SIFT | |
| CN105574505B (en) | The method and system that human body target identifies again between a kind of multiple-camera | |
| CN103390164B (en) | Method for checking object based on depth image and its realize device | |
| CN109325546B (en) | Step-by-step footprint identification method combining features of step method | |
| CN109583324A (en) | A kind of pointer meters reading automatic identifying method based on the more box detectors of single-point | |
| CN106529559A (en) | Pointer-type circular multi-dashboard real-time reading identification method | |
| CN112949572A (en) | Slim-YOLOv 3-based mask wearing condition detection method | |
| EP2733666B1 (en) | Method for superpixel life cycle management | |
| Blomley et al. | Classification of airborne laser scanning data using geometric multi-scale features and different neighbourhood types | |
| CN109800624A (en) | A kind of multi-object tracking method identified again based on pedestrian | |
| CN103415871B (en) | Irradiation field identifies | |
| CN110472081B (en) | Shoe picture cross-domain retrieval method based on metric learning | |
| CN105160303A (en) | Fingerprint identification method based on mixed matching | |
| CN105760472A (en) | Video retrieval method and system | |
| JP3809305B2 (en) | Image search apparatus, image search method, and computer-readable storage medium | |
| CN115272652A (en) | Dense object image detection method based on multiple regression and adaptive focus loss | |
| Peng et al. | Evaluation of segmentation quality via adaptive composition of reference segmentations | |
| CN110443261A (en) | A kind of more figure matching process restored based on low-rank tensor | |
| Yan et al. | Adaptive fusion of color and spatial features for noise-robust retrieval of colored logo and trademark images | |
| CN103854278A (en) | Printed circuit board image registration method based on shape context of mass center of communicated region | |
| CN108182705A (en) | A kind of three-dimensional coordinate localization method based on machine vision | |
| CN105469099B (en) | Pavement crack detection and identification method based on sparse representation classification | |
| CN109166093B (en) | Image salient region detection method | |
| CN110147749A (en) | A kind of civil aviaton's missing baggage auxiliary method for retrieving |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |