CN112861878A - Abnormal matching identification method based on structural offset characteristics - Google Patents
Abnormal matching identification method based on structural offset characteristics Download PDFInfo
- Publication number
- CN112861878A CN112861878A CN202110163367.5A CN202110163367A CN112861878A CN 112861878 A CN112861878 A CN 112861878A CN 202110163367 A CN202110163367 A CN 202110163367A CN 112861878 A CN112861878 A CN 112861878A
- Authority
- CN
- China
- Prior art keywords
- matching
- sequence
- image
- calculating
- graph
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
- G06V10/443—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components by matching or filtering
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/22—Matching criteria, e.g. proximity measures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
- G06F18/2415—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on parametric or probabilistic models, e.g. based on likelihood ratio or false acceptance rate versus a false rejection rate
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/42—Global feature extraction by analysis of the whole pattern, e.g. using frequency domain transformations or autocorrelation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/46—Descriptors for shape, contour or point-related descriptors, e.g. scale invariant feature transform [SIFT] or bags of words [BoW]; Salient regional features
- G06V10/462—Salient features, e.g. scale invariant feature transforms [SIFT]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
- G06V10/75—Organisation of the matching processes, e.g. simultaneous or sequential comparisons of image or video features; Coarse-fine approaches, e.g. multi-scale approaches; using context analysis; Selection of dictionaries
- G06V10/757—Matching configurations of points or features
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Multimedia (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Computation (AREA)
- General Engineering & Computer Science (AREA)
- Evolutionary Biology (AREA)
- Bioinformatics & Computational Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Probability & Statistics with Applications (AREA)
- Health & Medical Sciences (AREA)
- Computing Systems (AREA)
- Databases & Information Systems (AREA)
- General Health & Medical Sciences (AREA)
- Medical Informatics (AREA)
- Software Systems (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a method for carrying out mismatching rejection by applying structural offset characteristics, which belongs to the technical field of computer vision, and comprises the following steps of firstly, obtaining an initial matching set between two images by using a characteristic matching algorithm; secondly, selecting neighbor matching for each matching, extracting a local transformation matrix for each key point in order to reduce the probability that the traditional k neighbor contains error matching, and screening each pair of effective neighbors for initial matching according to the local transformation matrix; and finally, calculating the difference of the bit order of the effective neighbor matching on the two images relative to the center matching, and calling the obtained vector as the structural offset characteristic. The features provided by the invention are simple and intuitive, training and testing can be performed on various typical classifiers, and experiments prove that the error matching can be effectively eliminated by the provided method.
Description
Technical Field
The invention belongs to the technical field of computer vision, and particularly relates to an anomaly matching identification method based on structural offset characteristics.
Background
Image feature matching is a technique used to correlate the position of the same three-dimensional point in space on different images. The technology is generally applied to applications such as motion recovery structures, Simultaneous Localization And Mapping (SLAM), panoramic stitching, stereo matching And the like, And is a basic technology of many application directions in the field of computer vision.
The existing image feature matching technology generally comprises two steps: firstly, extracting key points on an image and describing the key points to obtain feature descriptors of the key points; then, similarity between the measurement descriptors (generally, Euclidean distance between the measurement descriptors) is measured, and the key points with high similarity are a pair of matching points. Although many of the existing methods have yielded good results, image feature matching remains a challenging problem: on one hand, the prior art mainly describes the local information of the key points, so that ambiguity or even error occurs when the matching of the key points is judged by means of local information descriptors in a scene with a repetitive structure; on the other hand, rotation, illumination change, viewpoint change, occlusion and other challenges of the image itself can also cause the generation of a mismatch. The introduction of these false matches can in turn affect the effectiveness of subsequent applications. Therefore, there is a need for an algorithm that can identify and reject anomalous false matches to ensure accuracy in subsequent applications.
There are many types of methods used for anomaly matching identification in recent years. One popular approach is to evaluate the global transforms of all matches to reject false matches that do not satisfy this transform model. However, this method has a great influence on the correctness of the result in the case that the matching set has a small correct matching ratio. Furthermore, this way of estimating the global transformation is not applicable to scenarios where multiple local consistency transformations and non-rigid transformations. Another method uses local discrimination to select the neighborhood around each matching key point (the nearest neighbor is generally used to represent the neighborhood), and compares the error of a certain structure quantization index defined by the key points at the two ends of the matching on the neighborhood to identify abnormal matching. The method is easily interfered by mismatching in a local neighborhood, and a distinguishing threshold value of abnormal matching needs to be preset, so that the method is difficult to adapt to different scenes. Yet another class of approaches employs learned strategies to identify anomalous matches. Some employ a deep learning framework to achieve classification tasks of correct matching and incorrect matching, but require a large amount of data (including information such as camera internal parameters) in the training process, which limits the usability of the method to some extent. Other methods extract matching features by means of artificially designed criteria and use these features for classification. However, the existing method describes the matched features from the aspects of similarity, direction, angle and the like of the spatial structure, the method is too complex and high in calculation difficulty, and the influence of mismatching in the neighborhood structure cannot be relieved.
Disclosure of Invention
Aiming at the defects or improvement requirements of the prior art, the invention provides an abnormal matching identification method based on the structural offset characteristics, a learning method is adopted, matched characteristics are extracted by using matched local structural information, the extracted characteristics have the characteristics of simple calculation and accurate classification, and therefore, an effective classifier can be trained for abnormal matching identification under the condition of only a small number of samples, so that the reliability of abnormal matching identification is greatly improved, and effective input is provided for later practical application.
In order to achieve the above object, the present invention provides an anomaly matching identification method based on structural offset features, which includes:
(1) respectively extracting key points from the image pairs in the training image set, obtaining an initial matching set c, and simultaneously calculating a local transformation matrix A corresponding to each key point;
(2) calculating the geometric characteristic identification matrix of each match for the initial matching set c obtained from the single image pair, wherein each match can obtain a local geometric characteristic identification matrix H taking the left image as a referencelLocal geometric feature identification matrix H with reference to right figurerCalculating similarity values S between matches by using the geometric feature identification matrix, and calculating similarity values S with the left graph as reference when calculating the similarity valueslSimilarity value S with reference to the right figurer;
(3) Determining a neighborhood range k, matching each center, and selecting a similarity value SlThe largest k matches generate a valid neighbor sequence N referenced to the leftlt}t=1,2,...kSelecting a similarity value SrThe largest k matches are generated as the right graphValid neighbor sequence for reference Nrt}t=1,2,...kRespectively calculating the offset of the distance sequence of the effective neighbor sequence taking the left image as the reference and the effective neighbor sequence taking the right image as the reference relative to the center matching, and generating the matched left image characteristic flAnd right graph feature frSplicing the matched left image characteristic and the right image characteristic to obtain a matched final characteristic [ fl,fr];
(4) Combining all matched structural deviation features in a training image set into a training sample, manually marking labels of the features of the training sample, preprocessing the features, and then using the preprocessed features as input of a classifier, training the classifier, obtaining feature processing parameters and a final classifier, and facilitating anomaly identification through the trained classifier.
In some alternative embodiments, step (1) comprises:
(1.1) carrying out key point detection on any pair of image pairs in the training image set to obtain M key points of a left image and N key points of a right image;
(1.2) carrying out initial matching to obtain n pairs of matching pairs corresponding to key points { k ] of the left imagei}i=1,2,...nAnd corresponding local transformation matrix { A }i}i=1,2,...nSimultaneously obtaining key points k of the right imagei'}i=1,2,…nAnd corresponding local transformation matrix { A }i'}i=1,2,…nWhen the matching relationship is { c }i=(ki,ki')}i=1,2,…n。
In some alternative embodiments, step (2) comprises:
(2.1) for each key point at the two matched ends, obtaining a local structure matrix T taking the left image as a reference for the key point in the left image, and obtaining a local structure matrix T' taking the right image as a reference for the key point in the right image;
(2.2) calculating a local geometric feature identification matrix H taking the left figure as reference according to the local structure matrix T taking the left figure as referencelCalculating local geometric feature identification matrix H with reference to the right figure according to local structure matrix T' with reference to the right figurer;
(2.3) identifying matrix H based on local geometric features with reference to left figurelCalculating a left graph-based similarity value SlIdentification matrix H based on local geometric features with reference to right figurerCalculating a similarity value S based on the right graphr。
In some of the alternative embodiments, the first and second,wherein the content of the first and second substances,the lambda is a mapping parameter which is,sigma (-) means to calculate the sum of the absolute values of all the elements inside, ρ is the calculation method to convert the homogeneous coordinate into the inhomogeneous coordinate, the indices i and j are used to distinguish between different objects,Hl={Hl1 Hl2 … Hln},Hr={Hr1 Hr2 … Hrn},Hli=Ti'Ti -1,i=1,2,…n,Hri=TiTi'-1,i=1,2,…n。
in some alternative embodiments, step (3) comprises:
(3.1) matching for one ci={ki,ki' } based on the valid neighbor sequence of the left graph Nlt}t=1,2,…kThe corresponding match represented is ctiK, with the left image key points k, respectivelytiK and a right graph key point k'ti,t=1,2,...k;
(3.2) calculating key points k of the left graph of the effective neighbor sequencetiK is relative to the left image center key point kiThe distance sequence is arranged according to the sequence of each effective neighbor in the effective neighbor sequence to obtain a left image neighborhood structure sequence Nfl;
(3.3) calculating key point k 'of right graph of effective neighbor sequence'tiK is relative to the right image center key point ki' the distance is sequenced, and the sequence N of the neighborhood structure of the right picture is obtained according to the sequence of each effective neighbor in the effective neighbor sequencefr;
(3.4) from fl=|Nfl-NfrI calculating left image neighborhood structure sequence NflAnd a right picture neighborhood structure sequence NfrOffset amount f oflShifting features for the left graph-based structure, where | represents taking the absolute value of the internal element;
(3.5) matching for onei={ki,ki' }, obtaining an effective neighbor sequence based on the right image and corresponding match expressed by the effective neighbor sequence, calculating the distance sequence of the key point of the right image of the effective neighbor sequence relative to the key point of the center of the right image, and arranging according to the sequence of each effective neighbor in the effective neighbor sequence to obtain a neighborhood structure sequence of the right image; calculating the distance sequence of the key points of the left graph of the effective neighbor sequence relative to the key points of the center of the left graph, and arranging according to the sequence of each effective neighbor in the effective neighbor sequence to obtain a neighborhood structure sequence of the left graph; calculating the offset of the neighborhood structure sequence of the right image and the neighborhood structure sequence of the left image into a structure offset characteristic f based on the right imager。
In some alternative embodiments, step (4) comprises:
(4.1) combining all matched structural offset features in the training image set into a training sample, and manually marking labels of the features of the training sample, wherein the labels are a positive sample and a negative sample, and the matching is represented as a correct matching by 1 and is the positive sample; denote this match as a false match with 0, which is a negative sample;
(4.2) carrying out normalization preprocessing on the structure deviation characteristics of the training samples: taking each sample as a row, taking the structure deviation characteristic of each dimension as a column, normalizing each column of all samples to a [0,1] interval, and storing the minimum value and the maximum value of the samples used in the normalization process as normalization parameters so as to normalize a new sample in the same way after the new sample arrives;
and (4.3) inputting the training samples with the manual labeling labels into a trainer, and training the trainer to obtain a trained classifier.
In some optional embodiments, the performing the anomaly identification through the trained classifier includes:
respectively executing the step (1) and the step (2) to the image pair needing abnormal matching identification to obtain a similarity value S based on the left image of each matchlSimilarity value S with right graphr;
Executing the step (3) to obtain the characteristics of the image, processing the structural offset characteristics by applying the normalization parameters obtained in the step (4), predicting the probability P that each match is a correct match by using the classifier obtained in the step (4), determining the weighting times q of multiple rounds, and outputting the final prediction probability P' if the weighting times reach q times;
and dividing the prediction probability P' according to a given threshold value t, wherein the probability greater than t is correct matching, the probability less than t is wrong matching, and a final correct matching set is output.
In some alternative embodiments, the final prediction probability P' is obtained by:
(a) using the prediction probability Pq-1Updating the similarity value S with reference to the left figurelSimilarity value S with reference to the right figurerObtaining an updated similarity valueAndwherein, Pq-1={p1 p2 … pn}, Wherein p isiRepresenting the probability that the ith pair of feature matches is predicted to be a correct match, pjRepresenting the probability that the jth pair of feature matches is predicted to be a correct match;
(c) With updated characteristicsFor input, a trained classifier is applied to predict the probability P of each match being a positive sampleq;
(d) And (c) repeatedly executing the steps (a) to (c) until the iteration number reaches a preset threshold value q, and outputting a final prediction probability P'.
In general, compared with the prior art, the above technical solution contemplated by the present invention can achieve the following beneficial effects:
(1) an abnormal matching identification method based on structural offset features is provided, and the method utilizes the offset of selecting effective neighbors and comparing the sequence order of the effective neighbors as matched features and uses the features to train out a usable classifier.
(2) In application, the adopted multi-round iteration strategy can further improve the reliability of the neighbors and the distribution of the prediction probability, reduce the influence of abnormal matching on the final characteristics and obtain more effective results.
(3) The structural offset feature has the advantages of simple and rapid calculation method when extracting the feature, and the obtained feature can be effectively applied to abnormal matching identification.
Drawings
Fig. 1 is a schematic flowchart of an anomaly matching identification method based on structural deviation features according to an embodiment of the present invention;
FIG. 2 is a schematic flow chart of another method for identifying an abnormal matching based on a structural deviation feature according to an embodiment of the present invention;
FIG. 3 is a training flow diagram provided by an embodiment of the present invention;
fig. 4 is a flowchart of an application test according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention. In addition, the technical features involved in the embodiments of the present invention described below may be combined with each other as long as they do not conflict with each other.
The invention provides an abnormal matching identification method based on structural offset characteristics, which comprises the steps of firstly, detecting key points on an image, obtaining a local transformation matrix corresponding to the key points, and obtaining an initial matching set by applying a matching method; secondly, a method for selecting effective neighbors based on space structure common potential is provided, each match comprises two key points which are respectively positioned on a left graph and a right graph, so that two effective neighbor sequences can be obtained for each match, wherein the effective neighbor sequences are obtained by using a similarity value calculated by taking the left graph as reference and a similarity value calculated by taking the right graph as reference; then, the relative bit order difference between the valid neighbor and the center match is computed as a feature, called a structure-shifted feature. In the training part, labels matched with the samples are labeled manually, and a proper machine learning method is selected for training, so that an effective classifier can be obtained under the condition of only a small number of samples. In the testing part, although the method of selecting the effective neighbors based on the space structure co-potential can avoid the false matching mixed in the effective neighbors under certain conditions, the influence of the false matching cannot be completely eliminated. Based on this, a multi-round weighting strategy is adopted in the test part: the classifier is used to predict the correct probability of a match and this probability is used to update the criteria for selecting valid neighbors, making the selected valid neighbors progressively cleaner. The finally obtained result is superior to the current mainstream method in the comprehensive performance of precision and recall rate.
As shown in fig. 1 and fig. 2, a flow chart of a mismatch elimination method based on structural offset features is shown, which includes the following steps:
s1: respectively extracting key points from the image pairs in the training image set, obtaining an initial matching set c, and simultaneously calculating a local transformation matrix A corresponding to each key point;
in the embodiment of the present invention, step S1 may be implemented as follows:
s1.1: performing key point detection on any pair of images in the training image set to obtain M key points of a left image and N key points of a right image;
wherein, the image key points can adopt hessian operator to extract the features.
The image Feature description may be characterized by using Scale Invariant Feature Transform (SIFT).
S1.2: performing initial matching to obtain n pairs of matching pairs corresponding to key points { k ] of the left imagei}i=1,2,…nAnd corresponding local transformation matrix { A }i}i=1,2,…nSimultaneously obtaining key points k of the right imagei'}i=1,2,...nAnd corresponding local transformation matrix { A }i'}i=1,2,...nWhen the matching relationship is { c }i=(ki,ki')}i=1,2,...n。
Wherein, the local transformation matrix may adopt a local affine transformation matrix.
S2: calculating the geometric characteristic identification matrix of each match for the initial matching set c obtained from the single image pair, wherein each match can obtain a local geometric characteristic identification matrix H taking the left image as a referencelTo the rightLocal geometric feature identification matrix H for referencerCalculating similarity values S between matches by using the geometric feature identification matrix, and calculating similarity values S with the left graph as reference when calculating the similarity valueslSimilarity value S with reference to the right figurer;
In the embodiment of the present invention, step S2 may be implemented as follows:
s2.1: for each key point at the two matched ends, local structure matrixes T and T' taking the corresponding image as reference are obtained;
wherein T and T' are specifically combined as shown in formula (1):
s2.2: respectively calculating local geometric feature identification matrixes H with left images as referenceslLocal geometric feature identification matrix H with reference to right figurerThe local structure matrix is composed of an affine transformation matrix and the position information of the key points, so that the local geometric feature identification matrix is called a local homography matrix;
wherein HlAnd HrSee formula (2), and concretely, a calculation formula is see formula (3);
Hl={Hl1 Hl2 ... Hln}Hr={Hr1 Hr2 ... Hrn} (2)
Hli=Ti'Ti -1,i=1,2,...n Hri=TiTi'-1,i=1,2,...n (3)
s2.3: calculating a left graph-based similarity value SlSimilarity value S with right graphr,SlAnd SrSee formula (4), and concretely, a formula is calculated see formula (5);
wherein λ is a mapping parameter, in particular elijAnd erijSee formula (6):
wherein, the meaning of σ (-) is to calculate the sum of absolute values of all internal elements, ρ is a calculation method for converting homogeneous coordinates into inhomogeneous coordinates, and the specific calculation method is shown in formula (7):
ρ([a b c])=[a/c b/c]T (7)
where the indices i and j are used to distinguish between different objects.
S3: determining a neighborhood range k, matching each center, and selecting a similarity value SlThe largest k matches generate a valid neighbor sequence N referenced to the leftlt}t=1,2,...kAnd a similarity value SrThe largest k matches generate a valid neighbor sequence N referenced to the right graphrt}t=1,2,...k(ii) a Respectively calculating the offset of the distance sequence of the effective neighbor sequence taking the left image as the reference and the effective neighbor sequence taking the right image as the reference relative to the center matching, and generating the matched left image characteristic flAnd right graph feature frSplicing the matched left image characteristic and the right image characteristic to obtain a matched final characteristic [ fl,fr];
The size of the neighborhood range k can be determined according to actual needs, and the embodiment of the invention is not limited uniquely.
In the embodiment of the present invention, step S3 may be implemented as follows:
s3.1: for one match ci={ki,ki' } based on the valid neighbor sequence of the left graph Nlt}t=1,2,...kThe corresponding match represented is ctiK, with the left image key points k, respectivelytiK and right graph key point kt'i,t=1,2,...k;
S3.2: calculating key point k of effective neighbor sequence left graphtiK is relative to the left image center key point kiThe distance sequence is arranged according to the sequence of each effective neighbor in the effective neighbor sequence to obtain a left image neighborhood structure sequence Nfl;
S3.3: calculating key point k 'of right graph of effective neighbor sequence'tiK is relative to the right image center key point ki' the distance is sequenced, and the sequence N of the neighborhood structure of the right picture is obtained according to the sequence of each effective neighbor in the effective neighbor sequencefr;
S3.4: calculating a left image neighborhood structure sequence NflAnd a right picture neighborhood structure sequence NfrOffset amount f oflCalculating a formula specifically shown in the formula (8) for the structural deviation characteristics based on the left graph;
fl=|Nfl-Nfr| (8)
wherein, | · | represents taking the absolute value of the internal element;
s3.5: for one match ci={ki,ki' }, obtaining an effective neighbor sequence based on the right image and corresponding match expressed by the effective neighbor sequence, calculating the distance sequence of the key point of the right image of the effective neighbor sequence relative to the key point of the center of the right image, and arranging according to the sequence of each effective neighbor in the effective neighbor sequence to obtain a neighborhood structure sequence of the right image; calculating the distance sequence of the key points of the left graph of the effective neighbor sequence relative to the key points of the center of the left graph, and arranging according to the sequence of each effective neighbor in the effective neighbor sequence to obtain a neighborhood structure sequence of the left graph; calculating the offset of the neighborhood structure sequence of the right image and the neighborhood structure sequence of the left image into a structure offset characteristic f based on the right imager。
S4: as shown in fig. 3, all matched structural deviation features in the training image set are combined into a training sample, and labels of the features of the training sample are manually labeled, where the labels are positive samples and negative samples: 1 means that this match is a correct match, a positive sample; 0 means that the match is a false match, is a negative sample, and then the feature is normalized: taking each sample as a row, taking the characteristic of each dimension as a column, normalizing each column of all samples to a [0,1] interval, storing the minimum value and the maximum value of the sample used in the normalization process as normalization parameters, so as to normalize the new sample in the same way after the new sample arrives, selecting a proper training method for training, inputting the training sample with the manual labeling label into a trainer, and obtaining the trained classifier.
Wherein, random forests can be selected as training classifiers.
In an embodiment of the present invention, the performing abnormality identification by using the trained classifier includes:
respectively executing the step S1 and the step S2 on the image pair needing abnormal matching identification, and obtaining a similarity value S based on the left image of each matchlSimilarity value S with right graphr;
Executing step S3, obtaining the characteristics of the image, processing the characteristics by applying the normalization parameters obtained in step S4, predicting the probability P that each match is correct by using the classifier obtained in step S4, determining the weighting times q of multiple rounds, and outputting the final prediction probability P' if the weighting times reaches q times;
dividing the prediction probability P' according to a given threshold value t, wherein the probability greater than t is correct matching, and the probability less than t is false matching; the final set of correct matches is output.
The size of the given threshold t may be determined according to actual needs, and the embodiment of the present invention is not limited uniquely.
In the embodiment of the present invention, as shown in fig. 4, the final prediction probability P' may be obtained by:
(a) using the prediction probability Pq-1The similarity value S calculated in step S2 with reference to the left figure is updatedlSimilarity value S with reference to the right figurerObtaining an updated similarity valueAndwherein, Pq-1See formula (9), piIndicating the probability that the ith pair of feature matches is predicted to be a correct match,andthe similarity value is updated in the formula (10)Andsee formula (11):
Pq-1={p1 p2 ... pn} (9)
(b) using updated similarity valuesAndrepeatedly executing the step S3 to obtain the updated characteristics
(c) With updated characteristicsFor input, the classifier obtained in step S4 is applied to predict the probability P of each match being a positive sampleq;
(d) And (c) repeatedly executing the steps (a) to (c) until the iteration number reaches a preset threshold value q, and outputting a final prediction probability P'.
It should be noted that, according to the implementation requirement, each step/component described in the present application can be divided into more steps/components, and two or more steps/components or partial operations of the steps/components can be combined into new steps/components to achieve the purpose of the present invention.
It will be understood by those skilled in the art that the foregoing is only a preferred embodiment of the present invention, and is not intended to limit the invention, and that any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the scope of the present invention.
Claims (8)
1. An abnormal matching identification method based on structural offset features is characterized by comprising the following steps:
(1) respectively extracting key points from the image pairs in the training image set, obtaining an initial matching set c, and simultaneously calculating a local transformation matrix A corresponding to each key point;
(2) calculating the geometric characteristic identification matrix of each match for the initial matching set c obtained from the single image pair, wherein each match can obtain a local geometric characteristic identification matrix H taking the left image as a referencelLocal geometric feature identification matrix H with reference to right figurerCalculating similarity values S between matches by using the geometric feature identification matrix, and calculating similarity values S with the left graph as reference when calculating the similarity valueslSimilarity value S with reference to the right figurer;
(3) Determining a neighborhood range k, matching each center, and selecting a similarity value SlThe largest k matches generate a valid neighbor sequence N referenced to the leftlt}t=1,2,...kSelecting a similarity value SrThe largest k matches generate a valid neighbor sequence N referenced to the right graphrt}t=1,2,...kRespectively calculating the offset of the distance sequence of the effective neighbor sequence taking the left image as the reference and the effective neighbor sequence taking the right image as the reference relative to the center matching, and generating the matched left image characteristic flAnd right graph feature frSplicing the matched left image characteristic and the right image characteristic to obtain a matched final characteristic [ fl,fr];
(4) Combining all matched structural deviation features in a training image set into a training sample, manually marking labels of the features of the training sample, preprocessing the features, and then using the preprocessed features as input of a classifier, training the classifier, obtaining feature processing parameters and a final classifier, and facilitating anomaly identification through the trained classifier.
2. The anomaly matching identification method according to claim 1, wherein step (1) comprises:
(1.1) carrying out key point detection on any pair of image pairs in the training image set to obtain M key points of a left image and N key points of a right image;
(1.2) carrying out initial matching to obtain n pairs of matching pairs corresponding to key points { k ] of the left imagei}i=1,2,...nAnd corresponding local transformation matrix { A }i}i=1,2,...nWhile obtaining key points { k 'of the right graph'i}i=1,2,...nAnd corresponding local transformation matrix { A'i}i=1,2,...nWhen the matching relationship is { c }i=(ki,k′i)}i=1,2,...n。
3. The anomaly matching identification method according to claim 2, wherein the step (2) comprises:
(2.1) for each key point at the two matched ends, obtaining a local structure matrix T taking the left image as a reference for the key point in the left image, and obtaining a local structure matrix T' taking the right image as a reference for the key point in the right image;
(2.2) calculating a local geometric feature identification matrix H taking the left figure as reference according to the local structure matrix T taking the left figure as referencelCalculating local geometric feature identification matrix H with reference to the right figure according to local structure matrix T' with reference to the right figurer;
(2.3) identifying matrix H based on local geometric features with reference to left figurelCalculating a left graph-based similarity value SlIdentification matrix H based on local geometric features with reference to right figurerCalculating a similarity value S based on the right graphr。
4. The abnormal matching identification method according to claim 3,wherein the content of the first and second substances,the lambda is a mapping parameter which is,sigma (-) means to calculate the sum of the absolute values of all the elements inside, ρ is the calculation method to convert the homogeneous coordinate into the inhomogeneous coordinate,the indices i and j are used to distinguish between different objects,Hl={Hl1 Hl2 ... Hln},Hr={Hr1 Hr2 ... Hrn},Hli=Ti'Ti -1,i=1,2,...n,Hri=TiTi'-1,i=1,2,...n。
5. the anomaly matching identification method according to claim 4, wherein the step (3) comprises:
(3.1) matching for one ci={ki,k′iIt is based on the valid neighbor sequence of the left graph Nlt}t=1,2,...kThe corresponding match represented is ctiK, with the left image key points k, respectivelytiK and a right graph key point k'ti,t=1,2,...k;
(3.2) calculating key points k of the left graph of the effective neighbor sequencetiK is relative to the left image center key point kiThe distance sequence is arranged according to the sequence of each effective neighbor in the effective neighbor sequence to obtain a left image neighborhood structure sequence Nfl;
(3.3) calculating key point k 'of right graph of effective neighbor sequence'tiK is relative to the right graph center key point k'iThe distance sequence is arranged according to the sequence of each effective neighbor in the effective neighbor sequence to obtain a right graph neighborhood structure sequence Nfr;
(3.4) from fl=|Nfl-NfrI calculating left image neighborhood structure sequence NflAnd a right picture neighborhood structure sequence NfrOffset amount f oflShifting features for the left graph-based structure, where | represents taking the absolute value of the internal element;
(3.5) matching for onei={ki,k′iObtaining an effective neighbor sequence based on a right image and corresponding matching expressed by the effective neighbor sequence, calculating distance sequence of key points of the right image of the effective neighbor sequence relative to key points of the center of the right image, and arranging according to the sequence of each effective neighbor in the effective neighbor sequence to obtain a neighborhood structure sequence of the right image; calculating the distance sequence of the key points of the left graph of the effective neighbor sequence relative to the key points of the center of the left graph, and arranging according to the sequence of each effective neighbor in the effective neighbor sequence to obtain a neighborhood structure sequence of the left graph; calculating the offset of the neighborhood structure sequence of the right image and the neighborhood structure sequence of the left image into a structure offset characteristic f based on the right imager。
6. The anomaly matching identification method according to claim 5, wherein the step (4) comprises:
(4.1) combining all matched structural offset features in the training image set into a training sample, and manually marking labels of the features of the training sample, wherein the labels are a positive sample and a negative sample, and the matching is represented as a correct matching by 1 and is the positive sample; denote this match as a false match with 0, which is a negative sample;
(4.2) carrying out normalization preprocessing on the structure deviation characteristics of the training samples: taking each sample as a row, taking the structure deviation characteristic of each dimension as a column, normalizing each column of all samples to a [0,1] interval, and storing the minimum value and the maximum value of the samples used in the normalization process as normalization parameters so as to normalize a new sample in the same way after the new sample arrives;
and (4.3) inputting the training samples with the manual labeling labels into a trainer, and training the trainer to obtain a trained classifier.
7. The method for identifying abnormal matches according to claim 6, wherein said identifying abnormal matches by a trained classifier comprises:
respectively executing the step (1) and the step (2) to the image pair needing abnormal matching identification to obtain a similarity value S based on the left image of each matchlSimilarity value S with right graphr;
Executing the step (3) to obtain the characteristics of the image, processing the structural offset characteristics by applying the normalization parameters obtained in the step (4), predicting the probability P that each match is a correct match by using the classifier obtained in the step (4), determining the weighting times q of multiple rounds, and outputting the final prediction probability P' if the weighting times reach q times;
and dividing the prediction probability P' according to a given threshold value t, wherein the probability greater than t is correct matching, the probability less than t is wrong matching, and a final correct matching set is output.
8. The abnormal matching identification method according to claim 7, wherein the final prediction probability P' is obtained by:
(a) using the prediction probability Pq-1Updating the similarity value S with reference to the left figurelSimilarity value S with reference to the right figurerObtaining an updated similarity valueAndwherein, Pq-1={p1 p2 ... pn}, Wherein p isiRepresenting the probability that the ith pair of feature matches is predicted to be a correct match, pjRepresenting the probability that the jth pair of feature matches is predicted to be a correct match;
(c) With updated characteristicsFor input, a trained classifier is applied to predict the probability P of each match being a positive sampleq;
(d) And (c) repeatedly executing the steps (a) to (c) until the iteration number reaches a preset threshold value q, and outputting a final prediction probability P'.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202110163367.5A CN112861878B (en) | 2021-02-05 | 2021-02-05 | Abnormal matching identification method based on structural offset characteristics |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202110163367.5A CN112861878B (en) | 2021-02-05 | 2021-02-05 | Abnormal matching identification method based on structural offset characteristics |
Publications (2)
Publication Number | Publication Date |
---|---|
CN112861878A true CN112861878A (en) | 2021-05-28 |
CN112861878B CN112861878B (en) | 2022-05-20 |
Family
ID=75989289
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN202110163367.5A Active CN112861878B (en) | 2021-02-05 | 2021-02-05 | Abnormal matching identification method based on structural offset characteristics |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN112861878B (en) |
Citations (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
GB201219749D0 (en) * | 2012-11-02 | 2012-12-19 | Zwipe As | Fingerprint matching algorithm |
US8520949B1 (en) * | 2008-06-20 | 2013-08-27 | Google Inc. | Self-similar descriptor filtering |
CN104166848A (en) * | 2014-08-28 | 2014-11-26 | 武汉虹识技术有限公司 | Matching method and system applied to iris recognition |
CN109840529A (en) * | 2019-02-01 | 2019-06-04 | 安徽大学 | A kind of image matching method based on local sensitivity confidence level estimation |
CN111508030A (en) * | 2020-04-10 | 2020-08-07 | 湖北工业大学 | Stereo matching method for computer vision |
CN111899303A (en) * | 2020-07-14 | 2020-11-06 | 中国人民解放军63920部队 | Novel feature matching and relative positioning method considering space inverse projection constraint |
-
2021
- 2021-02-05 CN CN202110163367.5A patent/CN112861878B/en active Active
Patent Citations (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US8520949B1 (en) * | 2008-06-20 | 2013-08-27 | Google Inc. | Self-similar descriptor filtering |
GB201219749D0 (en) * | 2012-11-02 | 2012-12-19 | Zwipe As | Fingerprint matching algorithm |
CN104166848A (en) * | 2014-08-28 | 2014-11-26 | 武汉虹识技术有限公司 | Matching method and system applied to iris recognition |
CN109840529A (en) * | 2019-02-01 | 2019-06-04 | 安徽大学 | A kind of image matching method based on local sensitivity confidence level estimation |
CN111508030A (en) * | 2020-04-10 | 2020-08-07 | 湖北工业大学 | Stereo matching method for computer vision |
CN111899303A (en) * | 2020-07-14 | 2020-11-06 | 中国人民解放军63920部队 | Novel feature matching and relative positioning method considering space inverse projection constraint |
Non-Patent Citations (1)
Title |
---|
杨帆等: "采用结构相似度和几何约束原理的影像误匹配剔除算法", 《信号处理》 * |
Also Published As
Publication number | Publication date |
---|---|
CN112861878B (en) | 2022-05-20 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
Rios-Cabrera et al. | Discriminatively trained templates for 3d object detection: A real time scalable approach | |
CN111126482B (en) | Remote sensing image automatic classification method based on multi-classifier cascade model | |
Guo et al. | Efficient center voting for object detection and 6D pose estimation in 3D point cloud | |
CN107633226B (en) | Human body motion tracking feature processing method | |
Zhang et al. | Robust metric reconstruction from challenging video sequences | |
CN112200121B (en) | Hyperspectral unknown target detection method based on EVM and deep learning | |
CN112330538B (en) | Damage temperature reconstruction image splicing method based on feature point optimization extraction | |
CN110969166A (en) | Small target identification method and system in inspection scene | |
CN113011509B (en) | Lung bronchus classification method and device, electronic equipment and storage medium | |
CN108537805B (en) | Target identification method based on feature geometric benefits | |
JP2021520007A (en) | Vehicle image comparison method and its system | |
CN111723852B (en) | Robust training method for target detection network | |
Yang et al. | Multivariate regression on the grassmannian for predicting novel domains | |
CN114529581A (en) | Multi-target tracking method based on deep learning and multi-task joint training | |
CN112364881A (en) | Advanced sampling consistency image matching algorithm | |
CN117576079A (en) | Industrial product surface abnormality detection method, device and system | |
CN114663687A (en) | Model training method, target recognition method, device, equipment and storage medium | |
CN113128518B (en) | Sift mismatch detection method based on twin convolution network and feature mixing | |
CN112652003A (en) | Three-dimensional point cloud registration method based on RANSAC measure optimization | |
Xie et al. | Automatic neuron tracing in volumetric microscopy images with anisotropic path searching | |
CN112861878B (en) | Abnormal matching identification method based on structural offset characteristics | |
CN117152504A (en) | Space correlation guided prototype distillation small sample classification method | |
CN116311391A (en) | High-low precision mixed multidimensional feature fusion fingerprint retrieval method | |
CN115018787A (en) | Anomaly detection method and system based on gradient enhancement | |
CN115511798A (en) | Pneumonia classification method and device based on artificial intelligence technology |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
GR01 | Patent grant | ||
GR01 | Patent grant |