CN112861878B - Abnormal matching identification method based on structural offset characteristics - Google Patents

Abnormal matching identification method based on structural offset characteristics Download PDF

Info

Publication number
CN112861878B
CN112861878B CN202110163367.5A CN202110163367A CN112861878B CN 112861878 B CN112861878 B CN 112861878B CN 202110163367 A CN202110163367 A CN 202110163367A CN 112861878 B CN112861878 B CN 112861878B
Authority
CN
China
Prior art keywords
matching
sequence
image
calculating
graph
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202110163367.5A
Other languages
Chinese (zh)
Other versions
CN112861878A (en
Inventor
孙琨
王力哲
刘娟
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
China University of Geosciences
Original Assignee
China University of Geosciences
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by China University of Geosciences filed Critical China University of Geosciences
Priority to CN202110163367.5A priority Critical patent/CN112861878B/en
Publication of CN112861878A publication Critical patent/CN112861878A/en
Application granted granted Critical
Publication of CN112861878B publication Critical patent/CN112861878B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
    • G06V10/443—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components by matching or filtering
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/22—Matching criteria, e.g. proximity measures
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G06F18/2415—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on parametric or probabilistic models, e.g. based on likelihood ratio or false acceptance rate versus a false rejection rate
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/42—Global feature extraction by analysis of the whole pattern, e.g. using frequency domain transformations or autocorrelation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/46—Descriptors for shape, contour or point-related descriptors, e.g. scale invariant feature transform [SIFT] or bags of words [BoW]; Salient regional features
    • G06V10/462—Salient features, e.g. scale invariant feature transforms [SIFT]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
    • G06V10/75—Organisation of the matching processes, e.g. simultaneous or sequential comparisons of image or video features; Coarse-fine approaches, e.g. multi-scale approaches; using context analysis; Selection of dictionaries
    • G06V10/757—Matching configurations of points or features

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • General Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Multimedia (AREA)
  • Artificial Intelligence (AREA)
  • Evolutionary Computation (AREA)
  • General Engineering & Computer Science (AREA)
  • Evolutionary Biology (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Probability & Statistics with Applications (AREA)
  • Health & Medical Sciences (AREA)
  • Computing Systems (AREA)
  • Databases & Information Systems (AREA)
  • General Health & Medical Sciences (AREA)
  • Medical Informatics (AREA)
  • Software Systems (AREA)
  • Image Analysis (AREA)

Abstract

The invention discloses a method for carrying out mismatching rejection by applying structural offset characteristics, which belongs to the technical field of computer vision, and comprises the following steps of firstly, obtaining an initial matching set between two images by using a characteristic matching algorithm; secondly, selecting neighbor matching for each matching, extracting a local transformation matrix for each key point in order to reduce the probability that the traditional k neighbor contains error matching, and screening each pair of effective neighbors for initial matching according to the local transformation matrix; and finally, calculating the difference of the bit order of the effective neighbor matching on the two images relative to the center matching, and calling the obtained vector as the structural offset characteristic. The features provided by the invention are simple and intuitive, training and testing can be performed on various typical classifiers, and experiments prove that the error matching can be effectively eliminated by the provided method.

Description

Abnormal matching identification method based on structural offset characteristics
Technical Field
The invention belongs to the technical field of computer vision, and particularly relates to an anomaly matching identification method based on structural offset characteristics.
Background
Image feature matching is a technique used to correlate the position of the same three-dimensional point in space on different images. The technology is generally applied to applications such as motion recovery structures, Simultaneous Localization And Mapping (SLAM), panoramic stitching, stereo matching And the like, And is a basic technology of many application directions in the field of computer vision.
The existing image feature matching technology generally comprises two steps: firstly, extracting key points on an image and describing the key points to obtain feature descriptors of the key points; then, similarity between the measurement descriptors (generally, Euclidean distance between the measurement descriptors) is measured, and the key points with high similarity are a pair of matching points. Although many of the existing methods have yielded good results, image feature matching remains a challenging problem: on one hand, the prior art mainly describes the local information of the key points, so that ambiguity or even error occurs when the matching of the key points is judged by means of local information descriptors in a scene with a repetitive structure; on the other hand, rotation, illumination change, viewpoint change, occlusion and other challenges of the image itself can also cause the generation of a mismatch. The introduction of these false matches can in turn affect the effectiveness of subsequent applications. Therefore, there is a need for an algorithm that can identify and reject anomalous false matches to ensure accuracy in subsequent applications.
There are many types of methods used for anomaly matching identification in recent years. One popular approach is to evaluate the global transforms of all matches to reject false matches that do not satisfy this transform model. However, this method has a great influence on the correctness of the result in the case that the matching set has a small correct matching ratio. Furthermore, this way of estimating the global transformation is not applicable to scenarios where multiple local consistency transformations and non-rigid transformations. Another method uses local discrimination to select the neighborhood around each matching key point (the nearest neighbor is generally used to represent the neighborhood), and compares the error of a certain structure quantization index defined by the key points at the two ends of the matching on the neighborhood to identify abnormal matching. The method is easily interfered by mismatching in a local neighborhood, and a distinguishing threshold value of abnormal matching needs to be preset, so that the method is difficult to adapt to different scenes. Yet another class of approaches employs learned strategies to identify anomalous matches. Some employ a deep learning framework to achieve classification tasks of correct matching and incorrect matching, but require a large amount of data (including information such as camera internal parameters) in the training process, which limits the usability of the method to some extent. Other methods extract matching features by means of artificially designed criteria and use these features for classification. However, the existing method describes the matched features from the aspects of similarity, direction, angle and the like of the spatial structure, the method is too complex and high in calculation difficulty, and the influence of mismatching in the neighborhood structure cannot be relieved.
Disclosure of Invention
Aiming at the defects or improvement requirements of the prior art, the invention provides an abnormal matching identification method based on the structural offset characteristics, a learning method is adopted, matched characteristics are extracted by using matched local structural information, the extracted characteristics have the characteristics of simple calculation and accurate classification, and therefore, an effective classifier can be trained for abnormal matching identification under the condition of only a small number of samples, so that the reliability of abnormal matching identification is greatly improved, and effective input is provided for later practical application.
In order to achieve the above object, the present invention provides an anomaly matching identification method based on structural offset features, which includes:
(1) respectively extracting key points from the image pairs in the training image set, obtaining an initial matching set c, and simultaneously calculating a local transformation matrix A corresponding to each key point;
(2) calculating the geometric characteristic identification matrix of each match for the initial matching set c obtained from the single image pair, wherein each match can obtain a local geometric characteristic identification matrix H taking the left image as a referencelLocal geometric feature identification matrix H with reference to right figurerCalculating similarity values S between matches by using the geometric feature identification matrix, and calculating similarity values S with the left graph as reference when calculating the similarity valueslSimilarity value S with reference to the right figurer；
(3) Determining a neighborhood range k, matching each center, and selecting a similarity value SlThe largest k matches generate a valid neighbor sequence N referenced to the leftlt}t＝1,2,...kSelecting a similarity value SrThe largest k matches generate a valid neighbor sequence N referenced to the right graphrt}t＝1,2,...kRespectively calculating the offset of the distance sequence of the effective neighbor sequence taking the left image as the reference and the effective neighbor sequence taking the right image as the reference relative to the center matching, and generating the matched left image characteristic flAnd right graph feature frSplicing the matched left image characteristic and the right image characteristic to obtain a matched final characteristic [ fl,fr]；
(4) Combining all matched structural deviation features in a training image set into a training sample, manually marking labels of the features of the training sample, preprocessing the features, and then using the preprocessed features as input of a classifier, training the classifier, obtaining feature processing parameters and a final classifier, and facilitating anomaly identification through the trained classifier.
In some alternative embodiments, step (1) comprises:
(1.1) carrying out key point detection on any pair of image pairs in the training image set to obtain M key points of a left image and N key points of a right image;
(1.2) carrying out initial matching to obtain n pairs of matching pairs corresponding to key points { k ] of the left imagei}i＝1,2,...nAnd corresponding local transformation matrix { A }i}i＝1,2,...nSimultaneously obtaining key points k of the right imagei'}i＝1,2,…nAnd corresponding local transformation matrix { A }i'}i＝1,2,…nWhen the matching relationship is { c }i＝(ki,ki')}i＝1，2，…n。
In some alternative embodiments, step (2) comprises:
(2.1) for each key point at the two matched ends, obtaining a local structure matrix T taking the left image as a reference for the key point in the left image, and obtaining a local structure matrix T' taking the right image as a reference for the key point in the right image;
(2.2) calculating a local geometric feature identification matrix H taking the left figure as reference according to the local structure matrix T taking the left figure as referencelCalculating local geometric feature identification matrix H with reference to the right figure according to local structure matrix T' with reference to the right figurer；
(2.3) identifying matrix H based on local geometric features with reference to left figurelCalculating a left graph-based similarity value SlIdentification matrix H based on local geometric features with reference to right figurerCalculating a similarity value S based on the right graphr。
In some of the alternative embodiments, the first and second,
Figure BDA0002936432620000041
wherein the content of the first and second substances,
Figure BDA0002936432620000042
the lambda is a mapping parameter which is,
Figure BDA0002936432620000043
sigma (-) means to calculate the sum of the absolute values of all the elements inside, ρ is the calculation method to convert the homogeneous coordinate into the inhomogeneous coordinate,
Figure BDA0002936432620000044
Figure BDA0002936432620000045
the indices i and j are used to distinguish between different objects,
Figure BDA0002936432620000046
Hl＝{Hl1 Hl2 … Hln}，Hr＝{Hr1 Hr2 … Hrn}，Hli＝Ti'Ti -1，i＝1,2,…n，Hri＝TiTi'-1，i＝1,2,…n。
in some alternative embodiments, step (3) comprises:
(3.1) matching for one ci＝{ki,ki' } based on the valid neighbor sequence of the left graph Nlt}t＝1，2，…kThe corresponding match represented is ctiK, with the left image key points k, respectivelytiK and a right graph key point k'ti,t＝1,2,...k；
(3.2) calculating key points k of the left graph of the effective neighbor sequencetiK is relative to the left image center key point kiThe distance sequence is arranged according to the sequence of each effective neighbor in the effective neighbor sequence to obtain a left image neighborhood structure sequence Nfl；
(3.3) calculating key point k 'of right graph of effective neighbor sequence'tiK is relative to the right image center key point ki' the distance is sequenced, and the sequence N of the neighborhood structure of the right picture is obtained according to the sequence of each effective neighbor in the effective neighbor sequencefr；
(3.4) from fl＝|Nfl-NfrI calculating left image neighborhood structure sequence NflAnd a right picture neighborhood structure sequence NfrOffset amount f oflShifting features for the left graph-based structure, where | represents taking the absolute value of the internal element;
(3.5) matching for onei＝{ki,ki' }, obtaining an effective neighbor sequence based on the right image and corresponding match expressed by the effective neighbor sequence, calculating the distance sequence of the key point of the right image of the effective neighbor sequence relative to the key point of the center of the right image, and arranging according to the sequence of each effective neighbor in the effective neighbor sequence to obtain a neighborhood structure sequence of the right image; calculating the distance sequence of the key points of the left graph of the effective neighbor sequence relative to the key points of the center of the left graph, and arranging according to the sequence of each effective neighbor in the effective neighbor sequence to obtain a neighborhood structure sequence of the left graph; calculating the offset of the neighborhood structure sequence of the right image and the neighborhood structure sequence of the left image into a structure offset characteristic f based on the right imager。
In some alternative embodiments, step (4) comprises:
(4.1) combining all matched structural offset features in the training image set into a training sample, and manually marking labels of the features of the training sample, wherein the labels are a positive sample and a negative sample, and the matching is represented as a correct matching by 1 and is the positive sample; denote this match as a false match with 0, which is a negative sample;
(4.2) carrying out normalization preprocessing on the structure deviation characteristics of the training samples: taking each sample as a row, taking the structure deviation characteristic of each dimension as a column, normalizing each column of all samples to a [0,1] interval, and storing the minimum value and the maximum value of the samples used in the normalization process as normalization parameters so as to normalize a new sample in the same way after the new sample arrives;
and (4.3) inputting the training samples with the manual labeling labels into a trainer, and training the trainer to obtain a trained classifier.
In some optional embodiments, the performing the anomaly identification through the trained classifier includes:
respectively executing the step (1) and the step (2) on the image pair needing abnormal matching identification to obtain a similarity value S based on the left image of each matchlSimilarity value S with right graphr；
Executing the step (3) to obtain the characteristics of the image, processing the structural offset characteristics by applying the normalization parameters obtained in the step (4), predicting the probability P that each match is a correct match by using the classifier obtained in the step (4), determining the weighting times q of multiple rounds, and outputting the final prediction probability P' if the weighting times reach q times;
and dividing the prediction probability P' according to a given threshold value t, wherein the probability greater than t is correct matching, the probability less than t is wrong matching, and a final correct matching set is output.
In some alternative embodiments, the final prediction probability P' is obtained by:
(a) using the prediction probability Pq-1Updating the similarity value S with reference to the left figurelSimilarity value S with reference to the right figurerObtaining an updated similarity value
Figure BDA0002936432620000061
And
Figure BDA0002936432620000062
wherein, Pq-1＝{p1 p2 … pn}，
Figure BDA0002936432620000063
Figure BDA0002936432620000064
Wherein p isiRepresenting the probability that the ith pair of feature matches is predicted to be a correct match, pjRepresenting the probability that the jth pair of feature matches is predicted to be a correct match;
(b) using updated similarity values
Figure BDA0002936432620000065
And
Figure BDA0002936432620000066
obtaining updated features
Figure BDA0002936432620000067
(c) With updated characteristics
Figure BDA0002936432620000068
For input, a trained classifier is applied to predict the probability P of each match being a positive sampleq；
(d) And (c) repeatedly executing the steps (a) to (c) until the iteration number reaches a preset threshold value q, and outputting a final prediction probability P'.
In general, compared with the prior art, the above technical solution contemplated by the present invention can achieve the following beneficial effects:
(1) an abnormal matching identification method based on structural offset features is provided, and the method utilizes the offset of selecting effective neighbors and comparing the sequence order of the effective neighbors as matched features and uses the features to train out a usable classifier.
(2) In application, the adopted multi-round iteration strategy can further improve the reliability of the neighbors and the distribution of the prediction probability, reduce the influence of abnormal matching on the final characteristics and obtain more effective results.
(3) The structural offset feature has the advantages of simple and rapid calculation method when extracting the feature, and the obtained feature can be effectively applied to abnormal matching identification.
Drawings
Fig. 1 is a schematic flowchart of an anomaly matching identification method based on structural deviation features according to an embodiment of the present invention;
FIG. 2 is a schematic flow chart of another method for identifying an abnormal matching based on a structural deviation feature according to an embodiment of the present invention;
FIG. 3 is a training flow diagram provided by an embodiment of the present invention;
fig. 4 is a flowchart of an application test according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention. In addition, the technical features involved in the embodiments of the present invention described below may be combined with each other as long as they do not conflict with each other.
The invention provides an abnormal matching identification method based on structural offset characteristics, which comprises the steps of firstly, detecting key points on an image, obtaining a local transformation matrix corresponding to the key points, and obtaining an initial matching set by applying a matching method; secondly, a method for selecting effective neighbors based on space structure common potential is provided, each match comprises two key points which are respectively positioned on a left graph and a right graph, so that two effective neighbor sequences can be obtained for each match, wherein the effective neighbor sequences are obtained by using a similarity value calculated by taking the left graph as reference and a similarity value calculated by taking the right graph as reference; then, the relative bit order difference between the valid neighbor and the center match is computed as a feature, called a structure-shifted feature. In the training part, labels matched with the samples are labeled manually, and a proper machine learning method is selected for training, so that an effective classifier can be obtained under the condition of only a small number of samples. In the testing part, although the method of selecting the effective neighbors based on the space structure co-potential can avoid the false matching mixed in the effective neighbors under certain conditions, the influence of the false matching cannot be completely eliminated. Based on this, a multi-round weighting strategy is adopted in the test part: the classifier is used to predict the correct probability of a match and this probability is used to update the criteria for selecting valid neighbors, making the selected valid neighbors progressively cleaner. The finally obtained result is superior to the current mainstream method in the comprehensive performance of precision and recall rate.
As shown in fig. 1 and fig. 2, a flow chart of a mismatch elimination method based on structural offset features is shown, which includes the following steps:
s1: respectively extracting key points from the image pairs in the training image set, obtaining an initial matching set c, and simultaneously calculating a local transformation matrix A corresponding to each key point;
in the embodiment of the present invention, step S1 may be implemented as follows:
s1.1: performing key point detection on any pair of images in the training image set to obtain M key points of a left image and N key points of a right image;
wherein, the image key points can adopt hessian operator to extract the features.
The image Feature description may be characterized by using Scale Invariant Feature Transform (SIFT).
S1.2: performing initial matching to obtain n pairs of matching pairs corresponding to key points { k ] of the left imagei}i＝1，2，…nAnd corresponding local transformation matrix { A }i}i＝1，2，…nSimultaneously obtaining key points k of the right imagei'}i＝1,2,...nAnd corresponding local transformation matrix { A }i'}i＝1，2，...nWhen the matching relationship is { c }i＝(ki,ki')}i＝1,2,...n。
Wherein, the local transformation matrix may adopt a local affine transformation matrix.
S2: calculating the geometric characteristic identification matrix of each match for the initial matching set c obtained from the single image pair, wherein each match can obtain a local geometric characteristic identification matrix H taking the left image as a referencelLocal geometric feature identification matrix H with reference to right figurerCalculating similarity values S between matches by using the geometric feature identification matrix, and calculating similarity values S with the left graph as reference when calculating the similarity valueslSimilarity value S with reference to the right figurer；
In the embodiment of the present invention, step S2 may be implemented as follows:
s2.1: for each key point at the two matched ends, local structure matrixes T and T' taking the corresponding image as reference are obtained;
wherein T and T' are specifically combined as shown in formula (1):
Figure BDA0002936432620000091
s2.2: respectively calculating local geometric feature identification matrixes H with left images as referenceslLocal geometric feature identification matrix H with reference to the right figurerThe local structure matrix is composed of an affine transformation matrix and the position information of the key points, so that the local geometric feature identification matrix is called a local homography matrix;
wherein HlAnd HrSee formula (2), and concretely, a calculation formula is see formula (3);
Hl＝{Hl1 Hl2 ... Hln}Hr＝{Hr1 Hr2 ... Hrn} (2)
Hli＝Ti'Ti -1，i＝1,2,...n Hri＝TiTi'-1，i＝1,2,...n (3)
s2.3: calculating a left graph-based similarity value SlSimilarity value S with right graphr，SlAnd SrSee formula (4), and concretely, a formula is calculated see formula (5);
Figure BDA0002936432620000092
Figure BDA0002936432620000093
wherein λ is a mapping parameter, in particular elijAnd erijSee formula (6):
Figure BDA0002936432620000094
wherein, the meaning of σ (-) is to calculate the sum of absolute values of all internal elements, ρ is a calculation method for converting homogeneous coordinates into inhomogeneous coordinates, and the specific calculation method is shown in formula (7):
ρ([a b c])＝[a/c b/c]T (7)
wherein the content of the first and second substances,
Figure BDA0002936432620000101
where the indices i and j are used to distinguish between different objects.
S3: determining a neighborhood range k, matching each center, and selecting a similarity value SlThe largest k matches generate a valid neighbor sequence N referenced to the leftlt}t＝1,2,...kAnd a similarity value SrThe largest k matches generate a valid neighbor sequence N referenced to the right graphrt}t＝1,2,...k(ii) a Respectively calculating the offset of the distance sequence of the effective neighbor sequence taking the left image as the reference and the effective neighbor sequence taking the right image as the reference relative to the center matching, and generating the matched left image characteristic flAnd right graph feature frSplicing the matched left image characteristic and the right image characteristic to obtain a matched final characteristic [ fl,fr]；
The size of the neighborhood range k can be determined according to actual needs, and the embodiment of the invention is not limited uniquely.
In the embodiment of the present invention, step S3 may be implemented as follows:
s3.1: for one match ci＝{ki,ki' } based on the valid neighbor sequence of the left graph Nlt}t＝1,2,...kThe corresponding match represented is ctiK, with the left image key points k, respectivelytiT 1,2,. k and the right graph key point kt'i,t＝1,2,...k；
S3.2: calculating key point k of effective neighbor sequence left graphtiK is relative to the left image center key point kiThe distance sequence is arranged according to the sequence of each effective neighbor in the effective neighbor sequence to obtain a left image neighborhood structure sequence Nfl；
S3.3: calculating key point k 'of right graph of effective neighbor sequence'tiK is relative to the right image center key point ki' the distance is sequenced, and the sequence N of the neighborhood structure of the right picture is obtained according to the sequence of each effective neighbor in the effective neighbor sequencefr；
S3.4: calculating a left image neighborhood structure sequence NflAnd a right picture neighborhood structure sequence NfrOffset amount f oflCalculating a formula specifically shown in the formula (8) for the structural deviation characteristics based on the left graph;
fl＝|Nfl-Nfr| (8)
wherein, | · | represents taking the absolute value of the internal element;
s3.5: for one match ci＝{ki,ki' }, obtaining an effective neighbor sequence based on the right image and corresponding match expressed by the effective neighbor sequence, calculating the distance sequence of the key point of the right image of the effective neighbor sequence relative to the key point of the center of the right image, and arranging according to the sequence of each effective neighbor in the effective neighbor sequence to obtain a neighborhood structure sequence of the right image; calculating the distance sequence of the key points of the left graph of the effective neighbor sequence relative to the key points of the center of the left graph, and arranging according to the sequence of each effective neighbor in the effective neighbor sequence to obtain a neighborhood structure sequence of the left graph; calculating the offset of the neighborhood structure sequence of the right image and the neighborhood structure sequence of the left image into a structure offset characteristic f based on the right imager。
S4: as shown in fig. 3, all matched structural deviation features in the training image set are combined into a training sample, and labels of the features of the training sample are manually labeled, where the labels are positive samples and negative samples: 1 means that this match is a correct match, a positive sample; 0 means that the match is a false match, is a negative sample, and then the feature is normalized: taking each sample as a line, taking the characteristic of each dimension as a column, normalizing each column of all samples to a [0,1] interval, saving the minimum value and the maximum value of the sample used in the normalization process as normalization parameters, so as to normalize the new sample in the same way after the new sample arrives, selecting a proper training method for training, inputting the training sample with the manual labeling label into a trainer, and obtaining the trained classifier.
Wherein, random forests can be selected as classifiers for training.
In an embodiment of the present invention, the performing anomaly identification by using the trained classifier includes:
respectively executing the step S1 and the step S2 on the image pair needing abnormal matching identification, and obtaining a similarity value S based on the left image of each matchlSimilarity value S with right graphr；
Executing step S3, obtaining the characteristics of the image, processing the characteristics by applying the normalization parameters obtained in step S4, predicting the probability P that each match is correct by using the classifier obtained in step S4, determining the weighting times q of multiple rounds, and outputting the final prediction probability P' if the weighting times reaches q times;
dividing the prediction probability P' according to a given threshold value t, wherein the probability greater than t is correct matching, and the probability less than t is false matching; the final set of correct matches is output.
The size of the given threshold t may be determined according to actual needs, and the embodiment of the present invention is not limited uniquely.
In the embodiment of the present invention, as shown in fig. 4, the final prediction probability P' may be obtained by:
(a) using the prediction probability Pq-1The similarity value S calculated in step S2 with reference to the left figure is updatedlSimilarity value S with reference to the right figurerObtaining an updated similarity value
Figure BDA0002936432620000121
And
Figure BDA0002936432620000122
wherein, Pq-1See formula (9), piIndicating that the ith pair of feature matches is predictedThe probability of a correct match is determined,
Figure BDA0002936432620000123
and
Figure BDA0002936432620000124
the similarity value is updated in the formula (10)
Figure BDA0002936432620000125
And
Figure BDA0002936432620000126
see formula (11):
Pq-1＝{p1 p2 ... pn} (9)
Figure BDA0002936432620000127
Figure BDA0002936432620000128
(b) using updated similarity values
Figure BDA0002936432620000129
And
Figure BDA00029364326200001210
repeatedly executing the step S3 to obtain the updated characteristics
Figure BDA00029364326200001211
(c) With updated characteristics
Figure BDA00029364326200001212
For input, the classifier obtained in step S4 is applied to predict the probability P of each match being a positive sampleq；
(d) And (c) repeatedly executing the steps (a) to (c) until the iteration number reaches a preset threshold value q, and outputting a final prediction probability P'.
It should be noted that, according to the implementation requirement, each step/component described in the present application can be divided into more steps/components, and two or more steps/components or partial operations of the steps/components can be combined into new steps/components to achieve the purpose of the present invention.
It will be understood by those skilled in the art that the foregoing is only a preferred embodiment of the present invention, and is not intended to limit the invention, and that any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the scope of the present invention.

Claims (8)

1. An abnormal matching identification method based on structural offset features is characterized by comprising the following steps:
(1) respectively extracting key points from the image pairs in the training image set, obtaining an initial matching set c, and simultaneously calculating a local transformation matrix A corresponding to each key point;
(2) calculating a geometric characteristic identification matrix of each match for an initial matching set c obtained from a single image pair, wherein each match can obtain a local geometric characteristic identification matrix H with the left image as a referencelLocal geometric feature identification matrix H with reference to right figurerCalculating similarity values S between matches by using the geometric feature identification matrix, and calculating similarity values S with the left graph as reference when calculating the similarity valueslSimilarity value S with reference to the right figurer；
(3) Determining a neighborhood range k, matching each center, and selecting a similarity value SlThe largest k matches generate a valid neighbor sequence N referenced to the leftlt}t＝1,2,...kSelecting a similarity value SrThe largest k matches generate a valid neighbor sequence N referenced to the right graphrt}t＝1,2,...kRespectively calculating the offset of the distance sequence of the effective neighbor sequence taking the left image as the reference and the effective neighbor sequence taking the right image as the reference relative to the center matching, and generating the matched left image characteristic flAnd right graph feature frSplicing the matched left image characteristic and the right image characteristic to obtain a matched final characteristic [ fl,fr]；
(4) Combining all matched structural deviation features in a training image set into a training sample, manually marking labels of the features of the training sample, preprocessing the features, and then using the preprocessed features as input of a classifier, training the classifier, obtaining feature processing parameters and a final classifier, and facilitating anomaly identification through the trained classifier.
2. The anomaly matching identification method according to claim 1, wherein step (1) comprises:
(1.1) carrying out key point detection on any pair of image pairs in the training image set to obtain M key points of a left image and N key points of a right image;
(1.2) carrying out initial matching to obtain n pairs of matching pairs corresponding to key points { k ] of the left imagei}i＝1,2,...nAnd corresponding local transformation matrix { A }i}i＝1,2,...nWhile obtaining key points { k 'of the right graph'i}i＝1,2,...nAnd corresponding local transformation matrix { A'i}i＝1,2,...nWhen the matching relationship is { c }i＝(ki,k′i)}i＝1,2,...n。
3. The abnormal match identification method of claim 2, wherein step (2) comprises:
(2.1) for each key point at the two matched ends, obtaining a local structure matrix T taking the left image as a reference for the key point in the left image, and obtaining a local structure matrix T' taking the right image as a reference for the key point in the right image;
(2.2) calculating a local geometric feature identification matrix H taking the left figure as reference according to the local structure matrix T taking the left figure as referencelCalculating local geometric feature identification matrix H with reference to the right figure according to local structure matrix T' with reference to the right figurer；
(2.3) identifying matrix H based on local geometric features with reference to left figurelComputing baseSimilarity value S in left graphlIdentification matrix H based on local geometric features with reference to right figurerCalculating a similarity value S based on the right graphr。
4. The abnormal matching identification method according to claim 3,
Figure FDA0002936432610000021
wherein, the first and the second end of the pipe are connected with each other,
Figure FDA0002936432610000022
the lambda is a mapping parameter which is,
Figure FDA0002936432610000023
sigma (-) means to calculate the sum of the absolute values of all the elements inside, ρ is the calculation method to convert the homogeneous coordinate into the inhomogeneous coordinate,
Figure FDA0002936432610000024
the indices i and j are used to distinguish between different objects,
Figure FDA0002936432610000025
Hl＝{Hl1 Hl2 ... Hln}，Hr＝{Hr1 Hr2 ... Hrn}，Hli＝Ti'Ti -1，i＝1,2,...n，Hri＝TiTi'-1，i＝1,2,...n。
5. the anomaly matching identification method according to claim 4, wherein the step (3) comprises:
(3.1) matching for one ci＝{ki,k′iIt is based on the valid neighbor sequence of the left graph Nlt}t＝1,2,...kThe corresponding match represented is ctiK, with the left image key points k, respectivelytiK and a right graph key point k'ti,t＝1,2,...k；
(3.2) calculating key points k of the left graph of the effective neighbor sequencetiK is relative to the left image center key point kiThe distance sequence is arranged according to the sequence of each effective neighbor in the effective neighbor sequence to obtain a left image neighborhood structure sequence Nfl；
(3.3) calculating key point k 'of right graph of effective neighbor sequence'tiK is relative to the right graph center key point k'iThe distance sequence is arranged according to the sequence of each effective neighbor in the effective neighbor sequence to obtain a neighborhood structure sequence N of the right picturefr；
(3.4) from fl＝|Nfl-NfrI calculating left image neighborhood structure sequence NflAnd a right picture neighborhood structure sequence NfrOffset f oflShifting features for the left graph-based structure, where | represents taking the absolute value of the internal element;
(3.5) matching for onei＝{ki,k′iObtaining an effective neighbor sequence based on a right image and corresponding matching expressed by the effective neighbor sequence, calculating distance sequence of key points of the right image of the effective neighbor sequence relative to key points of the center of the right image, and arranging according to the sequence of each effective neighbor in the effective neighbor sequence to obtain a neighborhood structure sequence of the right image; calculating the distance sequence of the key points of the left graph of the effective neighbor sequence relative to the key points of the center of the left graph, and arranging according to the sequence of each effective neighbor in the effective neighbor sequence to obtain a neighborhood structure sequence of the left graph; calculating the offset of the neighborhood structure sequence of the right image and the neighborhood structure sequence of the left image into a structure offset characteristic f based on the right imager。
6. The anomaly matching identification method according to claim 5, wherein the step (4) comprises:
(4.1) combining all matched structural offset features in the training image set into a training sample, and manually marking labels of the features of the training sample, wherein the labels are a positive sample and a negative sample, and the matching is represented as a correct matching by 1 and is the positive sample; denote this match as a false match with 0, which is a negative sample;
(4.2) carrying out normalization preprocessing on the structure deviation characteristics of the training samples: taking each sample as a row, taking the structure deviation characteristic of each dimension as a column, normalizing each column of all samples to a [0,1] interval, and storing the minimum value and the maximum value of the samples used in the normalization process as normalization parameters so as to normalize a new sample in the same way after the new sample arrives;
and (4.3) inputting the training samples with the manual labeling labels into a trainer, and training the trainer to obtain a trained classifier.
7. The method for identifying abnormal matches according to claim 6, wherein said identifying abnormal matches by a trained classifier comprises:
respectively executing the step (1) and the step (2) to the image pair needing abnormal matching identification to obtain a similarity value S based on the left image of each matchlSimilarity value S with right graphr；
Executing the step (3) to obtain the characteristics of the image, applying the normalization parameters obtained in the step (4) to process the structural offset characteristics, predicting the probability P that each match is a correct match by using the classifier obtained in the step (4), determining the weighting times q of multiple rounds, and outputting the final prediction probability P' if the weighting times reach q times;
and dividing the prediction probability P' according to a given threshold value t, wherein the probability greater than t is correct matching, the probability less than t is wrong matching, and a final correct matching set is output.
8. The abnormal match recognition method of claim 7, wherein the final prediction probability P' is obtained by:
(a) using the prediction probability Pq-1Updating the similarity value S with reference to the left figurelSimilarity value S with reference to the right figurerObtaining an updated similarity value
Figure FDA0002936432610000041
And
Figure FDA0002936432610000042
wherein, Pq-1＝{p1 p2 ... pn}，
Figure FDA0002936432610000051
Figure FDA0002936432610000052
Wherein p isiRepresenting the probability that the ith pair of feature matches is predicted to be a correct match, pjRepresenting the probability that the jth pair of feature matches is predicted to be a correct match;
(b) using updated similarity values
Figure FDA0002936432610000053
And
Figure FDA0002936432610000054
obtaining updated features
Figure FDA0002936432610000055
(c) With updated characteristics
Figure FDA0002936432610000056
For input, a trained classifier is applied to predict the probability P of each matching positive sampleq；
(d) And (c) repeatedly executing the steps (a) to (c) until the iteration number reaches a preset threshold value q, and outputting a final prediction probability P'.
CN202110163367.5A 2021-02-05 2021-02-05 Abnormal matching identification method based on structural offset characteristics Active CN112861878B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110163367.5A CN112861878B (en) 2021-02-05 2021-02-05 Abnormal matching identification method based on structural offset characteristics

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110163367.5A CN112861878B (en) 2021-02-05 2021-02-05 Abnormal matching identification method based on structural offset characteristics

Publications (2)

Publication Number Publication Date
CN112861878A CN112861878A (en) 2021-05-28
CN112861878B true CN112861878B (en) 2022-05-20

Family

ID=75989289

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110163367.5A Active CN112861878B (en) 2021-02-05 2021-02-05 Abnormal matching identification method based on structural offset characteristics

Country Status (1)

Country Link
CN (1) CN112861878B (en)

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
GB201219749D0 (en) * 2012-11-02 2012-12-19 Zwipe As Fingerprint matching algorithm
US8520949B1 (en) * 2008-06-20 2013-08-27 Google Inc. Self-similar descriptor filtering
CN104166848A (en) * 2014-08-28 2014-11-26 武汉虹识技术有限公司 Matching method and system applied to iris recognition
CN109840529A (en) * 2019-02-01 2019-06-04 安徽大学 A kind of image matching method based on local sensitivity confidence level estimation
CN111508030A (en) * 2020-04-10 2020-08-07 湖北工业大学 Stereo matching method for computer vision
CN111899303A (en) * 2020-07-14 2020-11-06 中国人民解放军63920部队 Novel feature matching and relative positioning method considering space inverse projection constraint

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8520949B1 (en) * 2008-06-20 2013-08-27 Google Inc. Self-similar descriptor filtering
GB201219749D0 (en) * 2012-11-02 2012-12-19 Zwipe As Fingerprint matching algorithm
CN104166848A (en) * 2014-08-28 2014-11-26 武汉虹识技术有限公司 Matching method and system applied to iris recognition
CN109840529A (en) * 2019-02-01 2019-06-04 安徽大学 A kind of image matching method based on local sensitivity confidence level estimation
CN111508030A (en) * 2020-04-10 2020-08-07 湖北工业大学 Stereo matching method for computer vision
CN111899303A (en) * 2020-07-14 2020-11-06 中国人民解放军63920部队 Novel feature matching and relative positioning method considering space inverse projection constraint

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
采用结构相似度和几何约束原理的影像误匹配剔除算法;杨帆等;《信号处理》;20160125(第01期);全文 *

Also Published As

Publication number Publication date
CN112861878A (en) 2021-05-28

Similar Documents

Publication Publication Date Title
Rios-Cabrera et al. Discriminatively trained templates for 3d object detection: A real time scalable approach
CN105844669B (en) A kind of video object method for real time tracking based on local Hash feature
Guo et al. Efficient center voting for object detection and 6D pose estimation in 3D point cloud
CN112200121B (en) Hyperspectral unknown target detection method based on EVM and deep learning
CN112330538B (en) Damage temperature reconstruction image splicing method based on feature point optimization extraction
CN111126482A (en) Remote sensing image automatic classification method based on multi-classifier cascade model
CN108229588B (en) Machine learning identification method based on deep learning
CN110969166A (en) Small target identification method and system in inspection scene
CN113011509B (en) Lung bronchus classification method and device, electronic equipment and storage medium
CN108537805B (en) Target identification method based on feature geometric benefits
CN111723852B (en) Robust training method for target detection network
Xu et al. GLORN: Strong generalization fully convolutional network for low-overlap point cloud registration
CN114529581A (en) Multi-target tracking method based on deep learning and multi-task joint training
CN117576079A (en) Industrial product surface abnormality detection method, device and system
CN112364881A (en) Advanced sampling consistency image matching algorithm
CN113128518B (en) Sift mismatch detection method based on twin convolution network and feature mixing
CN117152504A (en) Space correlation guided prototype distillation small sample classification method
Xie et al. Automatic neuron tracing in volumetric microscopy images with anisotropic path searching
CN112861878B (en) Abnormal matching identification method based on structural offset characteristics
Ding et al. HAPNet: a head-aware pedestrian detection network associated with the affinity field
CN116311391A (en) High-low precision mixed multidimensional feature fusion fingerprint retrieval method
CN115018787A (en) Anomaly detection method and system based on gradient enhancement
CN115511798A (en) Pneumonia classification method and device based on artificial intelligence technology
Zhang et al. Yolo-infrared: Enhancing yolox for infrared scene
CN114445649A (en) Method for detecting RGB-D single image shadow by multi-scale super-pixel fusion

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant