CN110245678B - Image matching method based on heterogeneous twin region selection network - Google Patents

Image matching method based on heterogeneous twin region selection network Download PDF

Info

Publication number
CN110245678B
CN110245678B CN201910376172.1A CN201910376172A CN110245678B CN 110245678 B CN110245678 B CN 110245678B CN 201910376172 A CN201910376172 A CN 201910376172A CN 110245678 B CN110245678 B CN 110245678B
Authority
CN
China
Prior art keywords
network
matched
template
convolution
module
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201910376172.1A
Other languages
Chinese (zh)
Other versions
CN110245678A (en
Inventor
杨卫东
蒋哲兴
王祯瑞
姜昊
王公炎
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Huazhong University of Science and Technology
Original Assignee
Huazhong University of Science and Technology
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Huazhong University of Science and Technology filed Critical Huazhong University of Science and Technology
Priority to CN201910376172.1A priority Critical patent/CN110245678B/en
Publication of CN110245678A publication Critical patent/CN110245678A/en
Application granted granted Critical
Publication of CN110245678B publication Critical patent/CN110245678B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features

Abstract

The invention discloses an image matching method based on a heterogeneous twin region selection network, and belongs to the field of computer vision. The system comprises a heterogeneous twin network and a regional matching network which are sequentially connected in series; the heterogeneous twin network is used for extracting a characteristic diagram of the template diagram and a characteristic diagram of a diagram to be matched; the area matching network is used for obtaining an area matching result according to the characteristic graph of the template graph and the characteristic graph of the graph to be matched; the heterogeneous twin network comprises a sub-network A and a sub-network B which are mutually connected in parallel, each sub-network comprises a feature extraction module, a feature fusion module and a maximum value pooling module which are sequentially connected in series, the modules of the two sub-networks are the same, and only the convolution kernels of the first layer of convolution of the feature extraction module are different. The heterogeneous twin region selection network is applied to image matching, so that the input of a non-fixed-scale template and a to-be-matched image is realized, the multi-layer characteristics of the image are fully utilized, the performance of the matching method is effectively improved, and the success rate and the speed of matching are increased.

Description

Image matching method based on heterogeneous twin region selection network
Technical Field
The invention belongs to the field of computer vision, and particularly relates to an image matching method based on a heterogeneous twin region selection network.
Background
Image matching refers to the process of finding images or image areas (sub-images) in an image (or a collection of images) that are similar to a given scene area image. The known scene area image is generally referred to as a template image, and the sub-image of the image to be searched, which may correspond to the known scene area image, is referred to as a scene area image to be matched of the template. Image matching is to find a correspondence between two or more images of the same scene from different times or different perspectives, and specific applications include object or scene recognition, solving 3D structures in multiple images, stereo correspondence, motion tracking, and the like.
Most of the current image matching algorithms only use shallow artificial features, such as gray features, gradient features and the like, because of the influence of the change of shooting time, shooting angle and natural environment, the defects of a sensor, noise and the like, the shot images have gray distortion and geometric distortion, scenes have large change in the images, so that the template images and the images to be matched have difference to a certain degree, and the shallow features often fail, so that a large amount of manpower is required for preparing the template at present, the operation process is complex, and the efficiency is low; at present, most deep neural networks have higher requirements on the number of data, and the types of samples with smaller number in a training set, namely the conditions of few samples and single sample, cannot be identified frequently.
Disclosure of Invention
Aiming at the defects of the prior art, the invention aims to solve the technical problems that the image matching algorithm in the prior art can not basically adapt to imaging viewpoint and scale change, the scene adaptability and the anti-interference capability are weak, and the matching success rate is low.
In order to achieve the above object, in a first aspect, an embodiment of the present invention provides an isomerous twin region selection network, where the isomerous twin region selection network includes an isomerous twin network and a region matching network that are sequentially connected in series;
the heterogeneous twin network is used for extracting a characteristic diagram of the template diagram and a characteristic diagram of a diagram to be matched;
the area matching network is used for obtaining an area matching result according to the characteristic diagram of the template diagram and the characteristic diagram of the diagram to be matched;
the heterogeneous twin network comprises a sub-network A and a sub-network B which are mutually connected in parallel, each sub-network comprises a feature extraction module, a feature fusion module and a maximum value pooling module which are sequentially connected in series, the two sub-networks only have different convolution kernels of the first layer of convolution of the feature extraction module, and the other sub-networks are the same.
Specifically, the feature extraction module is configured to extract a feature map of the input image, the feature fusion module is configured to fuse convolution features of the last three layers of the feature extraction module, and the maximum pooling module normalizes a scale of the fused features.
Specifically, the feature extraction module replaces the ResNet18 second layer with a convolution.
Specifically, the feature extraction module replaces the last layer of the ResNet18 with a convolutional layer.
Specifically, the area matching network includes: the system comprises a feature division module, a classification module and a position regression module; the classification module and the position regression module are connected in parallel, processed in parallel and connected in series behind the characteristic division module;
the feature division module includes: a first convolution, a second convolution, a third convolution and a fourth convolution;
the first convolution is used for extracting a template classification feature map from the feature map of the template map;
the second convolution is used for extracting a template position feature map from the feature map of the template map;
the third convolution is used for extracting a classification characteristic diagram of the image to be matched from the characteristic diagram of the image to be matched;
the fourth convolution is used for extracting the position characteristic graph of the graph to be matched from the characteristic graph of the graph to be matched;
the classification module is used for convolving the template classification characteristic graph with the classification characteristic graph of the graph to be matched by using the template classification characteristic graph as a convolution kernel and outputting a matched class;
and the position regression module is used for convolving the template position characteristic graph serving as a convolution kernel with the position characteristic graph of the graph to be matched and outputting a matched position.
In a second aspect, an embodiment of the present invention provides an image matching method based on the heterogeneous twin region selection network in the first aspect, including the following steps:
s1, training a heterogeneous twin region selection network by using a training sample, wherein the training sample is a template picture-picture to be matched pair, and a label of the training sample is position information of a region corresponding to the template picture in a corresponding region in the picture to be matched;
and S2, inputting the sample to be tested into the trained heterogeneous twin region selection network, and outputting the matching result of the sample to be tested.
Specifically, the total loss function during training is HCE + HSL;
Figure GDA0003064687370000031
Figure GDA0003064687370000032
Figure GDA0003064687370000033
Figure GDA0003064687370000034
Figure GDA0003064687370000035
Figure GDA0003064687370000036
Figure GDA0003064687370000037
Figure GDA0003064687370000038
Figure GDA0003064687370000041
Figure GDA0003064687370000042
d＝|t-t*|＝|(x0-xG)+(yo-yG)+(wo-wG)+(ho-hG)|
Figure GDA0003064687370000043
where HCE is the classification loss function, HSL is the positional regression loss, p represents the probability that the predicted sample class is positive, p*Is the corresponding label, N represents the total number of samples, t represents the sample position of the network output
Figure GDA0003064687370000044
t*Is the corresponding label, i.e. the actual position of the specimen
Figure GDA0003064687370000045
x and y represent abscissa and ordinate, and w and h represent width and length.
Specifically, if the output position and actual tag intersection ratio IOU >0.7, p is*1, otherwise, p*＝0。
Specifically, after step S1 and before step S2, the heterogeneous twin region selection network may be optimized by using the sample to be tested, specifically as follows:
inputting the template graph and the graph to be matched in the test sample set into the network, calculating the cross-over ratio of the output result and the label, measuring the matching success probability by the cross-over ratio, evaluating the performance of the image matching network and determining whether to continue training the network.
In a third aspect, an embodiment of the present invention provides a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the method for image matching based on a heterogeneous twin region selection network according to the second aspect is implemented.
Generally, compared with the prior art, the above technical solution conceived by the present invention has the following beneficial effects:
1. the heterogeneous twin region selection network is applied to image matching, the method is improved according to the problems existing in the practical application of the network model, the algorithm realizes the input of a non-fixed scale template and a graph to be matched, simultaneously, the multi-layer characteristic information of the image is fully utilized, the anti-interference capability of the matching method is effectively improved, the method is suitable for the change of the viewpoint and the scale of imaging, the success rate and the speed of matching are increased, the requirement of the matching on the quality of the template is reduced, the method is suitable for the matching under the condition of less samples and single sample, and the labor cost can be greatly reduced in the practical application.
2. In the process of training the network, the invention provides a novel loss function, namely, a balance loss function is added, wherein the balance loss function comprises two items of balance cross entropy loss HCE and balance regression loss HSL, the two loss items have excellent effect on improving the matching success rate of the network, and the network convergence speed is effectively improved.
Drawings
FIG. 1 is a schematic diagram of a training sample provided in an embodiment of the present invention;
FIG. 2 is a schematic diagram of a test sample provided in an embodiment of the present invention;
fig. 3 is a schematic diagram of a heterogeneous twin region selection network structure according to an embodiment of the present invention;
fig. 4 is a flowchart of an image matching method based on an isomeric twin region selection network according to an embodiment of the present invention;
FIG. 5(a) is a diagram of a template to be tested according to an embodiment of the present invention;
fig. 5(b) shows a matching result of the sample to be tested according to the embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
Sample generation
(1) Prepare n different scenes { P1，...，Pi...，Pn}, same scene PiContaining a plurality of visible light images { Pi1，...，Pij，...PiM}，PijI.e. scene PiThe jth graph of (1).
(2) For each image of the same scene, manually selecting a region { P } in the sceneij1，...，Pijk，...Pijs}，PijkI.e. representing a scene PiThe jth picture of (1) in the kth region. In different images in the same scene, the size, the brightness, the angle and the like of the area are different, and the area has certain deformation.
(3) In the same scene PiA plurality of visible light images { P }i1，...，Pij，...PiMIn the method, an image is randomly selected
Figure GDA0003064687370000069
Cutting out the selected region k as template drawing
Figure GDA0003064687370000061
Randomly selecting another image
Figure GDA0003064687370000062
As the graph to be matched, and marking k in the graph to be matched
Figure GDA0003064687370000063
The position of the corresponding area in (1) is used as a label (ground route).
Figure GDA0003064687370000064
Representing a scene PiLabel, x, corresponding to jth image area kGIndicating that region k is in image PijAbscissa of center of middle corresponding region, yGRepresenting a region kIn picture PijOrdinate of center of middle corresponding region, wGIndicating that region k is in image PijWidth of middle corresponding region, hGIndicating that region k is in image PijThe length of the corresponding region in (a). The above coordinates are pixel coordinates.
(4) Drawing template
Figure GDA0003064687370000065
With corresponding graph to be matched
Figure GDA0003064687370000066
As input to the network in graph pair form, is recorded as
Figure GDA0003064687370000067
And repeat this operation for the full scene { P }1，P2，P3...，PnThe method carries out traversal and selects partial scenes
Figure GDA0003064687370000068
As a training sample set, the rest of the scenarios
Figure GDA00030646873700000610
As the test sample set, in order to verify the adaptability of the network to the case of few samples and single sample during the test, the training set does not overlap with the sample classes of the test set, the training set does not contain the sample classes of the test set, for example, the test set contains airplanes, ships and the like, and the training set does not contain airplanes and ships. Some training samples in the training set are shown in fig. 1 and include the categories of car owners, cars, and pedestrians. The test samples of the test set are shown in FIG. 2 and include the categories building, rabbit, airplane.
Heterogeneous twin zone selection network
As shown in fig. 3, the heterogeneous twin region selection network includes a heterogeneous twin network and a region matching network connected in series, the heterogeneous twin network is used to extract a feature map of the template map and a feature map of the to-be-matched map, and the region matching network is used to obtain an image matching result according to the feature map of the template map and the feature map of the to-be-matched map. The heterogeneous twin region selection network is only suitable for sensor images in the same spectral band, such as a visible light template image and a visible light image to be matched, and is not suitable for matching images in different spectral bands, such as an SAR template image and a visible light image to be matched.
The heterogeneous twin network comprises a sub-network A and a sub-network B which are connected in parallel, and the sub-network comprises a feature extraction module, a feature fusion module and a maximum value pooling module which are connected in series. Considering that the imaging viewpoints and image scales of the template image and the image to be matched are different and the difference of the characteristics is large, the characteristics are extracted by adopting the same convolution kernels, and the effect is general, so that the convolution kernels of the first layer of convolution of the two sub-network characteristic extraction modules are different, and the rest modules are the same.
The feature extraction module is used for extracting a feature map of an input image, the fusion module is used for fusing convolution features of the last three layers of the feature extraction module, and the maximum pooling module is used for normalizing the fusion feature scale.
In the embodiment of the invention, the feature extraction module adopts a residual error network structure, preferably ResNet 18. Further, the second layer of ResNet18 is changed from the maximum pooling layer (down-sampling) to convolution, so that the network automatically learns the proper sampling kernel function; the last layer of the ResNet18 is changed into a convolution layer from a full connection layer and then connected with a maximum value pooling module, so that the network meets the image input requirements of different sizes; the parameters of the last three layers of the ResNet18 are modified so that the feature sizes of the last three layers are identical. The modified network is referred to as ResNet18v 2.
And the multi-layer convolution feature map fusion in the feature fusion module has better effect on multi-scale region matching. Assuming that the last three feature layers of the a network are Conv _ a1, Conv _ a2, Conv _ A3, and the last three feature layers of the B network are Conv _ B1, Conv _ B2, and Conv _ B3, the fusion feature Conv _ A, Conv _ B is:
Figure GDA0003064687370000071
Figure GDA0003064687370000072
wherein, w1＝2，w2＝4，w3＝6。
And performing block maximum pooling on the feature maps to enable the network to adapt to the input of template maps and to-be-matched maps with any sizes, and improving the processing speed, wherein the block maximum pooling means that feature maps with fixed sizes are obtained by using a maximum pooling method for the feature maps with different sizes.
The area matching network includes: the system comprises a feature division module, a classification module and a position regression module, wherein the classification module is connected with the position regression module in parallel, processed in parallel and connected behind the feature division module in series
The feature division module includes: the template matching method comprises a first convolution, a second convolution, a third convolution and a fourth convolution, wherein the first convolution is used for extracting a template classification feature map from a feature map of the template map, the second convolution is used for extracting a template position feature map from the feature map of the template map, the third convolution is used for extracting a to-be-matched map classification feature map, and the fourth convolution is used for extracting a to-be-matched map position feature map.
The classification module is used for using the template classification characteristic graph as a convolution kernel to be convolved with the classification characteristic graph of the graph to be matched and outputting matched classes including matching classes and unmatching classes.
And the position regression module is used for convolving the template position characteristic graph serving as a convolution kernel with the position characteristic graph of the graph to be matched and outputting a matched position.
Balanced loss function (HCE + HSL)
Classification loss function HCE
Consider a simple two-class cross entropy loss function (binary cross entropy loss):
Figure GDA0003064687370000081
Figure GDA0003064687370000082
wherein x is the initial class output of the HES-RPN network, p is the positive sample class probability of the network output, and the value range of p is [0, 1 ]]，p*Is a corresponding tag, p*The value is 0 or 1. The output position and the actual label intersection ratio IOU is more than 0.7, then p*1, otherwise, p*＝0。
Its gradient (derivative) to x is:
Figure GDA0003064687370000083
Figure GDA0003064687370000084
a gradient mode length can then be defined,
Figure GDA0003064687370000086
i denotes a sample i. Specifically, the range of values of the gradient modulo length is divided into a number of unit areas. For a sample, if its gradient mode length is g, its density is defined as the number of samples in the unit area where it is located divided by the length of this unit area, ε:
Figure GDA0003064687370000085
Figure GDA0003064687370000091
Figure GDA0003064687370000092
Figure GDA0003064687370000093
the initial value of alpha is 1, and then gradually rises to 2 along with the training process. k represents a sample k; n represents the total number of samples, and ε is a very small number, typically 0.01. And the reciprocal of the gradient density is the weight to be multiplied after the sample computation loss, the loss of the new classification is:
Figure GDA0003064687370000094
position regression loss HSL
Whereas the conventional commonly used regression branch loss function Smooth L1 is a piecewise function:
Figure GDA0003064687370000095
then smoothL is used when the distance deviation of the sample from the label is large, i.e. | x | ≧ 11The derivative of the function is constant at 1, and the influence on the update of the network parameters is the same, so that the specific difficulty of the sample cannot be distinguished specifically. To solve this problem, DSL losses are introduced:
Figure GDA0003064687370000096
Figure GDA0003064687370000097
d＝|t-t*|＝|(x0-xG)+(yo-yG)+(wo-wG)+(ho-hG)|
r is a position regression loss mark, and is convenient to distinguish from classification loss. The new position regression loss HSL is then expressed as:
Figure GDA0003064687370000098
wherein the content of the first and second substances,
Figure GDA0003064687370000101
the initial value of beta is 1, and then gradually rises to 1.5 along with the training process. u is a decimal number, typically 0.02, and t represents the sample position of the network output
Figure GDA0003064687370000102
xoIs shown in the graph P to be matchedNMAbscissa, y, of the center of the output area of the medium heterogeneous twin area selection networkoIs shown in the graph P to be matchedNMLongitudinal coordinate, w, of the center of the output area of the medium heterogeneous twin area selection networkoIs shown in the graph P to be matchedNMWidth of output area of medium heterogeneous twin area selection network, hoIs shown in the graph P to be matchedNMThe medium heterogeneous twin region selects the length of the output region of the network. t is t*Is the corresponding label, i.e. the actual position of the specimen
Figure GDA0003064687370000103
As shown in fig. 4, an image matching method based on a heterogeneous twin region selection network includes the following steps:
s1, training a heterogeneous twin region selection network by using a training sample, wherein the training sample is a template picture-picture to be matched pair, and a label of the training sample is the position of a region corresponding to the template picture in a corresponding region in the picture to be matched;
and S2, inputting the sample to be tested into the trained heterogeneous twin region selection network, and outputting the matching result of the sample to be tested.
Image matching is performed using a network. And selecting any two images with unlimited categories as templates and images to be matched, inputting the images into the trained image matching heterogeneous twin region selection network in an image pair mode, and obtaining a matching result.
FIG. 5(a) is a template map to be tested, and FIG. 5(b) is a matching result of the sample to be tested.
The heterogeneous twin region selection network in the invention and SiameseFC, SiamRPN + +, gray level cross-correlation and HOG matching in the prior art are respectively adopted to match the same data set, and the matching accuracy and the running time are shown in table 1.
TABLE 1
Figure GDA0003064687370000104
Figure GDA0003064687370000111
The running time refers to the time required for completing one matching under the condition that the size of the template graph is 127 × 127 and the size of the graph to be matched is 512 × 512.
After step S1 and before step S2, the heterogeneous twin region selection network may be optimized by using the sample to be tested.
Inputting the template graph and the graph to be matched in the test sample set into the network, calculating the cross-over ratio of the output result and the label, measuring the matching success probability by the cross-over ratio, evaluating the performance of the image matching network and determining whether to continue training the network.
Network output results
Figure GDA0003064687370000117
And actual label
Figure GDA0003064687370000118
The specific formula of the cross-over ratio IOU is as follows:
Figure GDA0003064687370000112
Figure GDA0003064687370000113
Figure GDA0003064687370000114
oarea＝wo*ho
garea＝wG*hG
and (4) regarding the image with the intersection ratio IOU being more than 0.3 as successful matching, and calculating the ratio of the successfully matched image to all the images, namely the success ratio of matching, wherein the higher the success ratio is, the better the matching performance is.
Set test set scenario
Figure GDA0003064687370000115
The total number of the images is T, wherein the intersection ratio of the A images is more than 0.3, and then the matching success rate SR (success rate) is:
Figure GDA0003064687370000116
along with the training process, if the SR does not rise any more, the network performance is not promoted any more, the network training is stopped, otherwise, the network training is continued.
The above description is only for the preferred embodiment of the present application, but the scope of the present application is not limited thereto, and any changes or substitutions that can be easily conceived by those skilled in the art within the technical scope of the present application should be covered within the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims (8)

1. An image matching method based on a heterogeneous twin region selection network is characterized by comprising the following steps:
s1, training an isomeric twin region selection network by using a training sample, wherein the training sample is a template picture-picture to be matched pair, a label of the training sample is position information of a region corresponding to the template picture in the region corresponding to the picture to be matched, and the isomeric twin region selection network comprises an isomeric twin network and a region matching network which are sequentially connected in series; the heterogeneous twin network is used for extracting a characteristic diagram of the template diagram and a characteristic diagram of a diagram to be matched; the area matching network is used for obtaining an area matching result according to the characteristic diagram of the template diagram and the characteristic diagram of the diagram to be matched; the heterogeneous twin network comprises a sub-network A and a sub-network B which are mutually connected in parallel, each sub-network comprises a feature extraction module, a feature fusion module and a maximum value pooling module which are sequentially connected in series, the two sub-networks only have different convolution kernels of the first layer of convolution of the feature extraction module, and the other sub-networks are the same;
and S2, inputting the sample to be tested into the trained heterogeneous twin region selection network, and outputting the matching result of the sample to be tested.
2. The image matching method of claim 1, wherein the feature extraction module is configured to extract a feature map of the input image, the feature fusion module is configured to fuse convolution features of the last three layers of the feature extraction module, and the maximum pooling module normalizes a scale of the fused features.
3. The image matching method of claim 1, wherein the feature extraction module replaces the ResNet18 second layer with a convolution.
4. The image matching method of claim 1, wherein the feature extraction module replaces the last layer of the ResNet18 with a convolutional layer.
5. The image matching method of claim 1, wherein the area matching network comprises: the system comprises a feature division module, a classification module and a position regression module; the classification module and the position regression module are connected in parallel, processed in parallel and connected in series behind the characteristic division module; the feature division module includes: a first convolution, a second convolution, a third convolution and a fourth convolution; the first convolution is used for extracting a template classification feature map from the feature map of the template map; the second convolution is used for extracting a template position feature map from the feature map of the template map; the third convolution is used for extracting a classification characteristic diagram of the image to be matched from the characteristic diagram of the image to be matched; the fourth convolution is used for extracting the position characteristic graph of the graph to be matched from the characteristic graph of the graph to be matched; the classification module is used for convolving the template classification characteristic graph with the classification characteristic graph of the graph to be matched by using the template classification characteristic graph as a convolution kernel and outputting a matched class; and the position regression module is used for convolving the template position characteristic graph serving as a convolution kernel with the position characteristic graph of the graph to be matched and outputting a matched position.
6. The image matching method of any one of claims 1 to 5, wherein the output position and the actual label have an intersection ratio IOU >0.7, the sample label p is 1, otherwise the sample label p is 0.
7. The image matching method of any one of claims 1 to 5, wherein after step S1 and before step S2, the heterogeneous twin region selection network is further optimized by using the sample to be tested, specifically as follows:
inputting the template graph and the graph to be matched in the test sample set into the network, calculating the cross-over ratio of the output result and the label, measuring the matching success probability by the cross-over ratio, evaluating the performance of the image matching network and determining whether to continue training the network.
8. A computer-readable storage medium, having stored thereon a computer program which, when being executed by a processor, implements the image matching method based on a heterogeneous twin region selection network according to any one of claims 1 to 7.
CN201910376172.1A 2019-05-07 2019-05-07 Image matching method based on heterogeneous twin region selection network Active CN110245678B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201910376172.1A CN110245678B (en) 2019-05-07 2019-05-07 Image matching method based on heterogeneous twin region selection network

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201910376172.1A CN110245678B (en) 2019-05-07 2019-05-07 Image matching method based on heterogeneous twin region selection network

Publications (2)

Publication Number Publication Date
CN110245678A CN110245678A (en) 2019-09-17
CN110245678B true CN110245678B (en) 2021-10-08

Family

ID=67883642

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201910376172.1A Active CN110245678B (en) 2019-05-07 2019-05-07 Image matching method based on heterogeneous twin region selection network

Country Status (1)

Country Link
CN (1) CN110245678B (en)

Families Citing this family (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110807793B (en) * 2019-09-29 2022-04-22 南京大学 Target tracking method based on twin network
CN110705479A (en) * 2019-09-30 2020-01-17 北京猎户星空科技有限公司 Model training method, target recognition method, device, equipment and medium
US11625834B2 (en) * 2019-11-08 2023-04-11 Sony Group Corporation Surgical scene assessment based on computer vision
CN111428875A (en) * 2020-03-11 2020-07-17 北京三快在线科技有限公司 Image recognition method and device and corresponding model training method and device
CN111401384B (en) * 2020-03-12 2021-02-02 安徽南瑞继远电网技术有限公司 Transformer equipment defect image matching method
CN111489361B (en) * 2020-03-30 2023-10-27 中南大学 Real-time visual target tracking method based on deep feature aggregation of twin network
CN111784644A (en) * 2020-06-11 2020-10-16 上海布眼人工智能科技有限公司 Printing defect detection method and system based on deep learning
CN112150467A (en) * 2020-11-26 2020-12-29 支付宝(杭州)信息技术有限公司 Method, system and device for determining quantity of goods
CN112785371A (en) * 2021-01-11 2021-05-11 上海钧正网络科技有限公司 Shared device position prediction method, device and storage medium
CN113705731A (en) * 2021-09-23 2021-11-26 中国人民解放军国防科技大学 End-to-end image template matching method based on twin network
CN115330876B (en) * 2022-09-15 2023-04-07 中国人民解放军国防科技大学 Target template graph matching and positioning method based on twin network and central position estimation
CN115861595A (en) * 2022-11-18 2023-03-28 华中科技大学 Multi-scale domain self-adaptive heterogeneous image matching method based on deep learning

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2017219515A1 (en) * 2016-06-22 2017-12-28 中兴通讯股份有限公司 Method for photographing focusing and electronic device
CN108388927A (en) * 2018-03-26 2018-08-10 西安电子科技大学 Small sample polarization SAR terrain classification method based on the twin network of depth convolution
CN109446889A (en) * 2018-09-10 2019-03-08 北京飞搜科技有限公司 Object tracking method and device based on twin matching network
CN109712121A (en) * 2018-12-14 2019-05-03 复旦大学附属华山医院 A kind of method, equipment and the device of the processing of medical image picture

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9953217B2 (en) * 2015-11-30 2018-04-24 International Business Machines Corporation System and method for pose-aware feature learning

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2017219515A1 (en) * 2016-06-22 2017-12-28 中兴通讯股份有限公司 Method for photographing focusing and electronic device
CN108388927A (en) * 2018-03-26 2018-08-10 西安电子科技大学 Small sample polarization SAR terrain classification method based on the twin network of depth convolution
CN109446889A (en) * 2018-09-10 2019-03-08 北京飞搜科技有限公司 Object tracking method and device based on twin matching network
CN109712121A (en) * 2018-12-14 2019-05-03 复旦大学附属华山医院 A kind of method, equipment and the device of the processing of medical image picture

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
High Performance Visual Tracking with Siamese Region Proposal Network;Bo Li et al.;《2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition》;20181217;第8971-8980页 *
基于小样本学习的目标匹配研究;柳青林;《中国优秀硕士学位论文全文数据库 信息科技辑》;20190215;I138-1627 *

Also Published As

Publication number Publication date
CN110245678A (en) 2019-09-17

Similar Documents

Publication Publication Date Title
CN110245678B (en) Image matching method based on heterogeneous twin region selection network
CN109271856B (en) Optical remote sensing image target detection method based on expansion residual convolution
CN108229468B (en) Vehicle appearance feature recognition and vehicle retrieval method and device, storage medium and electronic equipment
CN106504248B (en) Vehicle damage judging method based on computer vision
CN108960211B (en) Multi-target human body posture detection method and system
CN113065558A (en) Lightweight small target detection method combined with attention mechanism
CN112270249A (en) Target pose estimation method fusing RGB-D visual features
CN110163213B (en) Remote sensing image segmentation method based on disparity map and multi-scale depth network model
CN111222395A (en) Target detection method and device and electronic equipment
CN110796009A (en) Method and system for detecting marine vessel based on multi-scale convolution neural network model
CN110969166A (en) Small target identification method and system in inspection scene
CN108171249B (en) RGBD data-based local descriptor learning method
CN110309843B (en) Automatic identification method for multiple types of components in power equipment image
CN113159043A (en) Feature point matching method and system based on semantic information
CN113610905A (en) Deep learning remote sensing image registration method based on subimage matching and application
CN113095371A (en) Feature point matching method and system for three-dimensional reconstruction
CN117218201A (en) Unmanned aerial vehicle image positioning precision improving method and system under GNSS refusing condition
CN116385958A (en) Edge intelligent detection method for power grid inspection and monitoring
Cui et al. Global propagation of affine invariant features for robust matching
CN113011438A (en) Node classification and sparse graph learning-based bimodal image saliency detection method
CN110458234B (en) Vehicle searching method with map based on deep learning
CN117351078A (en) Target size and 6D gesture estimation method based on shape priori
CN115588178B (en) Automatic extraction method for high-precision map elements
CN115049842B (en) Method for detecting damage of aircraft skin image and positioning 2D-3D
Yan et al. Depth-only object tracking

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant