CN113240015A - Image matching method combined with deep learning - Google Patents
Image matching method combined with deep learning Download PDFInfo
- Publication number
- CN113240015A CN113240015A CN202110540653.9A CN202110540653A CN113240015A CN 113240015 A CN113240015 A CN 113240015A CN 202110540653 A CN202110540653 A CN 202110540653A CN 113240015 A CN113240015 A CN 113240015A
- Authority
- CN
- China
- Prior art keywords
- deep learning
- image
- region
- sub
- image matching
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 46
- 238000013135 deep learning Methods 0.000 title claims abstract description 35
- 238000003062 neural network model Methods 0.000 claims abstract description 20
- 238000013136 deep learning model Methods 0.000 claims abstract description 14
- 238000012549 training Methods 0.000 claims abstract description 11
- 239000013598 vector Substances 0.000 claims description 11
- 238000012360 testing method Methods 0.000 claims description 9
- 238000012795 verification Methods 0.000 claims description 9
- 230000006870 function Effects 0.000 claims description 5
- 238000002372 labelling Methods 0.000 claims description 5
- 230000011218 segmentation Effects 0.000 claims description 5
- 238000012545 processing Methods 0.000 claims description 4
- 230000000737 periodic effect Effects 0.000 claims description 3
- 238000011176 pooling Methods 0.000 claims description 3
- 230000004913 activation Effects 0.000 claims description 2
- 210000005036 nerve Anatomy 0.000 claims description 2
- 239000010410 layer Substances 0.000 description 14
- 208000037170 Delayed Emergence from Anesthesia Diseases 0.000 description 4
- 230000008569 process Effects 0.000 description 4
- 238000010586 diagram Methods 0.000 description 2
- 239000011159 matrix material Substances 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000013145 classification model Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 230000001537 neural effect Effects 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 238000005192 partition Methods 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 239000002356 single layer Substances 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/22—Matching criteria, e.g. proximity measures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/60—Rotation of whole images or parts thereof
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/11—Region-based segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20112—Image segmentation details
- G06T2207/20132—Image cropping
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Computation (AREA)
- Bioinformatics & Computational Biology (AREA)
- Evolutionary Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Image Analysis (AREA)
Abstract
The invention relates to the technical field of computer vision, in particular to an image matching method combined with deep learning. And inputting the image to be matched into the trained model to obtain a subregion where the image is located, and calculating by using an image matching algorithm to obtain the final position of the image. An image matching method combined with deep learning comprises the following steps: the method comprises the following steps: establishing a deep learning model; step two: training the deep learning model, and establishing a trained deep learning neural network model; step three: and carrying out image matching by using the trained deep learning neural network model.
Description
Technical Field
The invention relates to the technical field of computer vision, in particular to an image matching method combined with deep learning.
Background
Image matching is a method for searching for similar image objects by analyzing the correspondence, similarity and consistency of image contents, features, structures, relationships, textures, gray levels and the like. The invention provides an image matching method combining deep learning, which solves the problems that the existing image matching algorithm has high time complexity and long algorithm time consumption because the principle of the existing image matching algorithm needs line-by-line scanning, but the matching problem of a matching target in an original image after rotation or size change cannot be solved.
Disclosure of Invention
The invention aims to overcome the problems in the prior art and provides an image matching method combined with deep learning, wherein an image to be matched is input into a trained model to obtain a sub-region where the image is located, and then the final position of the image is calculated by using an image matching algorithm.
An image matching method combined with deep learning comprises the following steps:
the method comprises the following steps: establishing a deep learning model;
step two: training the deep learning model, and establishing a trained deep learning neural network model;
step three: and carrying out image matching by using the trained deep learning neural network model.
Further, the method for establishing the deep learning model comprises the following steps:
the method comprises the following steps: shooting through a camera lens to obtain the surface of the microstructure;
step two: performing row-by-row and column-by-column region segmentation on the microstructure surface image;
step three: labeling each sub-region to finally obtain a sub-region set;
step four: in each sub-region, continuously dividing the pattern into a plurality of pictures with smaller sizes, and enabling each sub-region to form a plurality of data sets under the label;
step five: in the data set under each sub-region label, a plurality of pictures are randomly extracted, and the labels of the pictures are reserved to form a test set and a verification set.
Further, in the second step of the method for establishing the deep learning model, the size of each region image in region segmentation is the same, and the region images are finally segmented into nxn sub-regions, wherein n is the number of segmented blocks in each row and each column. Labeling each sub-region to finally obtain { X11,X12.....X1n......XnnA set of sub-regions.
Further, the method for establishing the trained deep learning neural network model comprises the following steps:
and training the data set, the verification set and the test set as input layer input models of the deep learning framework, transmitting each microstructure pattern through nerve layer training, and finally accurately classifying the microstructure patterns into corresponding sub-pattern coordinates to obtain a trained neural network model.
Further, the microstructure surface is an object surface structure with a non-periodic pattern on the surface processed by using an ultra-precision processing technology, and the pattern has anisotropy, namely, the pattern shape is different at any position while the regularity is ensured, and the microstructure surface is not limited by the geometric dimension of the microstructure surface.
Further, the size of the data set, the number of pictures in the verification set and the test set can be expanded into the data set by the image enhancement of the sub-pattern image of each region.
Further, the image enhancement mode is cutting, rotating, translating and adding noise points.
Further, the method for performing image matching by using the trained deep learning neural network model comprises the following steps:
the method comprises the following steps: shooting by a camera lens to obtain an image to be matched;
step two: inputting a new picture to be matched into the trained neural network model, obtaining a sub-region after the new picture is transmitted and classified by the model, and obtaining the coordinate (x) of the sub-region through a sub-region label0,y0)；
Step three: in the subarea, corresponding image matching algorithm is applied to the image to be matched and the subarea image, and the coordinate value (x) of the matched image relative to the subarea image is calculated1,y1)；
Step four: finally, the position coordinate of the image to be matched is (x)0+x1,y0+y1)。
Further, the input received by the deep learning model is in a vector form, and the microstructure surface image and the subregion picture are both represented in the vector form.
Further, the neural network model for image classification includes a convolutional layer, an activation function, a pooling layer, and a full-link layer.
The image matching method combined with deep learning has the beneficial effects that:
and inputting the image to be matched into the trained model to obtain a subregion where the image is located, and calculating by using an image matching algorithm to obtain the final position of the image.
Drawings
FIG. 1 is a schematic block diagram of an overall process for establishing a trained deep learning neural network model according to the present invention;
fig. 2 is a schematic block diagram of the overall flow of the method for matching images by using the trained deep learning neural network model according to the present invention.
Detailed Description
An embodiment of the present invention will be described in detail below with reference to the accompanying drawings, but it should be understood that the scope of the present invention is not limited to the embodiment.
Specifically, as shown in fig. 1, an embodiment of the present invention provides a method for building a deep learning model, including the following steps:
firstly, the method comprises the following steps: shooting through an industrial camera lens to obtain a microstructure surface integral image of an image to be matched;
in the above step, the microstructure surface refers to an object surface structure with a non-periodic pattern on the surface processed by using an ultra-precision processing technology, and the pattern has anisotropy, that is, the pattern ensures regularity and different pattern shapes at any position, and is not limited by the geometric dimension of the microstructure surface. Other image data with non-periodicity and anisotropy can also be used, and the acquisition mode is not limited to the acquisition of an industrial camera.
Secondly, the method comprises the following steps: performing row-by-row and column-by-column region segmentation on the microstructure surface image; labeling each sub-region to finally obtain a sub-region set;
the size of each region image is the same, and the region images are finally divided into nxn sub-regions, wherein n is the number of blocks divided in each row and column. Labeling each sub-region to finally obtain { X11,X12.....X1n......XnnA set of sub-regions.
In the above steps, the number of the finally formed sub-regions should be determined according to the properties of the whole image and the requirements of image processing, and meanwhile, the subsequent training process is also affected.
Thirdly, in each sub-region, continuously dividing the pattern into a plurality of pictures with smaller sizes, and enabling each sub-region to form a plurality of data sets under the label; in the data set under each sub-region label, a plurality of pictures are randomly extracted, and the labels of the pictures are reserved to form a test set and a verification set.
In the above steps, the scale of the data set, the number of the pictures in the verification set and the test set are different according to different implementation modes, and the implementer can also determine the optimal scheme of the data set through experiments in the implementation process. The sub-pattern image of each region can also be expanded into a data set through image enhancement modes such as cropping, rotating, translating, adding noise points and the like.
Establishing a deep learning neural network model, training the data set, the verification set and the test set as an input layer input model of a deep learning frame, wherein each microstructure pattern is transmitted through the training of the neural layer and is finally accurately classified into corresponding sub-pattern coordinates. In this embodiment, ResNet50 is used as a main network.
In the above steps, the convolutional layer is divided into 5 parts, the first part is a single layer convolution of 7x7 convolution kernel, the second part is a three-layer residual block, the convolution kernel is 1x1, 3x3 and 1x1, the three times of convolution are repeated, the third part is the same three-layer residual block, the fourth part is the same residual block repeated six times, and the fifth part is a residual block repeated three times.
In the above steps, the feature vectors obtained by the convolutional layers are input into the pooling layer and the full-link layer for dimensionality reduction, and the output 512-dimensional feature vectors are input into the final classification layer.
And comparing the classification result with the classification label, calculating a loss value, and finally obtaining an image classification model with higher precision by reversely optimizing the loss function adjustment model, wherein the loss function uses a cross entropy loss function, and the optimization method uses random gradient descent.
In this embodiment, the input received by the deep learning model is in the form of a vector, and the microstructure image and the subregion picture are both represented in the form of a vector. Therefore, the input data can be more easily accepted by the model and is easy to extract.
An embodiment of the present invention will be described in detail below with reference to the accompanying drawings, but it should be understood that the scope of the present invention is not limited to the embodiment.
In this embodiment, the next image matching is performed according to the trained neural network model, as shown in fig. 2, the method includes the following steps:
firstly, an industrial camera lens shoots and acquires an image to be matched.
In the above steps, at this time, the field of view of the camera lens should be reduced, the smaller the range of the initial positioning, the smaller the search range of the subsequent matching, and the range of the initial positioning is determined by the size of the micro-structure surface partition region used for model training.
Secondly, inputting the new picture to be matched into the trained spiritObtaining the sub-region through the network model after being transferred and classified by the model, and obtaining the coordinate (x) of the sub-region through the sub-region label0,y0)。
Thirdly, in the subarea, corresponding image matching algorithm is applied to the image to be matched and the subarea image, and the coordinate value (x) of the matched image relative to the subarea image is calculated1,y1)。
In the above steps, the applied image matching algorithm is not limited to algorithms such as RPM, CPD, SM, MonteCarlo, GS, SIFT, SURF, PCA-SIFT, C-SIFT, ASIFT, DSP-SIFT, and the like.
Fourthly, the global position coordinate of the image to be matched is finally (x)0+x1,y0+y1)。
In the embodiment, the speeded up robust feature SURF algorithm is used for calculating the final sub-pattern image matching, so that the problem of image rotational matching is solved, the time complexity is reduced, and the calculation speed is increased.
The SURF algorithm flow is that a Hessian matrix and a Gaussian pyramid are constructed, each pixel point processed by the Hessian matrix is compared with 26 points of a 3-dimensional neighborhood, and if the pixel point is the maximum value of the 26 points, the pixel point is reserved and serves as a primary feature point.
And then determining the main direction of the feature points and constructing feature point descriptors, defining a description area around each feature point, wherein the area is composed of 4 multiplied by 4 sub-areas, and each sub-area comprises 5 multiplied by 5 pixel points. And after the image is rotated to the main direction of the pixel intensity gradient, obtaining a descriptor sub-vector of each sub-region through haar wavelet transformation, and combining different descriptor sub-vectors of 4 multiplied by 4 sub-regions into a 64-dimensional vector as an adjacent descriptor of the feature point. And finally, judging the similarity of the regions among the images through the Euclidean distance among the descriptors, and matching the region with the highest similarity among the images.
In the above steps, for the problem of mismatching of feature points in the matching process, a RANSAC algorithm may be used, according to the distribution design and constraint conditions of the feature points, interior points satisfying the constraints are defined, exterior points not satisfying the conditional constraints are defined, in a set of data including exterior points, a parameter model most suitable for the interior points is obtained by an iterative method, the number of the interior points and the number of the exterior points satisfying the requirements are recorded in the iteration, the maximum value of the number of the interior points is updated, and the result with the maximum number of the interior points is finally output.
The above disclosure is only for a few specific embodiments of the present invention, but the embodiments of the present invention are not limited thereto, and those skilled in the art can devise variations that fall within the scope of the present invention.
Claims (10)
1. An image matching method combined with deep learning is characterized by comprising the following steps:
the method comprises the following steps: establishing a deep learning model;
step two: training the deep learning model, and establishing a trained deep learning neural network model;
step three: and carrying out image matching by using the trained deep learning neural network model.
2. The image matching method combined with deep learning according to claim 1, wherein: the method for establishing the deep learning model comprises the following steps:
the method comprises the following steps: shooting through a camera lens to obtain the surface of the microstructure;
step two: performing row-by-row and column-by-column region segmentation on the microstructure surface image;
step three: labeling each sub-region to finally obtain a sub-region set;
step four: in each sub-region, continuously dividing the pattern into a plurality of pictures with smaller sizes, and enabling each sub-region to form a plurality of data sets under the label;
step five: in the data set under each sub-region label, a plurality of pictures are randomly extracted, and the labels of the pictures are reserved to form a test set and a verification set.
3. The image matching method combined with deep learning according to claim 2, wherein: in the second step of the method for establishing the deep learning model, the size of each region image in region segmentation is the same, and the region images are finally segmented into nxn sub-regions, wherein n is the number of segmented blocks in each row and each column.
4. The image matching method combined with deep learning according to claim 2, wherein: the method for establishing the trained deep learning neural network model comprises the following steps:
and training the data set, the verification set and the test set as input layer input models of the deep learning framework, transmitting each microstructure pattern through nerve layer training, and finally accurately classifying the microstructure patterns into corresponding sub-pattern coordinates to obtain a trained neural network model.
5. The image matching method combined with deep learning according to claim 2, wherein: the microstructure surface is an object surface structure with a non-periodic pattern on the surface processed by using an ultra-precision processing technology, and the pattern has anisotropy, namely, the pattern shape is different at any position while the regularity is ensured, and the microstructure surface is not limited by the geometric dimension of the microstructure surface.
6. The image matching method combined with deep learning according to claim 4, wherein: the scale of the data set, the number of pictures in the verification set and the test set can be expanded into the data set by the sub-pattern image of each region in an image enhancement mode.
7. The image matching method combined with deep learning according to claim 6, wherein: the image enhancement mode is cutting, rotating, translating and adding noise points.
8. The image matching method combined with deep learning according to claim 7, wherein: the method for matching the images by using the trained deep learning neural network model comprises the following steps:
the method comprises the following steps: shooting by a camera lens to obtain an image to be matched;
step two: inputting a new picture to be matched into the trained neural network model, obtaining a sub-region after the new picture is transmitted and classified by the model, and obtaining the coordinate (x) of the sub-region through a sub-region label0,y0)；
Step three: in the subarea, corresponding image matching algorithm is applied to the image to be matched and the subarea image, and the coordinate value (x) of the matched image relative to the subarea image is calculated1,y1)；
Step four: finally, the position coordinate of the image to be matched is (x)0+x1,y0+y1)。
9. The image matching method combined with deep learning according to claim 4, wherein: the input received by the deep learning model is in a vector form, and the microstructure surface image and the subregion picture are represented in the vector form.
10. The image matching method combined with deep learning according to any one of claims 1 to 9, wherein: the neural network model for image classification comprises a convolutional layer, an activation function, a pooling layer and a full-link layer.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110540653.9A CN113240015A (en) | 2021-05-18 | 2021-05-18 | Image matching method combined with deep learning |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110540653.9A CN113240015A (en) | 2021-05-18 | 2021-05-18 | Image matching method combined with deep learning |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113240015A true CN113240015A (en) | 2021-08-10 |
Family
ID=77135037
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110540653.9A Pending CN113240015A (en) | 2021-05-18 | 2021-05-18 | Image matching method combined with deep learning |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113240015A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114092301A (en) * | 2021-10-25 | 2022-02-25 | 海南大学 | Medical image robust multi-watermark algorithm research based on ShuffleNet transfer learning |
| CN115731436A (en) * | 2022-09-21 | 2023-03-03 | 东南大学 | Highway vehicle image retrieval method based on deep learning fusion model |
-
2021
- 2021-05-18 CN CN202110540653.9A patent/CN113240015A/en active Pending
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114092301A (en) * | 2021-10-25 | 2022-02-25 | 海南大学 | Medical image robust multi-watermark algorithm research based on ShuffleNet transfer learning |
| CN115731436A (en) * | 2022-09-21 | 2023-03-03 | 东南大学 | Highway vehicle image retrieval method based on deep learning fusion model |
| CN115731436B (en) * | 2022-09-21 | 2023-09-26 | 东南大学 | Highway vehicle image retrieval method based on deep learning fusion model |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10740897B2 (en) | Method and device for three-dimensional feature-embedded image object component-level semantic segmentation | |
| JP7417555B2 (en) | Human body pose analysis system and method | |
| CN110334762B (en) | Feature matching method based on quad tree combined with ORB and SIFT | |
| CN107358260B (en) | Multispectral image classification method based on surface wave CNN | |
| CN111178251B (en) | Pedestrian attribute identification method and system, storage medium and terminal | |
| CN106682700B (en) | Block rapid matching method based on key point description operator | |
| CN113240015A (en) | Image matching method combined with deep learning | |
| CN111369605B (en) | Infrared and visible light image registration method and system based on edge features | |
| CN111178190B (en) | Target detection method and device based on depth image and storage medium | |
| CN113592923B (en) | Batch image registration method based on depth local feature matching | |
| CN113449784B (en) | Image multi-classification method, device, equipment and medium based on priori attribute map | |
| CN111126412A (en) | Image key point detection method based on characteristic pyramid network | |
| Pei et al. | Effects of image degradations to cnn-based image classification | |
| CN104217459A (en) | Spherical feature extraction method | |
| CN111951283A (en) | Medical image identification method and system based on deep learning | |
| CN111199558A (en) | Image matching method based on deep learning | |
| CN115471682A (en) | Image matching method based on SIFT fusion ResNet50 | |
| Li et al. | Research on a product quality monitoring method based on multi scale PP-YOLO | |
| CN110516731B (en) | Visual odometer feature point detection method and system based on deep learning | |
| CN107798329A (en) | Adaptive particle filter method for tracking target based on CNN | |
| CN104268550A (en) | Feature extraction method and device | |
| Cheng et al. | Markov random field-based image inpainting with direction structure distribution analysis for maintaining structure coherence | |
| CN113191281A (en) | ORB feature extraction method based on region of interest and adaptive radius | |
| CN114170231A (en) | Image semantic segmentation method and device based on convolutional neural network and electronic equipment | |
| CN108985294B (en) | Method, device and equipment for positioning tire mold picture and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |