CN115731403A - Image alignment method and device, storage medium and electronic equipment - Google Patents
Image alignment method and device, storage medium and electronic equipment Download PDFInfo
- Publication number
- CN115731403A CN115731403A CN202111014707.4A CN202111014707A CN115731403A CN 115731403 A CN115731403 A CN 115731403A CN 202111014707 A CN202111014707 A CN 202111014707A CN 115731403 A CN115731403 A CN 115731403A
- Authority
- CN
- China
- Prior art keywords
- image
- matching
- determining
- matched
- descriptor
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 48
- 239000011159 matrix material Substances 0.000 claims abstract description 58
- 238000013519 translation Methods 0.000 claims abstract description 55
- 238000004422 calculation algorithm Methods 0.000 claims abstract description 44
- 238000005070 sampling Methods 0.000 claims abstract description 17
- 230000001131 transforming effect Effects 0.000 claims abstract description 13
- 230000008859 change Effects 0.000 claims description 31
- 230000009466 transformation Effects 0.000 claims description 31
- 238000012545 processing Methods 0.000 claims description 22
- 238000012216 screening Methods 0.000 claims description 10
- 238000004590 computer program Methods 0.000 claims description 9
- 238000013507 mapping Methods 0.000 claims 1
- 238000005259 measurement Methods 0.000 abstract description 11
- 210000000038 chest Anatomy 0.000 description 14
- 238000010586 diagram Methods 0.000 description 8
- 238000002474 experimental method Methods 0.000 description 7
- 230000006870 function Effects 0.000 description 6
- 239000004065 semiconductor Substances 0.000 description 6
- 238000000605 extraction Methods 0.000 description 5
- 230000008569 process Effects 0.000 description 5
- 206010007572 Cardiac hypertrophy Diseases 0.000 description 4
- 208000006029 Cardiomegaly Diseases 0.000 description 4
- 230000000694 effects Effects 0.000 description 4
- 238000012360 testing method Methods 0.000 description 4
- 238000001514 detection method Methods 0.000 description 3
- 238000007781 pre-processing Methods 0.000 description 3
- 238000002601 radiography Methods 0.000 description 3
- 238000011976 chest X-ray Methods 0.000 description 2
- 238000001914 filtration Methods 0.000 description 2
- 210000004072 lung Anatomy 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- PXFBZOLANLWPMH-UHFFFAOYSA-N 16-Epiaffinine Natural products C1C(C2=CC=CC=C2N2)=C2C(=O)CC2C(=CC)CN(C)C1C2CO PXFBZOLANLWPMH-UHFFFAOYSA-N 0.000 description 1
- 101001095088 Homo sapiens Melanoma antigen preferentially expressed in tumors Proteins 0.000 description 1
- 102100037020 Melanoma antigen preferentially expressed in tumors Human genes 0.000 description 1
- 238000007792 addition Methods 0.000 description 1
- 230000004075 alteration Effects 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 230000001174 ascending effect Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 230000000747 cardiac effect Effects 0.000 description 1
- 238000000354 decomposition reaction Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 201000010099 disease Diseases 0.000 description 1
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 description 1
- 208000019622 heart disease Diseases 0.000 description 1
- 230000036039 immunity Effects 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 238000005215 recombination Methods 0.000 description 1
- 230000006798 recombination Effects 0.000 description 1
Images
Landscapes
- Image Analysis (AREA)
Abstract
The invention provides an image alignment method, an image alignment device, a computer readable storage medium and an electronic device, wherein the method comprises the following steps: acquiring a reference image and an image to be matched; performing feature matching on the reference image and the image to be matched, and determining a matching point pair; determining an interior point pair based on the matching point pair and a random sampling consistency algorithm; and determining a translation matrix corresponding to the interior point pairs, and transforming the image to be matched based on the translation matrix. According to the technical scheme provided by the invention, the characteristic that the length-width ratio in distance measurement should be kept unchanged is considered, and the translation matrix corresponding to the inner point pair is utilized when the images to be matched are aligned and changed, so that the distance measurement result is more accurate after the images are aligned.
Description
Technical Field
The present invention relates to the field of artificial intelligence, and more particularly, to an image alignment method, apparatus, storage medium, and electronic device.
Background
The image alignment technology is a very important preprocessing step in the field of image processing, and a relatively accurate image alignment method is required in applications such as image classification and regression, and the current image alignment technology often causes the change of image proportion, so that the current image alignment technology is not suitable for applications such as distance measurement.
Disclosure of Invention
The invention provides an image alignment method, an image alignment device, a computer readable storage medium and electronic equipment, and aims to solve the technical problem that the existing image alignment method is not suitable for distance measurement and other applications.
According to a first aspect of the present invention, there is provided an image alignment method comprising:
acquiring a reference image and an image to be matched;
performing feature matching on the reference image and the image to be matched, and determining a matching point pair;
determining an interior point pair based on the matching point pair and a random sampling consistency algorithm;
and determining a translation matrix corresponding to the interior point pairs, and transforming the image to be matched based on the translation matrix.
According to a second aspect of the present invention, there is provided an image alignment apparatus comprising:
the acquisition processing module is used for acquiring a reference image and an image to be matched;
the matching processing module is used for carrying out feature matching on the reference image and the image to be matched and determining a matching point pair;
the interior point determining module is used for determining interior point pairs based on the matching point pairs and a random sampling consistency algorithm;
and the transformation processing module is used for determining a translation matrix corresponding to the inner point pairs and transforming the image to be matched based on the translation matrix.
According to a third aspect of the present invention, there is provided a computer-readable storage medium storing a computer program for performing the above-described method.
According to a fourth aspect of the present invention, there is provided an electronic apparatus comprising:
a processor;
a memory for storing the processor-executable instructions;
the processor is used for reading the executable instructions from the memory and executing the instructions to realize the method.
Compared with the prior art, the image alignment method, the image alignment device, the computer readable storage medium and the electronic equipment provided by the invention at least have the following beneficial effects:
according to the technical scheme, the reference image and the image to be matched are obtained, feature matching is conducted on the reference image and the image to be matched, the matching point pair is obtained, the interior point pair is determined in the matching point pair through a random sampling consistency algorithm, the random sampling consistency algorithm can effectively screen the matching point pair, outliers in the matching point pair are processed, matching accuracy of the determined interior point pair is higher, a translation matrix is determined according to the interior point pair, the translation matrix is used for conducting alignment transformation on the image to be matched, the length-width ratio of the image to be matched is kept unchanged in the transformation process, the distance measurement result is accurate after the images are aligned, and the method and the device are particularly suitable for a measurement task of critical dimensions.
Drawings
In order to more clearly illustrate the technical solution of the present invention, the drawings required to be used in the description of the present invention will be briefly introduced below, and it is apparent that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art that other drawings may be obtained according to the drawings without inventive labor.
Fig. 1 is a first flowchart illustrating an image alignment method according to an exemplary embodiment of the present invention;
FIG. 2 is a second flowchart illustrating an image alignment method according to an exemplary embodiment of the present invention;
FIG. 3 is a schematic view of an X-ray chest film provided in accordance with an exemplary embodiment of the present invention;
FIG. 4 is a third flowchart illustrating an image alignment method according to an exemplary embodiment of the present invention;
FIG. 5 is a fourth flowchart illustrating an image alignment method according to an exemplary embodiment of the present invention;
FIG. 6 is a flowchart illustrating a fifth method for aligning images according to an exemplary embodiment of the present invention;
FIG. 7 is a diagram illustrating matching results and transformation results of an image alignment method according to an exemplary embodiment of the present invention;
FIG. 8 is an enlarged diagram of a transformation result of an image alignment method according to an exemplary embodiment of the present invention;
fig. 9 is a schematic structural diagram of an image alignment apparatus according to an exemplary embodiment of the present invention;
fig. 10 is a block diagram of an electronic device provided in an exemplary embodiment of the invention.
Detailed Description
The technical solutions in the present invention will be described clearly and completely with reference to the accompanying drawings, and it is obvious that the described embodiments are some, not all embodiments of the present invention. All other embodiments, which can be obtained by a person skilled in the art without any inventive step based on the embodiments of the present invention, belong to the protection scope of the present embodiments.
Exemplary method
Fig. 1 is a flowchart illustrating an image alignment method according to an exemplary embodiment of the present invention. The embodiment can be applied to electronic equipment, and particularly can be applied to a server or a general computer. At least comprises the following steps:
and step 10, acquiring a reference image and an image to be matched.
In one embodiment, the reference image refers to an aligned image, and the image to be matched is an aligned image. Specifically, the reference image may be preselected by an expert based on experience, and remains unchanged in a subsequent matching process, and the image to be matched may be an image acquired in real time and requiring alignment preprocessing. In a possible application scenario, the image to be matched is used for critical dimension measurement, for example, the reference image is a semiconductor reference image, the image to be matched is a semiconductor image to be matched, or the reference image is a medical reference image, and the image to be matched is a medical image to be matched, where both the semiconductor image to be matched and the medical image to be matched have a characteristic that an aspect ratio is kept unchanged.
And 20, performing feature matching on the reference image and the image to be matched, and determining a matching point pair.
In one embodiment, the matching point pairs refer to points that match in the reference image and the image to be matched, where the number of the matching point pairs is plural, and each matching point pair includes one reference point in the reference image and one feature point in the image to be matched. In this embodiment, the algorithm for image feature detection and feature matching is not specifically limited, as long as the reference image and the image to be matched can be subjected to feature matching, for example, SURF, SIFT, ORB, FAST, and Harris corners may be used.
And step 30, determining an interior point pair based on the matching point pair and a random sampling consistency algorithm.
In one embodiment, a random sampling consistency algorithm is introduced to process the matching point pairs so as to process most outliers in the matching point pairs to obtain interior point pairs. That is to say, the interior point pairs are a part of the matching point pairs, the number of the interior point pairs is multiple, and the matching accuracy of the interior point pairs obtained after the processing of the random sampling consistency algorithm is higher.
Specifically, a homography transformation matrix between the point pairs is calculated based on the matching point pairs and a random sampling consistency algorithm, and interior point pairs are determined according to the obtained homography transformation matrix. The random sampling consistency algorithm inputs a group of inner point pairs and outputs model parameters with the best fitting effect on data, and calculates a homography matrix H of 3 x 3 containing 8 degrees of freedom in the following calculation mode:
wherein s represents a scaling factor, and [ x ', y',1] and [ x, y,1] are respectively corresponding alignment coordinates of the matching points. And finally, after the optimal model parameters are determined, the inner point pairs are matching points which accord with the optimal model parameters.
And step 40, determining a translation matrix corresponding to the interior point pairs, and transforming the image to be matched based on the translation matrix.
In an embodiment, in the distance measurement, the aspect ratio of the image to be matched remains unchanged, and the determined homography matrix includes operations such as scaling, rotation, and translation, and image transformation such as scaling and rotation actually affects the aspect ratio of the image, so that subsequent use of the image to be matched is affected if the homography matrix is directly used for transformation.
In the above embodiment, a reference image and an image to be matched are obtained, feature matching is performed on the reference image and the image to be matched, a matching point pair is obtained, an interior point pair is determined in the matching point pair by using a random sampling consistency algorithm, the matching point pair can be effectively screened by using the random sampling consistency algorithm, an outlier in the matching point pair is processed, so that matching accuracy of the determined interior point pair is better, a translation matrix is determined according to the interior point pair, an aspect ratio should be kept unchanged in distance measurement, and if a matrix containing scaling and rotation is used for transforming the image to be matched, inaccuracy of distance measurement due to image alignment is caused.
As shown in fig. 2, in an exemplary embodiment of the present invention, the step 40 of determining a translation matrix corresponding to the interior point pair, and transforming the image to be matched based on the translation matrix includes:
In the above embodiment, when determining the translation matrix, the abscissa and the ordinate of the interior point pair are mainly considered to ensure that the aspect ratio of the aligned transformed image to be matched is kept unchanged. Specifically, an abscissa and an ordinate corresponding to a reference point in a reference image in the interior point pair and an abscissa and an ordinate corresponding to a feature point in an image to be matched are respectively determined, and then an abscissa transformation value tx and an ordinate transformation value ty are calculated, and a translation transformation matrix M is determined, where M may be as follows:
affine transformation is carried out on the image to be matched by using the M, the aligned image to be matched is determined, and the length-width ratio of the image to be matched is consistent with that of the aligned image to be matched through transformation of the matrix M.
In a possible application scenario, after image alignment is completed for a mark point regression task, mark points are labeled, a regression model is trained to automatically identify the mark points, and the mark points identified by the model can be used for measuring critical dimensions, such as determining the critical dimension of a semiconductor or determining the critical dimension of a medical image, by constructing a connecting line between the two points. For example, cardiac hypertrophy is a serious cardiac enlargement disease, which can be detected by chest radiography in early stage, and a cardiothoracic ratio (CTR) is calculated according to a preset heart and lung width ratio, where CTR is a ratio of a maximum horizontal heart diameter to a maximum horizontal thorax diameter (inner rib margin), and if the CTR value is greater than 0.5, it indicates that there is a cardiac hypertrophy problem in the chest X-ray image, as shown in fig. 3, 4 marked points marked on the chest X-ray image are shown, and CTR is calculated by using the measurement value L, R, T in fig. 3 and the formula 3, where the formula 3 is as follows:
wherein L represents the distance of the left heart wall from the center line, R represents the distance of the right heart wall from the center line, and T represents the lung width. When the images are aligned, the aspect ratio of the images needs to be kept unchanged, otherwise, after the images are aligned and marked points are marked, the obtained CTR value is inaccurate, and the problem exists in the result of the marked point regression task.
As shown in fig. 4, in an exemplary embodiment of the present invention, in step 402, determining a translation matrix corresponding to the interior point pair based on the abscissa variation value and the ordinate variation value, and transforming the image to be matched based on the translation matrix includes:
And 4023, determining a translation matrix corresponding to the inner point pair based on the horizontal coordinate change median and the vertical coordinate change median, and transforming the image to be matched based on the translation matrix.
In the above embodiment, when determining the translation matrix based on the abscissa change value and the ordinate change value, the abscissa transformation median is determined according to the abscissa transformation value, the ordinate transformation median is determined according to the ordinate change value, then the translation matrix corresponding to the interior point pair is determined according to the abscissa transformation median and the ordinate change median, and the image to be matched is transformed by using the translation matrix. Specifically, the translated image is calculated by a warpAffeine function based on median mean.tx and median.ty of abscissa change value tx and ordinate change value ty, wherein the transformation function utilizes a predefined 2 × 3 translation matrix M. In this embodiment, the abscissa transformation median is determined according to the abscissa variation value, the ordinate variation median is determined according to the ordinate variation value, and the abscissa variation median and the ordinate variation median are used as the optimal values to determine that the translation matrix has better accuracy, so that the alignment effect is ensured.
As shown in fig. 5, on the basis of fig. 1, in an exemplary embodiment of the present invention, the step 20 performs feature matching on the reference image and the image to be matched, and determines a matching point pair, including:
In an embodiment, feature detection is performed on a reference image, image features of the reference image are extracted, so as to obtain a first key point and a first descriptor, wherein the first key point refers to a plurality of detected feature points in the reference image, the first descriptor is a vector result obtained by further processing the first key point, and is used for describing the first key point, and the first descriptor is lower in dimensionality and easier to detect. Specifically, the ORB technology is based on a BRIEF algorithm (binary robust independent basic feature algorithm), has rotation invariance and noise immunity, can adjust the maximum number of search features in an image, and can define the percentage of successful matching between detected features, and the maximum number and the percentage can be adjusted to control the feature matching level between the reference image and the image to be matched, so that a user can adjust the maximum number and the percentage according to actual application, and better matching results can be obtained subsequently.
In an embodiment, feature detection is performed on an image to be matched, image features of the image to be matched are extracted, so that a plurality of second key points and a plurality of second descriptors are obtained, the second key points refer to the detected feature points in the image to be matched, the second descriptors are vector results obtained by further processing the second key points and are used for describing the second key points, and the second descriptors are lower in dimensionality and easier to detect. Specifically, an ORB fast binary descriptor is used to extract image features of the image to be matched, and a second key point and a second descriptor are obtained.
In the above embodiment, after the first keypoint, the first descriptor, the second keypoint, and the second descriptor are determined, feature matching is performed according to the first keypoint, the first descriptor, the second keypoint, and the second descriptor, so as to determine a matching point pair matching between the first keypoint and the second keypoint.
As shown in fig. 6, on the basis of fig. 5, in an exemplary embodiment of the present invention, the step 203 of performing feature matching based on the first keypoint, the first descriptor, the second keypoint, and the second descriptor, and determining a matching point pair includes:
In one embodiment, feature matching is performed using a hamming distance based brute force matching algorithm. Specifically, for a first descriptor corresponding to one first key point in the reference image, the hamming distance of second descriptors corresponding to all second key points in the image to be matched is calculated, and in the second key points with the hamming distance smaller than the threshold value, the second descriptor with the closest hamming distance is returned as the first matching result.
Of course, a combination of brute force matching and cross matching may be used in determining the first matching result. Specifically, a hamming distance-based brute force matching algorithm is used for calculating the similar distance between a first descriptor corresponding to each first key point and a second descriptor corresponding to each second key point for matching, and a second matching result corresponding to the first key point and the second key point is determined; and screening the second matching result by utilizing cross matching to determine a first matching result. That is, two parameters are set, parameter 1 is used to indicate the hamming distance threshold, parameter 2 is used to indicate the cross-matching module, and the default value is false. The first key points form a point set 1, the second key points form a point set 2, according to a first descriptor in the point set 1, the Hamming distance is used for matching with all second descriptors in the point set 2, if the features in the point set 2 can still be matched with the features in the point set 1 in reverse, the correct matching is considered, and the value of the cross matching module is set to be true. And finally, returning the key point pair with the closest Hamming distance as the first matching result by the violent matching algorithm.
In an embodiment, after the first matching result is determined, the first matching result is further filtered by using a preset matching threshold value to determine a matching point pair. That is to say, the determined matching point pair is the result of multiple screening, so that the accuracy of the result of the determined matching point pair is higher. Specifically, a matching threshold is preset, the returned first matching results are sorted in ascending order, and the matching results with low similarity are deleted according to the set matching threshold.
In a possible application scenario, the acquired reference image is a sensor reference image, the acquired image to be matched is a sensor image to be matched, a test set of the image alignment task is a set of data sets containing 458 sensor images, and the sensor image to be matched is aligned by using a customized ORB algorithm (experiment one). The method comprises the following specific steps:
step 1: initializing an image width and height and a matching threshold;
step 2: and screening the reference sensor image, and setting a threshold value for successful matching. Detecting ORB features of a reference sensor image, and calculating a first key point 1 and a first descriptor 1 in the point set;
and step 3: detecting ORB characteristics of the sensor image to be matched, and calculating a second key point 2 and a second descriptor 2 in the point set;
and 4, step 4: calculating the similarity distance between descriptors by using a violence matching algorithm and a Hamming distance, and matching;
and 5: filtering the matching results by using a matching threshold value, visually drawing the sorted matching results, evaluating strong characteristic points and determining matching point pairs;
and 6: extracting the positions of the points 1 and 2 in the matching point pairs, and calculating a homography transformation matrix between the point pairs by using a random sampling consistency algorithm, namely:
matrix H, mask = findhomograph function (point l, point 2, ransac algorithm);
and 7: acquiring inner point pairs in _ pts1 and in _ pts2 from the homography matrix H according to the returned mask;
and 8: calculating translation values, namely a horizontal coordinate change value tx and a vertical coordinate change value ty according to the inner point pair, and optimizing by taking a median value of the tx and ty values;
and step 9: and obtaining a final translation matrix according to the median mean _ tx and the median _ ty, and performing alignment transformation on the sensor image to be matched by using the translation matrix.
Meanwhile, a basic ORB algorithm and an ORB algorithm based on a K-nearest neighbor algorithm are utilized to align 458 sensor images to be matched under the same reference image, wherein the model precision is calculated according to the pixel deviation of the image to be matched and the reference image, the generally acceptable pixel deviation range is 2-3 points, and the obtained result is as follows:
| Method | number of images | Number of unaligned images | Precision% |
| Basic ORB algorithm | 458 | 15 | 96.72 |
| ORB algorithm based on K-nearest neighbor algorithm | 458 | 13 | 97.16 |
| Customized ORB algorithm | 458 | 0 | 100 |
Experiment two was performed based on another set of semiconductor images, the test model was the same as experiment one, the reference images were based on the same distribution of different data, and the result was 6 misaligned images out of 62 sensor images with a 90.32% accuracy.
In one possible application scenario, model performance testing is performed on medical data, which is completely different from sensor images, with a higher requirement on the accuracy of the edge information. A set of X-ray Chest slices were extracted from the NIH Chest-Xray8 dataset at a resolution of 256X 256 and used to detect cardiac hypertrophy. The cardiac hypertrophy is a serious heart disease and can be identified by a cardiothoracic ratio (CTR) in an X-ray chest film, a feature extraction method of the task mainly depends on visible edges in an image, and in order to highlight fine edge information in the X-ray image, the image is subjected to data enhancement by using an edge enhancement technology and then is subjected to image alignment. It should be noted that whether image enhancement is required or not may be selected according to image quality. Specifically, the acquired reference image is an X-ray chest reference image, the acquired image to be matched is an X-ray chest image to be matched, the test set of the image alignment task is a data set containing 135X-ray chest images, and the X-ray chest image to be matched is aligned by using a customized ORB algorithm (experiment three). The specific steps are as follows:
step 1: initializing an image width and height and a matching threshold;
step 2: and screening the reference X-ray chest radiography image and setting a threshold value for successful matching. Detecting ORB characteristics of a reference X-ray chest film image, and calculating a first key point 1 and a first descriptor 1 in the point set;
and step 3: detecting ORB characteristics of the X-ray chest radiography image to be matched, and calculating a second key point 2 and a second descriptor 2 in the point set;
and 4, step 4: calculating the similarity distance between descriptors by using a violence matching algorithm and a Hamming distance, and matching;
and 5: filtering the matching results by using the matching threshold, visually drawing the sorted matching results, evaluating the strong characteristic points and determining matching point pairs;
step 6: extracting the positions of the points 1 and 2 in the matching point pairs, and calculating a homography transformation matrix between the point pairs by using a random sampling consistency algorithm, namely:
matrix H, mask = findhomograph function (point l, point 2, ransac algorithm);
and 7: acquiring an internal point pair in _ pts1 and in _ pts2 from the homography matrix H according to the returned mask;
and 8: calculating translation values, namely a horizontal coordinate change value tx and a vertical coordinate change value ty according to the inner point pair, and optimizing by taking a median value of the tx and ty values;
and step 9: and obtaining a final translation matrix according to the median mean _ tx and the median _ ty, and performing alignment transformation on the X-ray chest image to be matched by using the translation matrix.
The results show that 12 out of 135X-ray chest images are misaligned images with a precision of 91.1%, as shown in fig. 7, the left column in fig. 7 is the image to be matched, the middle is the best alignment result, and the right column is the translation transformation result, wherein fig. 8 is the enlargement result corresponding to the right column (1) (2) (3), and it can be seen from fig. 8 that (1) the bottom translation frame representation is vertically translated with respect to the reference image, (1) the left translation frame representation is horizontally translated with respect to the reference image, (2) the top translation frame representation is vertically translated with respect to the reference image, (2) the right translation frame representation is horizontally translated with respect to the reference image, (3) the bottom translation frame representation is vertically translated with respect to the reference image, and (3) the left translation frame representation is horizontally translated with respect to the reference image.
As can be seen from the first experiment, the second experiment and the third experiment, the customized ORB algorithm provided by the embodiment has better performance, and when the image features are stronger and have visibility, the method can obtain higher precision on the image alignment task. The pixel-level precision is crucial to a defect identification task based on measured values only, and the MAPE value (mean absolute percentage error) of the regression of the key mark points can be greatly reduced by integrating an image alignment algorithm in the preprocessing process.
Exemplary devices
Based on the same concept as the image alignment method provided by the embodiment of the invention, the embodiment of the invention also provides an image alignment device.
Fig. 9 is a schematic structural diagram of an image alignment apparatus according to an exemplary embodiment of the present invention, including:
the acquisition processing module 91 is used for acquiring a reference image and an image to be matched;
the matching processing module 92 is configured to perform feature matching on the reference image and the image to be matched, and determine a matching point pair;
an interior point determining module 93, configured to determine an interior point pair based on the matching point pair and a random sampling consistency algorithm;
and a transformation processing module 94, configured to determine a translation matrix corresponding to the interior point pair, and transform the image to be matched based on the translation matrix.
In an exemplary embodiment of the present invention, the transformation processing module includes:
a transformation value determining unit, configured to determine an abscissa change value and an ordinate change value corresponding to the pair of interior points, respectively, based on the abscissa and the ordinate corresponding to the pair of interior points, respectively;
and the transformation processing unit is used for determining a translation matrix corresponding to the inner point pair based on the abscissa change value and the ordinate change value and transforming the image to be matched based on the translation matrix.
In an exemplary embodiment of the present invention, the transformation processing unit is configured to determine the median abscissa variation based on the abscissa variation value; determining the median change of the ordinate based on the value of the change of the ordinate; and determining a translation matrix corresponding to the inner point pair based on the horizontal coordinate change median and the vertical coordinate change median, and transforming the image to be matched based on the translation matrix.
In an exemplary embodiment of the present invention, the matching processing module includes:
and the first extraction unit is used for extracting the image characteristics of the reference image and acquiring a first key point and a first descriptor. Or, the first extraction unit is configured to extract an image feature of the reference image by using the ORB fast binary descriptor, and obtain the first key point and the first descriptor.
And the second extraction unit is used for extracting the image characteristics of the image to be matched and acquiring a second key point and a second descriptor. Or, the second extraction unit is configured to extract image features of the image to be matched by using the ORB fast binary descriptor, and obtain the second key point and the second descriptor.
A matching processing unit, configured to perform feature matching based on the first keypoint, the first descriptor, the second keypoint, and the second descriptor, and determine a matching point pair.
In an exemplary embodiment of the present invention, the matching processing unit includes:
and the result determining subunit is used for calculating similar distances between first descriptors respectively corresponding to the first key points and second descriptors respectively corresponding to the second key points by using a hamming distance-based violent matching algorithm to perform matching, and determining first matching results corresponding to the first key points and the second key points. Or, the result determining subunit is configured to calculate, by using a brute force matching algorithm based on hamming distance, similar distances between first descriptors corresponding to the first keypoints and second descriptors corresponding to the second keypoints, and perform matching, and determine second matching results corresponding to the first keypoints and the second keypoints; and screening the second matching result by utilizing cross matching to determine a first matching result.
And the result screening subunit is used for screening the first matching result by using a preset matching threshold value to determine a matching point pair.
Exemplary electronic device
FIG. 10 illustrates a block diagram of an electronic device in accordance with an embodiment of the present invention.
As shown in fig. 10, the electronic device 100 includes one or more processors 101 and memory 102.
The processor 101 may be a Central Processing Unit (CPU) or other form of processing unit having data processing capabilities and/or instruction execution capabilities, and may control other components in the electronic device 100 to perform desired functions.
Memory 102 may include one or more computer program products that may include various forms of computer-readable storage media, such as volatile memory and/or non-volatile memory. The volatile memory may include, for example, random Access Memory (RAM), cache memory (cache), and/or the like. The non-volatile memory may include, for example, read Only Memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium and executed by the processor 101 to implement the image alignment methods of the various embodiments of the invention described above and/or other desired functions.
In one example, the electronic device 100 may further include: an input device 103 and an output device 104, which are interconnected by a bus system and/or other form of connection mechanism (not shown).
Of course, for simplicity, only some of the components of the electronic device 100 relevant to the present invention are shown in fig. 10, and components such as buses, input/output interfaces, and the like are omitted. In addition, electronic device 100 may include any other suitable components depending on the particular application.
Exemplary computer program product and computer-readable storage Medium
In addition to the above-described methods and apparatus, embodiments of the present invention may also be a computer program product comprising computer program instructions that, when executed by a processor, cause the processor to perform the steps in the image alignment method according to various embodiments of the present invention described in the "exemplary methods" section above of this specification.
The computer program product may write program code for carrying out operations for embodiments of the present invention in any combination of one or more programming languages, including an object oriented programming language such as Java, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computing device, partly on the user's device, as a stand-alone software package, partly on the user's computing device and partly on a remote computing device, or entirely on the remote computing device or server.
Furthermore, embodiments of the present invention may also be a computer-readable storage medium having stored thereon computer program instructions which, when executed by a processor, cause the processor to perform the steps in the image alignment method according to various embodiments of the present invention described in the "exemplary methods" section above in this specification.
The computer readable storage medium may take any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may include, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples (a non-exhaustive list) of the readable storage medium include: an electrical connection having one or more wires, a portable disk, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
The basic principles of the present invention have been described above with reference to specific embodiments, but it should be noted that the advantages, effects, etc. mentioned in the present invention are only examples and are not limiting, and the advantages, effects, etc. must not be considered to be possessed by various embodiments of the present invention. Furthermore, the foregoing detailed description of the invention is provided for the purpose of illustration and understanding only, and is not intended to be limiting, since the invention will be described in any way as it would be understood by one skilled in the art.
The block diagrams of devices, apparatuses, systems involved in the present invention are only given as illustrative examples and are not intended to require or imply that the connections, arrangements, configurations, etc. must be made in the manner shown in the block diagrams. These devices, apparatuses, devices, systems may be connected, arranged, configured in any manner, as will be appreciated by those skilled in the art. Words such as "including," "comprising," "having," and the like are open-ended words that mean "including, but not limited to," and are used interchangeably herein. The words "or" and "as used herein mean, and are used interchangeably with, the word" and/or, "unless the context clearly dictates otherwise. The word "such as" is used herein to mean, and is used interchangeably with, the phrase "such as but not limited to".
It should also be noted that in the apparatus, devices and methods of the present invention, the components or steps may be broken down and/or re-combined. These decompositions and/or recombinations are to be regarded as equivalents of the present invention.
The previous description of the inventive aspects is provided to enable any person skilled in the art to make or use the present invention. Various modifications to these aspects will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other aspects without departing from the scope of the invention. Thus, the present invention is not intended to be limited to the aspects shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
The foregoing description has been presented for purposes of illustration and description. Furthermore, this description is not intended to limit embodiments of the invention to the form disclosed herein. While a number of example aspects and embodiments have been discussed above, those of skill in the art will recognize certain variations, modifications, alterations, additions and sub-combinations thereof.
Claims (10)
1. An image alignment method, comprising:
acquiring a reference image and an image to be matched;
performing feature matching on the reference image and the image to be matched, and determining a matching point pair;
determining an interior point pair based on the matching point pair and a random sampling consistency algorithm;
and determining a translation matrix corresponding to the interior point pairs, and transforming the image to be matched based on the translation matrix.
2. The method of claim 1, wherein the determining the translation matrix to which the interior point pairs correspond comprises:
determining the abscissa change value and the ordinate change value respectively corresponding to the inner point pairs based on the abscissa and the ordinate respectively corresponding to the inner point pairs;
and determining a translation matrix corresponding to the inner point pair based on the abscissa variation value and the ordinate variation value.
3. The method according to claim 1, wherein the determining a translation matrix corresponding to the inner point pair based on the abscissa variation value and the ordinate variation value comprises:
determining the median change of abscissa based on the value of change of abscissa;
determining the median change of ordinate based on the value of change of ordinate;
and determining a translation matrix corresponding to the inner point pair based on the abscissa variation median value and the ordinate variation median value.
4. The method according to claim 1, wherein the performing feature matching on the reference image and the image to be matched and determining matching point pairs comprises:
extracting image features of the reference image, and acquiring a first key point and a first descriptor;
extracting image features of the image to be matched, and acquiring a second key point and a second descriptor;
and performing feature matching based on the first key point, the first descriptor, the second key point and the second descriptor to determine a matching point pair.
5. The method according to claim 4, wherein the extracting of the image features of the acquired reference image acquires a first key point and a first descriptor; extracting image features of the acquired sample image, and acquiring a second key point and a second descriptor, wherein the method comprises the following steps:
extracting image features of a reference image by using an ORB (object relational mapping) quick binary descriptor to obtain a first key point and a first descriptor;
and extracting the image features of the image to be matched by utilizing the ORB quick binary descriptor, and acquiring a second key point and a second descriptor.
6. The method of claim 5, wherein the determining a matching point pair based on feature matching of the first keypoint, the first descriptor, the second keypoint, and the second descriptor comprises:
calculating similar distances between first descriptors respectively corresponding to the first key points and second descriptors respectively corresponding to the second key points by using a hamming distance-based violent matching algorithm for matching, and determining first matching results corresponding to the first key points and the second key points;
and screening the first matching result by using a preset matching threshold value to determine a matching point pair.
7. The method of claim 6, wherein the determining the first matching result corresponding to the first keypoint and the second keypoint by calculating a similar distance between a first descriptor corresponding to the first keypoint and a second descriptor corresponding to the second keypoint using a hamming distance-based brute force matching algorithm comprises:
calculating similar distances between first descriptors corresponding to the first key points and second descriptors corresponding to the second key points respectively by using a force matching algorithm based on Hamming distance to perform matching, and determining second matching results corresponding to the first key points and the second key points;
and screening the second matching result by utilizing cross matching to determine a first matching result.
8. An image alignment apparatus, comprising:
the acquisition processing module is used for acquiring a reference image and an image to be matched;
the matching processing module is used for carrying out feature matching on the reference image and the image to be matched and determining a matching point pair;
the interior point determining module is used for determining interior point pairs based on the matching point pairs and a random sampling consistency algorithm;
and the transformation processing module is used for determining a translation matrix corresponding to the inner point pairs and transforming the image to be matched based on the translation matrix.
9. A computer-readable storage medium, the storage medium storing a computer program for executing the method of any of the preceding claims 1-7.
10. An electronic device, the electronic device comprising:
a processor;
a memory for storing the processor-executable instructions;
the processor is used for reading the executable instructions from the memory and executing the instructions to realize the method of any one of the claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111014707.4A CN115731403A (en) | 2021-08-31 | 2021-08-31 | Image alignment method and device, storage medium and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111014707.4A CN115731403A (en) | 2021-08-31 | 2021-08-31 | Image alignment method and device, storage medium and electronic equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115731403A true CN115731403A (en) | 2023-03-03 |
Family
ID=85291556
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111014707.4A Pending CN115731403A (en) | 2021-08-31 | 2021-08-31 | Image alignment method and device, storage medium and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115731403A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117907510A (en) * | 2024-03-20 | 2024-04-19 | 浙江灵析精仪科技发展有限公司 | Two-dimensional spectrogram alignment method and system |
-
2021
- 2021-08-31 CN CN202111014707.4A patent/CN115731403A/en active Pending
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117907510A (en) * | 2024-03-20 | 2024-04-19 | 浙江灵析精仪科技发展有限公司 | Two-dimensional spectrogram alignment method and system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Belan et al. | Segmentation-free approaches of computer vision for automatic calibration of digital and analog instruments | |
| CN110969245B (en) | Target detection model training method and device for medical image | |
| CN109376631A (en) | A kind of winding detection method and device neural network based | |
| Liu et al. | Optimal matching problem in detection and recognition performance evaluation | |
| CN109947273B (en) | Point reading positioning method and device | |
| CN101650784A (en) | Method for matching images by utilizing structural context characteristics | |
| CN115375917B (en) | Target edge feature extraction method, device, terminal and storage medium | |
| CN110659570A (en) | Target object posture tracking method, and neural network training method and device | |
| CN117078927A (en) | Combined target labeling method, device, equipment and storage medium | |
| CN115731403A (en) | Image alignment method and device, storage medium and electronic equipment | |
| CN115240029A (en) | Training method of image regression model, image regression analysis method, medium, and terminal | |
| CN111914841B (en) | CT image processing method and device | |
| CN110647826B (en) | Method and device for acquiring commodity training picture, computer equipment and storage medium | |
| CN114092542B (en) | Bolt measurement method and system based on two-dimensional vision | |
| CN115578778A (en) | Human face image feature extraction method based on trace transformation and LBP (local binary pattern) | |
| Wu et al. | An accurate feature point matching algorithm for automatic remote sensing image registration | |
| CN113095384A (en) | Remote sensing image matching method based on context characteristics of straight line segments | |
| CN110210476B (en) | Character component clustering method, device, equipment and computer readable storage medium | |
| CN110222688B (en) | Instrument positioning method based on multi-level correlation filtering | |
| Konishi et al. | Fast and precise template matching based on oriented gradients | |
| CN108364013B (en) | Image key point feature descriptor extraction method and system based on neighborhood Gaussian differential distribution | |
| CN108121994B (en) | Method and device for extracting features in detection of target shape | |
| Järvinen | Configurable pointer meter reader based on computer vision | |
| CN114882482B (en) | Seal anti-counterfeiting identification method and device | |
| CN113283448B (en) | Training method and device for electroencephalogram image feature extraction model |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20240405 Address after: No. 101, Building 7, Jinyuan Community, No. 38 Hetai Road, Yuetang Street, Hetang District, Zhuzhou City, Hunan Province, 2011 Applicant after: Zhuzhou Yawen Technology Co.,Ltd. Country or region after: China Address before: 201821 Room 1119, Building A, No. 1355, Chengbei Road, Jiading Industrial Zone, Jiading District, Shanghai Applicant before: Shanghai Lianlu Semiconductor Technology Co.,Ltd. Country or region before: China |
|
| TA01 | Transfer of patent application right | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20240417 Address after: 201508 1st floor, No. 1000, Tingwei Road, Jinshan District, Shanghai (Bay area science and Innovation Center) Applicant after: Shanghai Ruisiheguang Semiconductor Technology Co.,Ltd. Country or region after: China Address before: No. 101, Building 7, Jinyuan Community, No. 38 Hetai Road, Yuetang Street, Hetang District, Zhuzhou City, Hunan Province, 2011 Applicant before: Zhuzhou Yawen Technology Co.,Ltd. Country or region before: China |
|
| TA01 | Transfer of patent application right |