CN115205118A - Underwater image splicing method and device, computer equipment and storage medium - Google Patents
Underwater image splicing method and device, computer equipment and storage medium Download PDFInfo
- Publication number
- CN115205118A CN115205118A CN202210799255.3A CN202210799255A CN115205118A CN 115205118 A CN115205118 A CN 115205118A CN 202210799255 A CN202210799255 A CN 202210799255A CN 115205118 A CN115205118 A CN 115205118A
- Authority
- CN
- China
- Prior art keywords
- image
- matched
- images
- training
- points
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformation in the plane of the image
- G06T3/40—Scaling the whole image or part thereof
- G06T3/4038—Scaling the whole image or part thereof for image mosaicing, i.e. plane images composed of plane sub-images
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/50—Image enhancement or restoration by the use of more than one image, e.g. averaging, subtraction
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/46—Descriptors for shape, contour or point-related descriptors, e.g. scale invariant feature transform [SIFT] or bags of words [BoW]; Salient regional features
- G06V10/462—Salient features, e.g. scale invariant feature transforms [SIFT]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
- G06V10/75—Organisation of the matching processes, e.g. simultaneous or sequential comparisons of image or video features; Coarse-fine approaches, e.g. multi-scale approaches; using context analysis; Selection of dictionaries
- G06V10/757—Matching configurations of points or features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
- G06V10/761—Proximity, similarity or dissimilarity measures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2200/00—Indexing scheme for image data processing or generation, in general
- G06T2200/32—Indexing scheme for image data processing or generation, in general involving image mosaicing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20212—Image combination
- G06T2207/20221—Image fusion; Image merging
Abstract
The application belongs to the technical field of image splicing, and discloses an underwater image splicing method, an underwater image splicing device, computer equipment and a storage medium, wherein the method comprises the following steps: acquiring two images to be matched containing an overlapping area; respectively extracting feature points in two images to be matched through an SIFT algorithm; matching the characteristic points in the two images to be matched to obtain a characteristic point matching pair of the two images to be matched; calculating according to the feature point matching pair to obtain a corresponding homography matrix, and performing rotation transformation and correction on the first training image according to the homography matrix to obtain a second training image; taking the reference image and the second training image as two new images to be matched; performing paste type splicing on the reference image and the second training image to obtain a primary spliced image; and carrying out fusion processing on the image overlapping area in the primary spliced image by a gradual-in and gradual-out fusion algorithm to obtain a final spliced image. The splicing double image and splicing gap problem of the underwater image can be solved.
Description
Technical Field
The application relates to the technical field of image splicing, in particular to an underwater image splicing method and device, computer equipment and a storage medium.
Background
The panoramic image has more and more extensive application scenes underwater, and has important significance in the aspects of underwater archaeology, oceanographic survey and the like. Since a single camera is limited in photographing angle and photographing range, a plurality of images collected by a plurality of image collecting devices are generally stitched to form a panoramic image. The traditional stitching method comprises the steps of selecting a specific scene point from images collected in a centralized mode, extracting image features of a plurality of images corresponding to the scene point, conducting registration, conducting image fusion in an overlapped area after registration, and finally obtaining a panoramic image. However, due to disturbance of water flow, photographing cannot be stabilized as in the case of land, resulting in a certain degree of imaging deviation. Even if the existing underwater image splicing method realizes a splicing result, the problems of double images, splicing gaps and the like are easy to occur.
Disclosure of Invention
The application provides an underwater image splicing method, an underwater image splicing device, computer equipment and a storage medium, which can solve the problems of splicing ghosts and splicing gaps of underwater images.
In a first aspect, an embodiment of the present application provides an underwater image stitching method, where the method includes:
acquiring two images to be matched containing an overlapping area, wherein the two images to be matched comprise a reference image and a first training image;
respectively extracting feature points in two images to be matched through an SIFT algorithm;
matching the characteristic points in the two images to be matched to obtain a characteristic point matching pair of the two images to be matched;
calculating according to the feature point matching pair to obtain a corresponding homography matrix, and performing rotation transformation and correction on the first training image according to the homography matrix to obtain a second training image;
taking the reference image and the second training image as new two images to be matched, repeating the steps of respectively extracting the feature points in the two images to be matched through an SIFT algorithm and matching the feature points in the two images to be matched to obtain a feature point matching pair of the two images to be matched, and obtaining a feature point matching pair of the reference image and the second training image;
performing paste type splicing on the reference image and the second training image based on the feature point matching pair of the reference image and the second training image to obtain a primary spliced image;
and carrying out fusion processing on the image overlapping region in the primary spliced image by a gradual-in gradual-out fusion algorithm to obtain a final spliced image.
In one embodiment, the extracting the feature points in the two images to be matched respectively through the SIFT algorithm comprises the following steps:
respectively establishing a Gaussian pyramid for the two images to be matched, generating an image scale space of each image to be matched, and obtaining a Gaussian difference scale space of each image to be matched based on the image scale space of each image to be matched;
determining the characteristic points of each image to be matched in the Gaussian difference scale space of each image to be matched;
and describing the feature points of each image to be matched to generate a feature descriptor of each feature point, wherein the feature descriptor comprises position information, scale information and direction information.
In one embodiment, determining feature points of each image to be matched in the gaussian difference scale space of the image to be matched comprises:
in the Gaussian difference scale space of each image to be matched, comparing the pixel point value in the image to be matched with 8 pixel points in the neighborhood of the same scale and 18 adjacent pixel points in the upper and lower scale domains in the scale space respectively, and identifying the obtained maximum value point or minimum value point as a first candidate feature point;
screening the first candidate feature points through a three-dimensional quadratic function to obtain second candidate feature points;
and removing second candidate characteristic points with serious edge effect from the second candidate characteristic points through a Hessian matrix to obtain the characteristic points of each image to be matched.
In one embodiment, matching the feature points in the two images to be matched to obtain a feature point matching pair of the two images to be matched, includes:
carrying out rough matching on the feature points in the two images to be matched through a KNN nearest neighbor algorithm;
and performing fine matching on the feature points in the two images to be matched after the coarse matching through an RANSAC algorithm to obtain a feature point matching pair of the two images to be matched.
In one embodiment, performing rotation transformation and correction on the first training image according to the homography matrix to obtain a second training image includes:
performing rotation transformation on the first training image according to the homography matrix to obtain an intermediate image;
and correcting coordinate values corresponding to all pixel points in the intermediate image according to the coordinate values of the horizontal axis and the coordinate values of the vertical axis in the four vertex coordinate information of the intermediate image, and eliminating redundant blank areas generated due to rotation transformation to obtain a second training image.
In one embodiment, the step of performing paste-type stitching on the reference image and the second training image based on the feature point matching pair of the reference image and the second training image to obtain a primary stitched image includes:
carrying out summation average calculation on the difference results of all the feature point matching pairs of the reference image and the second training image to obtain a relative position modifier of the reference image and the second training image;
and determining the relative positions of the reference image and the second training image during splicing according to the relative position modifier, and performing paste splicing on the reference image and the second training image according to the relative positions to obtain a primary spliced image.
In one embodiment, the obtaining of the final stitched image by fusing the image overlapping region in the primary stitched image through a fade-in and fade-out fusion algorithm includes:
determining the height and width of an image overlapping region in the primary mosaic image;
when the height of the image overlapping area is larger than the width, performing fusion processing on the image overlapping area in the primary spliced image by adopting a horizontal gradually-in and gradually-out fusion algorithm to obtain a final spliced image;
and when the height of the image overlapping area is smaller than the width, performing fusion processing on the image overlapping area in the primary spliced image by adopting a vertical gradually-in and gradually-out fusion algorithm to obtain a final spliced image.
In a second aspect, an embodiment of the present application provides an underwater image stitching device, where the device includes:
the image acquisition module is used for acquiring two images to be matched containing an overlapping area, wherein the two images to be matched comprise a reference image and a first training image;
the characteristic point extraction module is used for respectively extracting characteristic points in the two images to be matched through an SIFT algorithm;
the characteristic point matching module is used for matching the characteristic points in the two images to be matched to obtain a characteristic point matching pair of the two images to be matched;
the image conversion module is used for calculating to obtain a corresponding homography matrix according to the feature point matching pair, and performing rotation transformation and correction on the first training image according to the homography matrix to obtain a second training image;
the image processing module is used for taking the reference image and the second training image as two new images to be matched, repeating the steps of respectively extracting the feature points in the two images to be matched through an SIFT algorithm and matching the feature points in the two images to be matched to obtain a feature point matching pair of the two images to be matched, and obtaining a feature point matching pair of the reference image and the second training image;
the image splicing module is used for splicing the reference image and the second training image in a sticking mode based on the feature point matching pair of the reference image and the second training image to obtain a primary spliced image;
and the image fusion module is used for carrying out fusion processing on the image overlapping area in the primary spliced image through a gradual-in and gradual-out fusion algorithm to obtain a final spliced image.
In a third aspect, an embodiment of the present application provides a computer device, including a memory and a processor, where the memory stores a computer program, and when the computer program is executed by the processor, the processor is caused to execute the steps of the underwater image stitching method according to any one of the above embodiments.
In a fourth aspect, the present application provides a computer-readable storage medium, on which a computer program is stored, where the computer program is executed by a processor to implement the steps of the underwater image stitching method according to any one of the above embodiments.
In summary, compared with the prior art, the beneficial effects brought by the technical scheme provided by the embodiment of the application at least include:
according to the underwater image splicing method, two images to be matched including an overlapping area can be obtained, wherein the two images to be matched comprise a reference image and a first training image; the method realizes that the first training image carries out rotation transformation according to the corresponding characteristic points in the reference image through characteristic point extraction and matching in the first stage, obtains the second training image as a new image to be matched, and ensures the consistency of the size and the shape of the characteristic targets in the two images; then, the expression capability and the registration accuracy of the feature points between the images to be matched are improved through feature point re-extraction and matching operation in the second stage, the images in the second stage can be spliced according to the feature points with strong information expression capability, the image splicing error is reduced, and the problem of splicing double images can be solved; and finally, the primary splicing image is subjected to fusion processing by using a gradual-in and gradual-out method, so that the problems of splicing gaps and the like can be solved. The method can solve the problems of splicing ghosting and splicing gaps of the underwater images.
Drawings
Fig. 1 is a schematic two-stage flow diagram of an underwater image stitching method according to an exemplary embodiment of the present application.
Fig. 2 is a flowchart of an underwater image stitching method according to an exemplary embodiment of the present application.
Fig. 3 is a structural diagram of an underwater image stitching device according to an exemplary embodiment of the present application.
Detailed Description
The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments obtained by a person of ordinary skill in the art based on the embodiments in the present application without making any creative effort belong to the protection scope of the present application.
Referring to fig. 1, an embodiment of the present application provides an underwater image stitching method, which may be divided into two stages as shown in fig. 1. Taking the execution main body as a terminal as an example, please refer to fig. 2, the method specifically includes the following steps:
the first stage is as follows:
step S1, two images to be matched containing an overlapping area are obtained, and the two images to be matched comprise a reference image and a first training image.
The two images to be matched can be images of the same object or scene collected by image collecting equipment from two different shooting angles and shooting ranges, so that the two images to be matched contain overlapping regions.
Specifically, the two acquired images to be matched may be defined as a reference image a and a first training image B, respectively.
And S2, respectively extracting feature points in the two images to be matched through a SIFT algorithm.
The SIFT (Scale Invariant Feature Transform) algorithm is a local Feature extraction algorithm, and is characterized in that an extreme point is firstly found in a Scale space, unstable edge points are removed to obtain key points, and finally Feature descriptors at the key points are extracted to serve as matching bases.
And S3, matching the characteristic points in the two images to be matched to obtain a characteristic point matching pair of the two images to be matched.
And S4, calculating according to the feature point matching pairs to obtain corresponding homography matrixes, and performing rotation transformation and correction on the first training image according to the homography matrixes to obtain a second training image.
In particular, the second training image C = the homography matrix H × the first training image B may be used.
Wherein, the Homography Matrix (homographic Matrix) can constrain 2D homogeneous coordinates of the same 3D space point at two pixel planes. In computer vision, plane homography is defined as the projection mapping of one plane to another, so that the mapping of points on a two-dimensional plane onto the camera imager is an example of plane homography, which can be represented by a homography matrix.
And a second stage:
and S5, taking the reference image and the second training image as two new images to be matched, repeating the steps of respectively extracting the feature points in the two images to be matched through an SIFT algorithm and matching the feature points in the two images to be matched to obtain a feature point matching pair of the two images to be matched, and obtaining the feature point matching pair of the reference image and the second training image.
Specifically, the step S2 and the step S3 are repeated, and the precise feature point matching pair information corresponding to the reference image a and the second training image C is obtained through calculation, where the feature point matching pair information may include coordinate information and matching scores of each feature point matching pair in each image.
In the above steps, the second training image C is regarded as a new image C to be matched, and is used as two new images to be matched together with the reference image a.
And S6, performing paste type splicing on the reference image and the second training image based on the matching pair of the feature points of the reference image and the second training image to obtain a primary spliced image.
And S7, carrying out fusion processing on the image overlapping area in the primary spliced image through a gradual-in and gradual-out fusion algorithm to obtain a final spliced image.
In specific implementation, in order to avoid the problems of double images, obvious splicing traces and the like of the primary spliced images, a gradual-in and gradual-out fusion algorithm is adopted to perform fusion processing on the spliced positions of the images. The gradual-in and gradual-out method can distribute proportion of weight according to the distance between the overlapped area of the two images and the two original images in a certain direction, so as to realize the gradual fusion effect of the overlapped images and the original images. The gradual-in and gradual-out fusion algorithm mainly processes the gray values of the pixel points in the overlapping area, and the calculation formula is as follows:
in the formula, f (x, y) represents the gray value of the pixel point of the fused image; f. of 1 (x, y) and f 2 (x, y) represents gray values of pixel points of two images to be spliced; c. C 1 And c 2 Is a weighting coefficient, and c 1 +c 2 ＝1，0＜c 1 ＜1，0＜c 2 ＜1。
The underwater image splicing method provided in the above embodiment can acquire two images to be matched including an overlapping region, where the two images to be matched include a reference image and a first training image; according to the method, in the first stage, the first training image is subjected to rotation transformation according to corresponding feature points in a reference image through feature point extraction and matching, the second training image is obtained and used as a new image to be matched, and the size and shape consistency of feature targets in two images is ensured; then, the expression capability and the registration accuracy of the feature points between the images to be matched are improved through feature point re-extraction and matching operation in the second stage, the images in the second stage can be spliced according to the feature points with strong information expression capability, the image splicing error is reduced, and the problem of splicing ghosts can be solved; and finally, the primary splicing image is subjected to fusion processing by using a gradual-in and gradual-out method, so that the problems of splicing gaps and the like can be solved. The method can solve the problems of splicing ghosting and splicing gaps of the underwater images.
In some embodiments, step S2 specifically includes the following steps:
step 201, respectively establishing a gaussian pyramid for two images to be matched, generating an image scale space of each image to be matched, and obtaining a gaussian difference scale space of each image to be matched based on the image scale space of each image to be matched.
Specifically, a gaussian pyramid is established for the reference image a and the first training image B, respectively, to generate a scale space.
The scale space of the image is defined as:
wherein, L (x, y, σ) represents the obtained image scale space, and is obtained by convolution of a scale-variable gaussian function G (x, y, σ) and the original image I (x, y); σ denotes a scale space factor, and the larger the σ value, the larger the degree to which the image is smoothed.
The scale-variable gaussian function G is represented as:
and (3) calculating by using a DoG difference operator to obtain a Gaussian difference scale space, wherein the formula is as follows:
k is a multiple of two adjacent scale spaces, namely the detection of the feature point on a certain scale can be obtained by subtracting two adjacent Gaussian scale space images in the same group.
Step 202, determining the characteristic points of each image to be matched in the Gaussian difference scale space of each image to be matched.
Firstly, in the Gaussian difference scale space of each image to be matched, comparing the pixel point value in the image to be matched with 8 pixel points in the neighborhood of the same scale and 18 adjacent pixel points in the upper and lower scale domains in the scale space, and determining the obtained maximum value point or minimum value point as a first candidate feature point.
And then, screening the first candidate characteristic points through a three-dimensional quadratic function to obtain second candidate characteristic points.
Specifically, a second-order taylor expansion is adopted to fit the response curve of the characteristic points:
wherein, X = (X, y, sigma) T After derivation, the derivative equation is equal to 0, and the offset of the feature point is calculated as:
then will be
Brought back into D (x) to give:
the threshold value can be set to 0.03 if
The feature point is removed from the existing first candidate feature points.
And finally, removing second candidate characteristic points with serious edge effect from the second candidate characteristic points through a Hessian matrix to obtain the characteristic points of each image to be matched.
Specifically, hessian matrix is used for removing corresponding interference of edges caused by Gaussian difference operation, and therefore the selection of characteristic points is optimized. The specific process is as follows:
Tr(H)＝D xx +D yy ＝α+β
Det(H)＝D xx D yy -(D xy ) 2 ＝αβ
wherein, the first and the second end of the pipe are connected with each other,
D xx 、D xy 、D yy representing the second derivative of D (X), and α and β are defined as the larger eigenvalues and smaller eigenvalues, respectively, of matrix H. Let α = γ β, then:
the present embodiment can refer to γ =10 in the Lowe method to realize the elimination of the feature points with severe edge effect.
And step 203, describing the feature points of each image to be matched, and generating a feature descriptor of each feature point.
Wherein the feature descriptor includes location information, scale information, and direction information.
Specifically, neighborhood gradient information corresponding to each feature point in a gaussian difference scale space where the feature point is located is calculated, wherein the neighborhood gradient information comprises a gradient value m and a direction theta, and a calculation formula is as follows:
and constructing a gradient histogram by using the gradient module value obtained by calculation, wherein the gradient histogram equally divides the circular region of 0-360 degrees into 36 parts, and each part is 10 degrees. The peak value of the histogram represents the direction of the gradient of the neighborhood of the feature point, and the maximum peak value in the histogram is taken as the main direction of the feature point. In order to enhance the robustness of matching, a direction greater than 80% of the gradient peak is reserved as a secondary direction of the feature point. And rotating the characteristic points to the main direction, and establishing a descriptor containing three information quantities of position, scale and direction for each characteristic point. Gradient values in eight directions in the neighborhood of the feature points 4 × 4 are calculated, and a gaussian window is used for performing weighting operation to obtain vector information in eight directions of the feature points in each local block, so that a feature descriptor with dimensions 4 × 4 × 8=128 is finally generated.
The embodiment can respectively extract the feature points in the two images to be matched through the SIFT algorithm, and because the SIFT algorithm has good robustness, even a few objects can generate a large number of SIFT feature vectors, the feature points can be extracted more quickly and accurately.
In some embodiments, step S3 specifically includes the following steps:
and carrying out rough matching on the feature points in the two images to be matched through a KNN nearest neighbor algorithm.
Among them, KNN (K-Nearest Neighbor) is a classification algorithm of machine learning entry level and is the simplest algorithm. It enables to classify closely spaced sample points into the same category. The K in KNN is the number of neighbors, that is, the nearest K points, and the category to which the K points belong is determined according to what category the nearest K points are.
Specifically, KNN uses the euclidean distance between feature points as a similarity determination measure for keypoints in two images. Selecting a certain key point of the reference image A, finding and recording two feature points which are closest to the feature points in the reference image A in the first training image B through traversal, calculating the distance ratio of the closest feature point to the second close feature point, and if the ratio is less than 0.75 of a threshold value, keeping the closest feature point at the moment.
And performing fine matching on the characteristic points in the two images to be matched after the coarse matching through an RANSAC algorithm to obtain a characteristic point matching pair of the two images to be matched.
The RANSAC is an abbreviation of Random Sample Consensus, namely a Random Sample Consensus algorithm, and is an algorithm for calculating mathematical model parameters of data according to a group of Sample data sets containing abnormal data to obtain effective Sample data.
In specific implementation, in step 301, a part of elements are selected from the rough-matching feature point set and set as an inner group, each element point in the inner group is substituted into a model to solve model parameters, and the model can be expressed as:
where (x ', y') and (x, y) represent a set of pairs of characteristic points.
For the homography matrix, a geometric transformation relationship between images is described.
And 302, substituting the rest points except all the points of the inner group in the feature point set into the model obtained in the step 301, calculating the distance between each element point and the model, recording the feature points meeting the error allowance range as the element points of the inner group, and taking the feature points not meeting the condition as the outer points, and setting the error threshold value to be 4.
Step 303, repeating step 301 and step 302N times by using all element points included in the current inner group, wherein the iteration number N is expressed as:
wherein p is the confidence coefficient, the value range is generally (0.95-0.99), phi represents the proportion of the group element points in the data set, and m represents the number of all the characteristic points in the data set.
And 304, selecting the model with the largest number of the element points of the inner group in the iteration process as an optimal fitting model, and taking the element points of the inner group at the moment as a final feature point matching pair, namely the feature point matching pair of the two images to be matched.
In the embodiment, the characteristic points can be roughly matched through a KNN nearest neighbor algorithm, and the characteristic points in the two images to be matched which are roughly matched are finely matched through a RANSAC algorithm, so that the optimal characteristic point matching pairs of the two images to be matched are obtained, and the accuracy of the finally determined characteristic point matching pairs can be improved.
In some embodiments, step S4 specifically includes the following steps:
and calculating to obtain a corresponding homography matrix according to the feature point matching pairs.
And carrying out rotation transformation on the first training image according to the homography matrix to obtain an intermediate image.
Specifically, the first training image B is rotation-transformed by using the homography matrix H, and the formula is as follows:
in the formula I B (x)、I B (y) the coordinate value corresponding to the pixel point of the training image B, I tmp (x)、I tmp (y) representing the image I after the rotation transformation of the training image B tmp The pixel point of (a) corresponds to a coordinate value, and i represents any channel component in the RGB three channels.
And correcting coordinate values corresponding to all pixel points in the intermediate image according to the coordinate values of the horizontal axis and the coordinate values of the vertical axis in the four vertex coordinate information of the intermediate image, and eliminating redundant blank areas generated due to rotation transformation to obtain a second training image.
Specifically, an image I is selected tmp The coordinate value of the horizontal axis and the coordinate value of the vertical axis with the maximum absolute value in the four vertex coordinates are used as coordinate correction points
And
and (3) removing redundant blank areas generated by rotation transformation, wherein a coordinate point correction formula is as follows:
wherein, I c (x)、I c And (y) representing the corresponding coordinate value of the pixel point of the second training image C.
According to the embodiment, the first training image can be subjected to rotation transformation according to the homography matrix, the second training image is obtained by correcting and eliminating redundant blank areas and is used as the image to be matched, and the size and shape consistency of the feature target in the two images to be matched can be ensured.
In some embodiments, step S6 specifically includes the following steps:
and performing summation average calculation on the difference results of all the feature point matching pairs of the reference image and the second training image to obtain a relative position modifier of the reference image and the second training image.
The relative position modifier of the reference image A and the second training image C is used for determining the relative position when the two images are spliced.
Specifically, the difference results of all the feature point matching pairs of the reference image a and the second training image C are summed and averaged to obtain the relative position modifier x of the reference image a and the second training image C sub ，y sub The formula is as follows:
and determining the relative positions of the reference image and the second training image during splicing according to the relative position modifier, and performing paste splicing on the reference image and the second training image according to the relative positions to obtain a primary spliced image.
Specifically, the correction symbol x is determined based on the relative position sub ，y sub The pasting condition of the reference image A and the second training image C is judged according to the positive and negative conditions of the reference image A and the second training image C. Such as when x sub Positive, the representative second training image C is on the left and the reference image a is on the right. When y is sub Positive, it represents the second training image C above and the reference image a below. The positions of the reference image A and the second training image C are determined, and the final splicing effect of the two images can be realized by using the relative position modifier.
The embodiment can determine the relative position of the reference image and the second training image during splicing based on the relative position modifier, so that the accuracy of splicing is improved, and a better splicing effect can be ensured.
In some embodiments, step S7 specifically includes the following steps:
the height and width of the image overlap region in the primary stitched image are determined.
And when the height of the image overlapping area is larger than the width, performing fusion processing on the image overlapping area in the primary spliced image by adopting a horizontal gradually-in and gradually-out fusion algorithm to obtain a final spliced image.
And when the height of the image overlapping area is smaller than the width, performing fusion processing on the image overlapping area in the primary spliced image by adopting a vertical gradually-in and gradually-out fusion algorithm to obtain a final spliced image.
During specific implementation, aiming at the situation that the image is spliced horizontally and vertically in the actual splicing process, the gradual-in and gradual-out fusion algorithm is divided into horizontal gradual-in and gradual-out and vertical gradual-in and gradual-out in the embodiment. In the horizontal fade-in fade-out fusion method, f 1 (x, y) and f 2 (x, y) is specifically expressed as the gray value of pixel points of the left and right images to be spliced; and c is a 1 And c 2 The calculation formula of (c) is as follows:
in the formula, x i Representing the abscissa of the current pixel point; x is the number of l A left boundary representing an overlap region; x is the number of r Indicating the right boundary of the overlap region.
In the vertical fade-in fade-out fusion method, f 1 (x, y) and f 2 (x, y) is specifically expressed as the gray value of the pixel points of the upper and lower images to be spliced; c. C 1 And c 2 The calculation formula of (a) is as follows:
in the formula, y i Representing the ordinate of the current pixel point; y is u An upper boundary representing an overlap region; y is d Representing the lower boundary of the overlap region.
When the height of the overlapped part is larger than the width, the two images are considered to be basically horizontally spliced, and the fusion effect is better by adopting a horizontal gradually-in and gradually-out method; when the height of the overlapped part is larger than the width, the two images are considered to be approximately vertically spliced, and the fusion effect is better by adopting a vertical gradually-in and gradually-out method.
The embodiment can perform corresponding fusion processing according to the height and the width of the image overlapping area, thereby realizing the effect of eliminating the splicing gap better.
Another embodiment of the present application provides an underwater image stitching device, please refer to fig. 3, which includes:
the image obtaining module 101 is configured to obtain two images to be matched including an overlapping region, where the two images to be matched include a reference image and a first training image.
And the feature point extraction module 102 is configured to extract feature points in two images to be matched respectively through a SIFT algorithm.
The feature point matching module 103 is configured to match feature points in the two images to be matched to obtain a feature point matching pair of the two images to be matched.
And the image conversion module 104 is configured to calculate to obtain a corresponding homography matrix according to the feature point matching pairs, and perform rotation transformation and correction on the first training image according to the homography matrix to obtain a second training image.
The image processing module 105 is configured to take the reference image and the second training image as new two images to be matched, repeat the steps of extracting feature points in the two images to be matched respectively through an SIFT algorithm and matching the feature points in the two images to be matched to obtain a feature point matching pair of the two images to be matched, and obtain a feature point matching pair of the reference image and the second training image.
And the image stitching module 106 is configured to perform paste-type stitching on the reference image and the second training image based on the feature point matching pair of the reference image and the second training image to obtain a primary stitched image.
And the image fusion module 107 is used for performing fusion processing on the image overlapping area in the primary spliced image through a gradual-in and gradual-out fusion algorithm to obtain a final spliced image.
In some embodiments, the feature point extraction module 102 comprises:
and the Gaussian difference scale space unit is used for establishing a Gaussian pyramid for the two images to be matched respectively, generating an image scale space of each image to be matched, and obtaining the Gaussian difference scale space of each image to be matched based on the image scale space of each image to be matched.
And the characteristic point determining unit is used for determining the characteristic points of the images to be matched in the Gaussian difference scale space of each image to be matched.
And the characteristic point description unit is used for describing the characteristic points of each image to be matched and generating a characteristic descriptor of each characteristic point, wherein the characteristic descriptor comprises position information, scale information and direction information.
In some embodiments, the feature point determining unit is specifically configured to: in the Gaussian difference scale space of each image to be matched, comparing the pixel point value in the image to be matched with 8 pixel points in the neighborhood of the same scale and 18 adjacent pixel points in the upper and lower scale domains in the scale space respectively, and identifying the obtained maximum value point or minimum value point as a first candidate feature point; screening the first candidate feature points through a three-dimensional quadratic function to obtain second candidate feature points; and removing second candidate characteristic points with serious edge effect from the second candidate characteristic points through a Hessian matrix to obtain the characteristic points of each image to be matched.
In some embodiments, the feature point matching module 103 is specifically configured to: carrying out rough matching on the feature points in the two images to be matched through a KNN nearest neighbor algorithm; and performing fine matching on the feature points in the two images to be matched after the coarse matching through an RANSAC algorithm to obtain a feature point matching pair of the two images to be matched.
In some embodiments, the image conversion module 104 is specifically configured to: carrying out rotation transformation on the first training image according to the homography matrix to obtain an intermediate image; and correcting coordinate values corresponding to all pixel points in the intermediate image according to the coordinate values of the horizontal axis and the coordinate values of the vertical axis in the four vertex coordinate information of the intermediate image, and eliminating redundant blank areas generated due to rotation transformation to obtain a second training image.
In some embodiments, the image stitching module 106 is specifically configured to: carrying out summation average calculation on the difference results of all the feature point matching pairs of the reference image and the second training image to obtain a relative position modifier of the reference image and the second training image; and determining the relative positions of the reference image and the second training image during splicing according to the relative position modifier, and performing paste splicing on the reference image and the second training image according to the relative positions to obtain a primary spliced image.
In some embodiments, the image fusion module 107 is specifically configured to: determining the height and width of an image overlapping region in the primary mosaic image; when the height of the image overlapping area is larger than the width, performing fusion processing on the image overlapping area in the primary spliced image by adopting a horizontal gradually-in and gradually-out fusion algorithm to obtain a final spliced image; and when the height of the image overlapping area is smaller than the width, performing fusion processing on the image overlapping area in the primary spliced image by adopting a vertical gradually-in and gradually-out fusion algorithm to obtain a final spliced image.
For specific limitations of the underwater image stitching device provided in this embodiment, reference may be made to the above embodiments of the underwater image stitching method, which is not described herein again. The modules in the underwater image splicing device can be wholly or partially realized by software, hardware and a combination thereof. The modules can be embedded in a hardware form or independent from a processor in the computer device, and can also be stored in a memory in the computer device in a software form, so that the processor can call and execute operations corresponding to the modules.
Embodiments of the present application provide a computer device that may include a processor, a memory, a network interface, and a database connected by a system bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device comprises a nonvolatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operating system and the computer program to run on the non-volatile storage medium. The network interface of the computer device is used for communicating with an external terminal through a network connection. When executed by a processor, the computer program causes the processor to perform the steps of the underwater image stitching method as in any one of the embodiments described above.
The working process, working details and technical effects of the computer device provided by this embodiment may refer to the above embodiments of the underwater image stitching method, which are not described herein again.
The embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored, and the computer program, when executed by a processor, implements the steps of the underwater image stitching method according to any one of the embodiments. The computer-readable storage medium refers to a carrier for storing data, and may include, but is not limited to, floppy disks, optical disks, hard disks, flash memories, flash disks and/or Memory sticks (Memory sticks), etc., and the computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices.
The working process, working details and technical effects of the computer-readable storage medium provided in this embodiment may refer to the above embodiments of the underwater image stitching method, which are not described herein again.
It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by hardware instructions of a computer program, which can be stored in a non-volatile computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. Any reference to memory, storage, database, or other medium used in the embodiments provided herein may include non-volatile and/or volatile memory, among others. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically Programmable ROM (EPROM), electrically Erasable Programmable ROM (EEPROM), or flash memory. Volatile memory can include Random Access Memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms such as Static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double Data Rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous Link DRAM (SLDRAM), rambus (Rambus) direct RAM (RDRAM), direct Rambus Dynamic RAM (DRDRAM), and Rambus Dynamic RAM (RDRAM), among others.
The technical features of the above embodiments can be arbitrarily combined, and for the sake of brevity, all possible combinations of the technical features in the above embodiments are not described, but should be considered as the scope of the present specification as long as there is no contradiction between the combinations of the technical features.
The above examples only express several embodiments of the present application, and the description thereof is more specific and detailed, but not construed as limiting the scope of the invention. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the concept of the present application, which falls within the scope of protection of the present application. Therefore, the protection scope of the present patent shall be subject to the appended claims.
Claims (10)
1. An underwater image stitching method, characterized by comprising:
acquiring two images to be matched containing an overlapping area, wherein the two images to be matched comprise a reference image and a first training image;
respectively extracting feature points in the two images to be matched through an SIFT algorithm;
matching the characteristic points in the two images to be matched to obtain a characteristic point matching pair of the two images to be matched;
calculating to obtain a corresponding homography matrix according to the feature point matching pair, and performing rotation transformation and correction on the first training image according to the homography matrix to obtain a second training image;
taking the reference image and the second training image as new two images to be matched, repeating the step of respectively extracting the feature points in the two images to be matched through an SIFT algorithm until the feature points in the two images to be matched are matched to obtain the feature point matching pairs of the two images to be matched, and obtaining the feature point matching pairs of the reference image and the second training image;
performing paste type splicing on the reference image and the second training image based on the feature point matching pair of the reference image and the second training image to obtain a primary spliced image;
and carrying out fusion processing on the image overlapping area in the primary spliced image through a gradual-in and gradual-out fusion algorithm to obtain a final spliced image.
2. The method according to claim 1, wherein the extracting feature points in the two images to be matched respectively through a SIFT algorithm comprises:
respectively establishing a Gaussian pyramid for the two images to be matched, generating an image scale space of each image to be matched, and obtaining a Gaussian difference scale space of each image to be matched based on the image scale space of each image to be matched;
determining the characteristic points of the image to be matched in the Gaussian difference scale space of each image to be matched;
and describing the characteristic points of each image to be matched to generate a characteristic descriptor of each characteristic point, wherein the characteristic descriptor comprises position information, scale information and direction information.
3. The method according to claim 2, wherein the determining the feature points of each image to be matched in the gaussian difference scale space of the image to be matched comprises:
in the Gaussian difference scale space of each image to be matched, comparing the pixel point value in the image to be matched with 8 pixel points in the neighborhood of the same scale and 18 adjacent pixel points in the upper and lower scale domains in the scale space respectively, and determining the obtained maximum value point or minimum value point as a first candidate feature point;
screening the first candidate feature points through a three-dimensional quadratic function to obtain second candidate feature points;
and removing second candidate characteristic points with serious edge effect in the second candidate characteristic points through a Hessian matrix to obtain the characteristic points of each image to be matched.
4. The method according to claim 2, wherein the matching the feature points in the two images to be matched to obtain a feature point matching pair of the two images to be matched comprises:
carrying out rough matching on the feature points in the two images to be matched through a KNN nearest neighbor algorithm;
and performing fine matching on the feature points in the two images to be matched after the coarse matching through an RANSAC algorithm to obtain a feature point matching pair of the two images to be matched.
5. The method of claim 1, wherein the rotationally transforming and correcting the first training image according to the homography matrix to obtain a second training image comprises:
carrying out rotation transformation on the first training image according to the homography matrix to obtain an intermediate image;
and correcting coordinate values corresponding to all pixel points in the intermediate image according to the coordinate values of the horizontal axis and the coordinate values of the vertical axis in the coordinate information of the four vertexes of the intermediate image, and eliminating redundant blank areas generated due to rotation transformation to obtain a second training image.
6. The method according to any one of claims 1 to 5, wherein the performing the paste-together stitching on the reference image and the second training image based on the feature point matching pair of the reference image and the second training image to obtain a primary stitched image comprises:
performing summation average calculation on difference results of all feature point matching pairs of the reference image and the second training image to obtain a relative position modifier of the reference image and the second training image;
and determining the relative positions of the reference image and the second training image during splicing according to the relative position modifier, and performing paste splicing on the reference image and the second training image according to the relative positions to obtain a primary spliced image.
7. The method according to claim 6, wherein the fusing the image overlapping regions in the primary stitched image by a fade-in and fade-out fusion algorithm to obtain a final stitched image comprises:
determining the height and width of an image overlapping region in the primary stitched image;
when the height of the image overlapping area is larger than the width, performing fusion processing on the image overlapping area in the primary spliced image by adopting a horizontal gradually-in and gradually-out fusion algorithm to obtain a final spliced image;
and when the height of the image overlapping area is smaller than the width, performing fusion processing on the image overlapping area in the primary spliced image by adopting a vertical gradually-in and gradually-out fusion algorithm to obtain a final spliced image.
8. An underwater image stitching device, characterized in that the device comprises:
the image acquisition module is used for acquiring two images to be matched containing an overlapping area, wherein the two images to be matched comprise a reference image and a first training image;
the characteristic point extraction module is used for respectively extracting the characteristic points in the two images to be matched through an SIFT algorithm;
the characteristic point matching module is used for matching the characteristic points in the two images to be matched to obtain a characteristic point matching pair of the two images to be matched;
the image conversion module is used for calculating to obtain a corresponding homography matrix according to the feature point matching pair, and performing rotation transformation and correction on the first training image according to the homography matrix to obtain a second training image;
the image processing module is used for taking the reference image and the second training image as two new images to be matched, repeating the step of respectively extracting the feature points in the two images to be matched through an SIFT algorithm to match the feature points in the two images to be matched to obtain a feature point matching pair of the two images to be matched, and obtaining a feature point matching pair of the reference image and the second training image;
the image splicing module is used for performing paste splicing on the reference image and the second training image based on the feature point matching pair of the reference image and the second training image to obtain a primary spliced image;
and the image fusion module is used for carrying out fusion processing on the image overlapping area in the primary spliced image through a gradual-in and gradual-out fusion algorithm to obtain a final spliced image.
9. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the steps of the method according to any of claims 1 to 7 are implemented when the computer program is executed by the processor.
10. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the steps of the method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210799255.3A CN115205118A (en) | 2022-07-08 | 2022-07-08 | Underwater image splicing method and device, computer equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210799255.3A CN115205118A (en) | 2022-07-08 | 2022-07-08 | Underwater image splicing method and device, computer equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115205118A true CN115205118A (en) | 2022-10-18 |
Family
ID=83579197
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210799255.3A Pending CN115205118A (en) | 2022-07-08 | 2022-07-08 | Underwater image splicing method and device, computer equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115205118A (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115861927A (en) * | 2022-12-01 | 2023-03-28 | 中国南方电网有限责任公司超高压输电公司大理局 | Image identification method and device for power equipment inspection image and computer equipment |
| CN115953567A (en) * | 2023-03-14 | 2023-04-11 | 广州市玄武无线科技股份有限公司 | Detection method and device for number of stacked boxes, terminal equipment and storage medium |
| CN117541469A (en) * | 2024-01-10 | 2024-02-09 | 中山大学 | SAR image stitching method and device based on graph theory |
| CN117541469B (en) * | 2024-01-10 | 2024-05-10 | 中山大学 | SAR image stitching method and device based on graph theory |
-
2022
- 2022-07-08 CN CN202210799255.3A patent/CN115205118A/en active Pending
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115861927A (en) * | 2022-12-01 | 2023-03-28 | 中国南方电网有限责任公司超高压输电公司大理局 | Image identification method and device for power equipment inspection image and computer equipment |
| CN115953567A (en) * | 2023-03-14 | 2023-04-11 | 广州市玄武无线科技股份有限公司 | Detection method and device for number of stacked boxes, terminal equipment and storage medium |
| CN117541469A (en) * | 2024-01-10 | 2024-02-09 | 中山大学 | SAR image stitching method and device based on graph theory |
| CN117541469B (en) * | 2024-01-10 | 2024-05-10 | 中山大学 | SAR image stitching method and device based on graph theory |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108648240B (en) | Non-overlapping view field camera attitude calibration method based on point cloud feature map registration | |
| CN109410207B (en) | NCC (non-return control) feature-based unmanned aerial vehicle line inspection image transmission line detection method | |
| KR101117837B1 (en) | Multi-image feature matching using multi-scale oriented patches | |
| CN110111250B (en) | Robust automatic panoramic unmanned aerial vehicle image splicing method and device | |
| CN115205118A (en) | Underwater image splicing method and device, computer equipment and storage medium | |
| Jiang et al. | Multiscale locality and rank preservation for robust feature matching of remote sensing images | |
| CN107844750A (en) | A kind of water surface panoramic picture target detection recognition methods | |
| CN106447601B (en) | Unmanned aerial vehicle remote sensing image splicing method based on projection-similarity transformation | |
| CN111553939B (en) | Image registration algorithm of multi-view camera | |
| CN110992263B (en) | Image stitching method and system | |
| US20230252664A1 (en) | Image Registration Method and Apparatus, Electronic Apparatus, and Storage Medium | |
| CN110717936B (en) | Image stitching method based on camera attitude estimation | |
| CN113689331B (en) | Panoramic image stitching method under complex background | |
| Lee et al. | Accurate registration using adaptive block processing for multispectral images | |
| CN107220996A (en) | A kind of unmanned plane linear array consistent based on three-legged structure and face battle array image matching method | |
| CN107680035B (en) | Parameter calibration method and device, server and readable storage medium | |
| CN115526781A (en) | Splicing method, system, equipment and medium based on image overlapping area | |
| CN113706635B (en) | Long-focus camera calibration method based on point feature and line feature fusion | |
| CN111127353A (en) | High-dynamic image ghost removing method based on block registration and matching | |
| CN113066173B (en) | Three-dimensional model construction method and device and electronic equipment | |
| KR101938067B1 (en) | Method and Apparatus for Stereo Matching of Wide-Angle Images using SIFT Flow | |
| US20230379422A1 (en) | Method for vehicle hinge point calibration and corresponding calibration apparatus, computer device, and storage medium | |
| CN113744307A (en) | Image feature point tracking method and system based on threshold dynamic adjustment | |
| Mehrdad et al. | Toward real time UAVS’image mosaicking | |
| CN111739158B (en) | Three-dimensional scene image recovery method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |