CN112288655A - Sea surface image stabilization method based on MSER region matching and low-rank matrix decomposition - Google Patents

Sea surface image stabilization method based on MSER region matching and low-rank matrix decomposition Download PDF

Info

Publication number
CN112288655A
CN112288655A CN202011242901.3A CN202011242901A CN112288655A CN 112288655 A CN112288655 A CN 112288655A CN 202011242901 A CN202011242901 A CN 202011242901A CN 112288655 A CN112288655 A CN 112288655A
Authority
CN
China
Prior art keywords
image
sea
mser
sea surface
rank matrix
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202011242901.3A
Other languages
Chinese (zh)
Other versions
CN112288655B (en
Inventor
王欢
马云飞
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Nanjing University of Science and Technology
Original Assignee
Nanjing University of Science and Technology
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Nanjing University of Science and Technology filed Critical Nanjing University of Science and Technology
Priority to CN202011242901.3A priority Critical patent/CN112288655B/en
Publication of CN112288655A publication Critical patent/CN112288655A/en
Application granted granted Critical
Publication of CN112288655B publication Critical patent/CN112288655B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/77—Retouching; Inpainting; Scratch removal
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/13—Edge detection

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Image Analysis (AREA)

Abstract

The invention discloses a sea surface image stabilization method based on MSER region matching and low-rank matrix decomposition, which is used for carrying out binarization processing on a sea surface image so as to distinguish the sea surface from the sky; and (3) obtaining candidate sea antennas by using Canny operators and Hough edge detection, selecting the sea antennas, rotating and translating the images by taking the sea antennas as a reference, and placing the sea antennas in the center of the images and horizontally. And rotating the grayed image, selecting a proper size characteristic sea wave block below the sea antenna through an MSER method, matching the sea wave block based on the area, the length-width ratio and the position of the sea wave block after completing the image rotation based on the sea antenna in the next frame, and translating the image by taking the coordinates of two matched central points as a reference, thereby realizing the matching of the sea surface image, and further optimizing the matching parameters by adopting a low-rank matrix decomposition method. The method achieves better effect in image stabilization under the scenes of disordered textures, poor characteristic points and the like such as sea surface.

Description

Sea surface image stabilization method based on MSER region matching and low-rank matrix decomposition
Technical Field
The invention belongs to an image stabilization technology, and particularly relates to an image stabilization method based on MSER region matching and low-rank matrix decomposition.
Background
Existing image stabilization techniques align a current frame with a reference frame based on the motion of the current frame relative to the reference frame, where the key is the estimation of the image motion. The conventional image stabilization problem can be well solved by the existing method, but for some scenes (such as sea surfaces) with disordered and rapidly changed textural features, corresponding features are difficult to extract to realize image matching. The existing image matching method has poor effect in realizing image stabilization of sea surface images with poor characteristics.
Disclosure of Invention
The invention relates to an image stabilization method based on MSER region matching and low-rank matrix decomposition, which solves the problems of image feature matching and image stabilization under the condition of disordered textures and has better image stabilization capability.
The technical scheme for realizing the purpose of the invention is as follows: a sea surface image stabilizing method based on MSER region matching and low-rank matrix decomposition comprises the following steps:
step 1, detecting a sea antenna in a sea surface image by using a Hough transformation algorithm;
step 2, aligning the image by using a MSER method, which comprises the following specific steps:
step 21, MSER feature detection is carried out on the sea surface part of the image to obtain an MSER feature block in the frame;
step 22, MSER feature detection is carried out on the next frame image according to step 21 to obtain an MSER feature block, and the error of the MSER feature block in the two frame images is calculated;
step 23, selecting three pairs of MSER feature blocks with the minimum error as feature matching blocks, taking the average value of MSER region center horizontal coordinate differences in the three pairs of feature matching blocks as motion estimation of the current frame relative to a reference frame, and translating the current frame according to the motion estimation;
and 3, stabilizing the image processed in the step 2 by using a low-rank matrix decomposition method.
Preferably, the specific steps of detecting the sea-sky-line in the sea surface image by using the Hough transform algorithm are as follows:
step 11, binarizing the image by taking the average value of the sky gray value and the sea gray value in the image as a threshold value;
step 12, edge detection is carried out by using a Canny method;
step 13, carrying out Hough straight line detection according to the edge detected by the Canny algorithm;
step 14, searching a straight line with the length above a set threshold value and the top left end in the candidate straight lines, and selecting the straight line as the sea-sky line of the frame;
and step 15, rotating the image by taking the sea-sky-line rotated to the horizontal position as a reference, and translating the sea-sky-line to the image center height position as a reference.
Preferably, the specific step of performing MSER feature detection on the sea surface portion of the image includes: binarizing the image by taking [0,255] as a threshold value, detecting a connected domain with the area change lower than the threshold value in the process of increasing the threshold value, inverting the image, then binarizing once again and searching the connected domain, and selecting a region with the area of total images 1/5000 to 1/50 as the MSER feature block of the frame.
Preferably, the error between the ith block in the reference frame and the jth block in the current frame is calculated as
Figure BDA0002768968080000021
Wherein alpha isiRepresents the included angle between the major axis of the MSER elliptical area and the horizontal direction, siArea of the representative region, /)iRepresents the ratio of the length of the major axis to the minor axis of the region, xiRepresenting the horizontal coordinate, and gamma is an empirical coefficient.
Preferably, the step of image stabilization by using low rank matrix decomposition in step 3 is:
step 31, using each frame of the video as an image
Figure BDA0002768968080000022
Inputting, setting initial value tau of deformation parameter1,…,τn；
Step 32, calculating a jacobian matrix about τ:
Figure BDA0002768968080000023
step 33, deforming and normalizing the image matrix:
Figure BDA0002768968080000024
step 34, solving a linear convex optimization problem;
Figure BDA0002768968080000025
Figure BDA0002768968080000026
wherein Δ τ is the amount of change in strain parameter, A*,E*,Δτ*Solving A, E, delta tau of the iteration, wherein A is a low-rank matrix;
step 35, update the morph parameter τ ← τ + Δ τ*If not, returning to the step 32, otherwise, performing the step 36;
and step 36, synthesizing the processed image sequence into a video and outputting the video.
Compared with the prior art, the invention has the following remarkable advantages: the method solves the problem that the traditional method is difficult to find the matched features for image stabilization in the scenes with disordered textures and poor feature points such as the sea surface, and the like.
Drawings
Fig. 1 shows the detection effect of the MSER method between two frames, and the detected MSER feature blocks are outlined by the green oval.
Fig. 2 shows the comparison result between the present invention and the original image and the results obtained by 2 representative methods on six sea surface images, where the results are given by the SSIM value between two adjacent frames, which shows that the present invention has a better image stabilization effect on the sea surface images.
Detailed Description
A sea surface image stabilization method based on MSER region matching and low-rank matrix decomposition comprises the following specific steps:
step 1, detecting a sea antenna in a sea surface image by using a Hough transformation algorithm to preliminarily align the image, and the method specifically comprises the following steps:
and step 11, considering that more clutters exist in the sea surface image, firstly, using the average value of the gray values of the sky and the sea surface in the image as a threshold value to carry out binarization on the image so as to reduce the interference of sea waves on subsequent Hough detection.
And step 12, performing edge detection on the binarized image by using a Canny method, firstly performing Gaussian filtering, then calculating the gradient amplitude and direction of each pixel point, performing non-maximum suppression on the gradient of each pixel point, and finally connecting edges.
And step 13, carrying out Hough linear line detection according to the edge detected by the Canny algorithm, firstly mapping the point detected by the edge into a parameter space, searching a peak point in a transformation space, and searching a straight line segment through the peak point.
Step 14, searching a straight line with the length above a set threshold value and the top left end in the candidate straight lines, and selecting the straight line as the sea-sky line of the frame;
and step 15, rotating the image to enable the sea-sky-line to be located at a horizontal position, and translating the image in a vertical direction to enable the sea-sky-line to be located in the center of the image.
Step 2, aligning the image by using a MSER method, which comprises the following specific steps:
and step 21, performing MSER feature detection on the lower half part of the image subjected to sea-sky-line transformation. The method specifically comprises the following steps:
and 21, carrying out binarization on the image by taking 0-255 as a threshold value, detecting a connected domain with the area change lower than the threshold value in the threshold value rising process, carrying out binarization and searching the connected domain again after inverting the image, and selecting a region with the area of total images 1/5000-1/50 as the MSER feature block of the frame.
And step 22, operating in the next frame (reference frame) according to step 21, and selecting the feature block as a matched feature block according to the minimum error standard. The error between the ith block in the reference frame and the jth block in the current frame is defined as:
Figure BDA0002768968080000041
wherein alpha isiRepresenting the major axis and horizontal square of the MSER elliptical regionAngle of inclination of direction, siArea of the representative region, /)iRepresents the ratio of the length of the major axis to the minor axis of the region, xiRepresenting the horizontal coordinate, and gamma is an empirical coefficient.
And step 23, selecting three pairs of matching with the minimum error in the feature block matching of the two frames, taking the average value of the horizontal coordinate difference of the center of the MSER area in the three pairs of matching as the motion estimation of the current frame relative to the reference frame, and translating the current frame according to the motion estimation.
Step 3, applying a low-rank matrix decomposition method to the processed image for image stabilization, and converting each frame of image into a column vector D1,…,DnThen arranging the two together to form D, and recovering a low-rank matrix A and a series of geometric transformation tau ═ tau from D1,…,τnSolving the image stabilization problem, and resolving an error E. Wherein the geometric transformation will be performed on each column of D separately, i.e. D ° τ; the method comprises the following specific steps:
step 31, using each frame of the video as an image
Figure BDA0002768968080000042
Inputting, setting initial value tau of deformation parameter1,…,τn；
Step 32, respectively calculating jacobian matrixes related to tau:
Figure BDA0002768968080000043
step 33, deforming and normalizing the image matrix:
Figure BDA0002768968080000044
step 34, solving a linear convex optimization problem;
Figure BDA0002768968080000045
Figure BDA0002768968080000046
wherein Δ τ is the amount of change in strain parameter, A*,E*,Δτ*Solving A, E and delta tau of the iteration, wherein lambda is a parameter for balancing sparsity of solution and error, and A is a low-rank matrix;
step 35, update the morph parameter τ ← τ + Δ τ*Repeating the steps until convergence;
and step 36, synthesizing the processed image sequences into a video for output, and finishing image stabilization.
As shown in FIG. 2, the comparison result of the invention with the original image and 2 representative methods (including the optical flow method and the VidoProc method) on six sea surface images is given by SSIM value between two adjacent frames, and it can be seen that the invention has better image stabilization effect on the sea surface images.

Claims (5)

1. A sea surface image stabilizing method based on MSER region matching and low-rank matrix decomposition is characterized by comprising the following steps:
step 1, detecting a sea antenna in a sea surface image by using a Hough transformation algorithm;
step 2, aligning the image by using a MSER method, which comprises the following specific steps:
step 21, MSER feature detection is carried out on the sea surface part of the image to obtain an MSER feature block in the frame;
step 22, MSER feature detection is carried out on the next frame image according to step 21 to obtain an MSER feature block, and the error of the MSER feature block in the two frame images is calculated;
step 23, selecting three pairs of MSER feature blocks with the minimum error as feature matching blocks, taking the average value of MSER region center horizontal coordinate differences in the three pairs of feature matching blocks as motion estimation of the current frame relative to a reference frame, and translating the current frame according to the motion estimation;
and 3, stabilizing the image processed in the step 2 by using a low-rank matrix decomposition method.
2. The sea surface image stabilization method based on MSER region matching and low-rank matrix decomposition according to claim 1, characterized in that the specific steps of using Hough transform algorithm to detect sea-sky-line in sea surface image are as follows:
step 11, binarizing the image by taking the average value of the sky gray value and the sea gray value in the image as a threshold value;
step 12, edge detection is carried out by using a Canny method;
step 13, carrying out Hough straight line detection according to the edge detected by the Canny algorithm;
step 14, searching a straight line with the length above a set threshold value and the top left end in the candidate straight lines, and selecting the straight line as the sea-sky line of the frame;
and step 15, rotating the image by taking the sea-sky-line rotated to the horizontal position as a reference, and translating the sea-sky-line to the image center height position as a reference.
3. The sea surface image stabilization method based on MSER region matching and low rank matrix decomposition according to claim 1, characterized in that, the specific steps of MSER feature detection on the sea surface part of the image comprise: binarizing the image by taking [0,255] as a threshold value, detecting a connected domain with the area change lower than the threshold value in the process of increasing the threshold value, inverting the image, then binarizing once again and searching the connected domain, and selecting a region with the area of total images 1/5000 to 1/50 as the MSER feature block of the frame.
4. The method for sea surface image stabilization based on MSER region matching and low rank matrix decomposition of claim 1, wherein the error between the ith block in the reference frame and the jth block in the current frame is calculated as
Figure FDA0002768968070000021
Wherein alpha isiRepresents the included angle between the major axis of the MSER elliptical area and the horizontal direction, siArea of the representative region, /)iRepresenting the major and minor axes of the regionLength ratio, xiRepresenting the horizontal coordinate, and gamma is an empirical coefficient.
5. The sea surface image stabilization method based on MSER region matching and low rank matrix decomposition according to claim 1, characterized in that the step of stabilizing images by using low rank matrix decomposition in step 3 is:
step 31, using each frame of the video as an image
Figure FDA0002768968070000022
Inputting, setting initial value tau of deformation parameter1,…,τn；
Step 32, calculating a jacobian matrix about τ:
Figure FDA0002768968070000023
step 33, deforming and normalizing the image matrix:
Figure FDA0002768968070000024
step 34, solving a linear convex optimization problem;
Figure FDA0002768968070000025
Figure FDA0002768968070000026
wherein Δ τ is the amount of change in strain parameter, A*,E*,Δτ*Solving A, E and delta tau of the iteration, wherein lambda is a parameter for balancing sparsity of solution and error, and A is a low-rank matrix;
step 35, update the morph parameter τ ← τ + Δ τ*If not, go back to step 32, otherwise go to stepStep 36;
and step 36, synthesizing the processed image sequence into a video and outputting the video.
CN202011242901.3A 2020-11-09 2020-11-09 Sea surface image stabilization method based on MSER region matching and low-rank matrix decomposition Active CN112288655B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202011242901.3A CN112288655B (en) 2020-11-09 2020-11-09 Sea surface image stabilization method based on MSER region matching and low-rank matrix decomposition

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202011242901.3A CN112288655B (en) 2020-11-09 2020-11-09 Sea surface image stabilization method based on MSER region matching and low-rank matrix decomposition

Publications (2)

Publication Number Publication Date
CN112288655A true CN112288655A (en) 2021-01-29
CN112288655B CN112288655B (en) 2022-11-01

Family

ID=74351672

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202011242901.3A Active CN112288655B (en) 2020-11-09 2020-11-09 Sea surface image stabilization method based on MSER region matching and low-rank matrix decomposition

Country Status (1)

Country Link
CN (1) CN112288655B (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116128877A (en) * 2023-04-12 2023-05-16 山东鸿安食品科技有限公司 Intelligent exhaust steam recovery monitoring system based on temperature detection

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106357958A (en) * 2016-10-10 2017-01-25 山东大学 Region-matching-based fast electronic image stabilization method
CN106529591A (en) * 2016-11-07 2017-03-22 湖南源信光电科技有限公司 Improved MSER image matching algorithm
CN107862705A (en) * 2017-11-21 2018-03-30 重庆邮电大学 A kind of unmanned plane small target detecting method based on motion feature and deep learning feature
CN110414375A (en) * 2019-07-08 2019-11-05 北京国卫星通科技有限公司 Recognition methods, device, storage medium and the electronic equipment of low target

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106357958A (en) * 2016-10-10 2017-01-25 山东大学 Region-matching-based fast electronic image stabilization method
CN106529591A (en) * 2016-11-07 2017-03-22 湖南源信光电科技有限公司 Improved MSER image matching algorithm
CN107862705A (en) * 2017-11-21 2018-03-30 重庆邮电大学 A kind of unmanned plane small target detecting method based on motion feature and deep learning feature
CN110414375A (en) * 2019-07-08 2019-11-05 北京国卫星通科技有限公司 Recognition methods, device, storage medium and the electronic equipment of low target

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116128877A (en) * 2023-04-12 2023-05-16 山东鸿安食品科技有限公司 Intelligent exhaust steam recovery monitoring system based on temperature detection

Also Published As

Publication number Publication date
CN112288655B (en) 2022-11-01

Similar Documents

Publication Publication Date Title
CN111260543B (en) Underwater image splicing method based on multi-scale image fusion and SIFT features
Smith et al. Effective Corner Matching.
US9846818B2 (en) Objective assessment method for color image quality based on online manifold learning
CN109584282B (en) Non-rigid image registration method based on SIFT (scale invariant feature transform) features and optical flow model
CN110866924A (en) Line structured light center line extraction method and storage medium
CN106570913B (en) monocular SLAM rapid initialization method based on characteristics
CN111079545A (en) Three-dimensional target detection method and system based on image restoration
CN109376641B (en) Moving vehicle detection method based on unmanned aerial vehicle aerial video
CN107977960B (en) Saloon car surface scratch detection algorithm based on improved SUSAN operator
CN111583279A (en) Super-pixel image segmentation method based on PCBA
CN108805915B (en) Visual angle change resistant close-range image region feature matching method
CN102819827A (en) Self-adaption moment matching stripe noise removing method based on gray-level segmentation
CN108346135A (en) A kind of improved Criminisi image repair methods
CN112288655B (en) Sea surface image stabilization method based on MSER region matching and low-rank matrix decomposition
CN115471682A (en) Image matching method based on SIFT fusion ResNet50
CN111639560A (en) Finger vein feature extraction method and device based on dynamic fusion of vein skeleton line and topographic relief characteristic
CN109741258B (en) Image super-resolution method based on reconstruction
CN110751680A (en) Image processing method with fast alignment algorithm
Wang et al. Depth map recovery based on a unified depth boundary distortion model
KR101766431B1 (en) Method and apparatus for detecting disparity by using hierarchical stereo matching
CN114066954A (en) Feature extraction and registration method for multi-modal images
CN106651756B (en) Image registration method based on SIFT and verification mechanism
CN113255449A (en) Real-time matching method of binocular video images
CN111311657A (en) Infrared image homologous registration method based on improved corner main direction distribution
CN115760549A (en) Processing method for flattening 3D data of curved surface

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant