CN109993708B - Underwater distorted image recovery method based on registration of dark primary color and B spline - Google Patents

Underwater distorted image recovery method based on registration of dark primary color and B spline Download PDF

Info

Publication number
CN109993708B
CN109993708B CN201910163082.4A CN201910163082A CN109993708B CN 109993708 B CN109993708 B CN 109993708B CN 201910163082 A CN201910163082 A CN 201910163082A CN 109993708 B CN109993708 B CN 109993708B
Authority
CN
China
Prior art keywords
image
iteration
sequence
primary color
restored
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201910163082.4A
Other languages
Chinese (zh)
Other versions
CN109993708A (en
Inventor
廖翔宇
徐向民
青春美
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
South China University of Technology SCUT
Original Assignee
South China University of Technology SCUT
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by South China University of Technology SCUT filed Critical South China University of Technology SCUT
Priority to CN201910163082.4A priority Critical patent/CN109993708B/en
Publication of CN109993708A publication Critical patent/CN109993708A/en
Application granted granted Critical
Publication of CN109993708B publication Critical patent/CN109993708B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/77—Retouching; Inpainting; Scratch removal
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10004—Still image; Photographic image

Landscapes

  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Image Processing (AREA)

Abstract

The invention discloses an underwater distorted image restoration method based on registration of a dark primary color and a B spline. The method effectively solves the problem of recovering the underwater distorted image by a dark primary color prior and B spline iterative registration method. The method has higher robustness to the environment and better recovery effect, can save more image details while removing distortion, has higher speed, and improves the adverse effect of excessive distortion on image recovery.

Description

Underwater distorted image recovery method based on registration of dark primary color and B spline
Technical Field
The invention relates to the technical field of underwater image processing, in particular to an underwater distorted image recovery method based on registration of dark primary colors and B splines.
Background
When people want to shoot the underwater situation on the shore, the underwater picture which needs to be shot often shows a distorted state due to the disturbance of the water surface. This is because, with the fluctuation of the water body, the propagation direction of the light ray changes due to refraction when passing through the two propagation media, which in turn causes the viewed picture to assume a distorted state. This not only results in an overall poor image forming effect, but also brings inconvenience to subsequent operations such as image processing. Therefore, the restoration of the distorted image is an important issue.
Since the fluctuation of water has the characteristics of randomness and high speed, the recovery of a distorted image by using a single-frame picture is very difficult, and the recovery of the distorted image is mainly a method of acquiring a single-frame non-distorted picture by using a video sequence. The traditional warped image recovery algorithm mainly comprises a video sequence mean algorithm and a lucky block algorithm. The video sequence mean algorithm is to calculate the mean value or the median value of the sequence for the pixel points at the corresponding positions of all the pictures in the video sequence so as to obtain the pictures without distortion. This method is less effective in situations where the water surface fluctuates significantly. The lucky block algorithm assumes that there will always be a frame of picture in the video sequence with distortion-free regions, and the algorithm finds out the distortion-free regions in each frame of picture and then splices these regions together, thereby obtaining a distortion-free picture. However, if there are no distortion-free regions in the video, the effect of this algorithm is not ideal.
At present, an algorithm for two-step iterative warped image restoration is proposed, wherein a reference picture with relatively small distortion is obtained by adopting a video sequence mean algorithm or a lucky block algorithm in the first step, then all pictures of a video sequence are subjected to iterative registration on the reference picture by adopting a B-spline to reduce distortion, and finally a warped restored image is obtained. The method can effectively realize the distortion recovery of the image, however, if the image with larger distortion exists in the video, the edge distortion of the recovered image can still be caused, and the robustness is poor. Image sampling and B-spline iterative algorithms based on atmospheric disturbance image recovery are proposed, which process images by Laplace filtering and calculate the sum of pixel values as a warping parameter. However, this method is not very effective underwater due to the particularities of the underwater environment.
Disclosure of Invention
The invention aims to overcome the defects of the prior art and provide an underwater distorted image recovery method based on registration of a dark primary color and a B spline aiming at the robustness problem of underwater distorted video recovery. The method has the advantages of high robustness and good distortion recovery effect.
In order to achieve the purpose, the technical scheme provided by the invention is as follows: an underwater distorted image recovery method based on registration of dark primary colors and B splines comprises the following steps:
1) calculating a sequence mean value of an original image sequence to generate an initial standard image, then calculating a dark primary color image of each frame of image, using all pixel values of each frame of dark primary color image as distortion parameters, and normalizing;
2) iterating the image sequence by adopting a dark channel prior method to obtain a proper image sequence to be restored and a standard image, which are specifically as follows:
2.1) calculating a torsion degree evaluation parameter of each frame of image by using the image sequence to be restored, the standard image and the torsion parameter which are generated by the last iteration;
2.2) calculating energy parameters by using the torsion evaluation parameters generated in the step 2.1), selecting a sequence with the minimum energy parameters to obtain an image sequence to be restored, and generating a new standard image by using the obtained image sequence to be restored;
3) when judging the iteration stopping condition, if not, continuing the iteration in the step 2); if the iteration stopping condition is met, stopping iteration, taking the image sequence to be recovered of the iteration result and the standard image to perform B-spline iterative registration, wherein the iteration steps are as follows:
3.1) B-spline transformation is carried out by using the image sequence to be restored of the previous iteration, the normalized mutual information value of the transformed image and the standard image is calculated, and the image with the largest value is used as a new image sequence to be restored;
3.2) calculating a sequence mean value of the new image sequence to be restored generated in the step 3.1) to obtain a new standard image;
4) judging whether the iteration reaches the maximum iteration value, if not, continuing the iteration in the step 3); and if the iteration stopping condition is met, stopping iteration, and taking the finally generated standard image as the optimal estimation of the distorted image recovery.
In step 1), the standard image is defined as:
Figure BDA0001985350020000031
in the formula I0Is a standard image, IiRepresenting the ith picture of the image sequence to be restored; considering that the dark primary color image of the distorted image has higher pixel values than the undistorted dark primary color image, the sum of the pixel values of the dark primary color image of each frame image is used as a distortion parameter, and the higher the distortion parameter is, the greater the distortion degree of the image is represented; dark primary color image DiDefined as the minimum filtered image of an image:
Figure BDA0001985350020000032
in the formula, Ω (x) represents a window centered on a pixel x; and then summing the pixel values of each dark primary color image, wherein the obtained value is the distortion parameter Si：
Si＝||Di||1
In the formula, | · the luminance | |1Means for summing pixel values of the image; in finding all SiThen, normalizing the distortion parameter of the picture to obtain a normalized distortion parameter Q (I) of the corresponding picturei)：
Figure BDA0001985350020000041
Calculating a standard image I0With each frame image IiThe similarity of the data is measured by using Euclidean distance as similarity measure, and then the similarity measure is added with a weighted distortion parameter Q to be used as a distortion evaluation parameter Niλ is a weight parameter:
Figure BDA0001985350020000042
considering that more pictures bring more information, but not all pictures are suitable for recovery of the twisted image, a regularization term is needed to weigh the number of images; using a concave increasing function of incremental decrease as the regularization term, the final energy parameter is defined as follows:
Figure BDA0001985350020000043
in the formula, J represents the selected image sequence, | J | represents the number of the image sequences, τ is a weight parameter of a regularization term, and ρ is a normal number used for balancing the importance of the number of the selected images;
in order to find the optimal image more quickly, namely the image with the maximum mutual information value with the standard image, a gradient descent method is adopted for iterative optimization.
Compared with the prior art, the invention has the following advantages and beneficial effects:
the method effectively solves the problem of recovery of the underwater distorted image through the dark channel prior and the B spline iterative registration algorithm. Due to the particularity of the underwater environment, the method for sampling the distorted images in the atmosphere is not suitable for the underwater environment, so that the method improves the sampling algorithm by utilizing the dark primary color prior information, and is suitable for the underwater environment. Because the image with larger distortion is removed, the method has better recovery effect, can save more image details while removing the image distortion, has higher robustness and higher speed, is more beneficial to engineering realization, and is worthy of popularization.
Drawings
FIG. 1 is a block diagram of the process of the present invention.
Fig. 2 is a block diagram of a sampling algorithm flow.
Fig. 3 is a block diagram of the B-spline iterative algorithm flow.
Fig. 4a is an underwater distorted image.
Fig. 4b is an underwater undistorted image.
Fig. 4c is a dark primary image of an underwater warped image.
Fig. 4d is a dark primary image of an underwater undistorted image.
Fig. 5a is a conventional mean standard image of an image sequence.
Fig. 5b is a mean standard image obtained after sampling.
Detailed Description
The present invention will be further described with reference to the following specific examples.
As shown in fig. 1 to 5B, the method for restoring an underwater distorted image based on registration of a dark primary color and a B-spline provided in this embodiment includes the following steps:
1) calculating a sequence mean value of an original image sequence to generate an initial standard image, calculating a dark primary color image of each frame of image, using all pixel values of each frame of dark primary color image as distortion parameters, and normalizing, wherein the method comprises the following specific steps:
1.1) generating an initial standard image, and taking an image sequence mean picture as an initial standard image I0The formula is as follows:
Figure BDA0001985350020000051
1.2) calculating dark primary color images of all images, wherein the dark primary color images refer to images generated by minimum value filtering of the images, and the whole images are obtained as operation objects. Dark primary color image D of ith pictureiIs defined as:
Figure BDA0001985350020000052
where Ω (x) denotes a window centered on pixel x.
1.3) calculating the sum of pixel values of the i-th frame of dark primary color image as a distortion parameter Si
Si＝||Di||1
In the formula, | · the luminance | |1Means that the pixel values of the image are summed, after the warping parameters of all dark primary images are found, they are normalized to obtain the normalized warping parameter Q (I) of the corresponding imagei)：
Figure BDA0001985350020000061
2) And iterating the image sequence by adopting a dark channel prior method to obtain a proper image sequence to be restored and a standard image. We select a suitable image sequence to be restored according to the values of an energy equation defined as:
Figure BDA0001985350020000062
in the formula, J represents an image sequence to be restored, | J | represents the number of the image sequences, and λ, τ, ρ are weight parameters. Wherein the unknown parameter is a standard image I0And an image sequence J to be restored. Because the formula contains two unknown parameters, the common solving method cannot solve the problem. Therefore, we choose to solve using an iterative approach. The steps of the tth iteration are shown below:
the first step is as follows: fixing
Figure BDA0001985350020000063
Finding the image sequence J to be restoredtThen, the following is obtained:
Figure BDA0001985350020000064
to understand this equation, we can first calculate
Figure BDA0001985350020000065
Namely, the torsion degree evaluation parameter N of the k-th imagek. Then sort it in ascending order by size:
N1≤N2≤...≤Nn
then substituted into an energy formula with SjTo show that:
Figure BDA0001985350020000066
in the formula, j represents an imageThe number of (2); this is an accumulative process, S1Represents the k-th1Energy coefficient of picture, S2Represents the k-th1And k is2Energy coefficient of picture, SnThe energy coefficients of the n pictures are represented.
Then, the picture sequence with the minimum energy coefficient is taken as Jt。
Second step, fix JtObtaining a standard image
Figure BDA0001985350020000071
Namely, the following are obtained:
Figure BDA0001985350020000072
due to JtAfter being fixed, the above formula is solved:
Figure BDA0001985350020000073
the above formula is derived from I, and the solution of the above formula is the image sequence J to be restoredtMean image of (2):
Figure BDA0001985350020000074
repeating the first step and the second step until the difference between the energy coefficients of the previous iteration and the next iteration is less than a constant epsilon, and taking the image sequence J to be restored obtained by the last iteration and the average value image I thereof0As input to the B-spline iterative registration algorithm.
3) Using a B spline iterative registration algorithm to combine the image sequence J to be restored obtained in the step 2) with the standard image I0Performing iterative registration, specifically comprising the following steps:
3.1) use size sx×syAnd carrying out cubic B spline iterative transformation on the image to be restored by the control point grid with the distance delta to obtain a transformed image. The transformation is completed by calculating the displacement value D (x, y) of each pixel point of each frame of image to be restored, namely:
Figure BDA0001985350020000075
in the formula (I), the compound is shown in the specification,
Figure BDA0001985350020000076
Figure BDA0001985350020000077
indicating a rounding down operation. Where B is a standard B-spline basis function defined as:
Figure BDA0001985350020000081
Figure BDA0001985350020000082
Figure BDA0001985350020000083
Figure BDA0001985350020000084
where Φ represents the value of a 4 × 4 control grid point around the pixel point in the control grid, and m and l represent the distances between the surrounding pixel points and the control point, respectively.
3.2) calculating the image I after the first step of transformation and the standard image I0Normalized mutual information value NMI (I, I)0) It is defined as:
Figure BDA0001985350020000085
wherein H: (I0) As a standard image I0Entropy of, i.e.
Figure BDA0001985350020000086
The entropy H (I), H (I) of the transformed image can be obtained by the same method0And I) is a standard image I0Joint entropy with transformed image I, i.e.
Figure BDA0001985350020000087
And then taking the maximum normalized mutual information value and the corresponding image as an optimal solution.
For faster acquisition of optimal solution of image, we adopt gradient descent method to perform iterative optimization, and NMI (I, I)0) Instead of NMI (I, I)0Phi), indicating being affected by phi. The gradient descent method comprises the following iteration steps:
initializing phi to 0, and calculating NMI (I, I)0Φ) gradient vector G versus Φ:
Figure BDA0001985350020000088
if the value of G is greater than a normal number eta, updating
Figure BDA0001985350020000089
Figure BDA0001985350020000091
Alpha is a weight parameter of the gradient; then calculate
Figure BDA0001985350020000092
If Δ NMI is less than or equal to 0, then update Φ to
Figure BDA0001985350020000093
And repeating the steps, otherwise, ending the iteration.
3.3) calculating the mean value of the transformed image sequence as the standard image of the next iteration.
And repeating the steps 3.1) to 3.3) until a set iteration threshold value is reached, and taking the standard image of the last iteration as the final underwater distortion recovery image.
The above-mentioned embodiments are merely preferred embodiments of the present invention, and the scope of the present invention is not limited thereto, so that the changes in the shape and principle of the present invention should be covered within the protection scope of the present invention.

Claims (1)

1. An underwater distorted image recovery method based on registration of a dark primary color and a B spline is characterized by comprising the following steps:
1) calculating a sequence mean value of an original image sequence to generate an initial standard image, then calculating a dark primary color image of each frame of image, using all pixel values of each frame of dark primary color image as distortion parameters, and normalizing;
the standard image is defined as:
Figure FDA0002809513360000011
in the formula I0Is a standard image, IiRepresenting the ith picture of the image sequence to be restored; considering that the dark primary color image of the distorted image has higher pixel values than the undistorted dark primary color image, the sum of the pixel values of the dark primary color image of each frame image is used as a distortion parameter, and the higher the distortion parameter is, the greater the distortion degree of the image is represented; dark primary color image DiDefined as the minimum filtered image of an image:
Figure FDA0002809513360000012
in the formula, Ω (x) represents a window centered on a pixel x; and then summing the pixel values of each dark primary color image, wherein the obtained value is the distortion parameter Si：
Si＝||Di||1
In the formula, | · the luminance | |1Means for summing pixel values of the image; in finding all SiThen, normalizing the distortion parameter of the picture to obtain a normalized distortion parameter Q (I) of the corresponding picturei)：
Figure FDA0002809513360000013
Calculating a standard image I0With each frame image IiThe similarity of the data is measured by using Euclidean distance as similarity measure, and then the similarity measure is added with a weighted distortion parameter Q to be used as a distortion evaluation parameter Niλ is a weight parameter:
Figure FDA0002809513360000014
considering that more pictures bring more information, but not all pictures are suitable for recovery of the twisted image, a regularization term is needed to weigh the number of images; using a concave increasing function of incremental decrease as the regularization term, the final energy parameter is defined as follows:
Figure FDA0002809513360000021
in the formula, J represents the selected image sequence, | J | represents the number of the image sequences, τ is a weight parameter of a regularization term, and ρ is a normal number used for balancing the importance of the number of the selected images;
in order to find the optimal image more quickly, namely the image with the maximum mutual information value with the standard image, a gradient descent method is adopted for iterative optimization;
2) iterating the image sequence by adopting a dark channel prior method to obtain a proper image sequence to be restored and a standard image, which are specifically as follows:
2.1) calculating a torsion degree evaluation parameter of each frame of image by using the image sequence to be restored, the standard image and the torsion parameter which are generated by the last iteration;
2.2) calculating energy parameters by using the torsion evaluation parameters generated in the step 2.1), selecting a sequence with the minimum energy parameters to obtain an image sequence to be restored, and generating a new standard image by using the obtained image sequence to be restored;
3) when judging the iteration stopping condition, if not, continuing the iteration in the step 2); if the iteration stopping condition is met, stopping iteration, taking the image sequence to be recovered of the iteration result and the standard image to perform B-spline iterative registration, wherein the iteration steps are as follows:
3.1) B-spline transformation is carried out by using the image sequence to be restored of the previous iteration, the normalized mutual information value of the transformed image and the standard image is calculated, and the image with the largest value is used as a new image sequence to be restored;
3.2) calculating a sequence mean value of the new image sequence to be restored generated in the step 3.1) to obtain a new standard image;
4) judging whether the iteration reaches the maximum iteration value, if not, continuing the iteration in the step 3); and if the iteration stopping condition is met, stopping iteration, and taking the finally generated standard image as the optimal estimation of the distorted image recovery.
CN201910163082.4A 2019-03-05 2019-03-05 Underwater distorted image recovery method based on registration of dark primary color and B spline Active CN109993708B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201910163082.4A CN109993708B (en) 2019-03-05 2019-03-05 Underwater distorted image recovery method based on registration of dark primary color and B spline

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201910163082.4A CN109993708B (en) 2019-03-05 2019-03-05 Underwater distorted image recovery method based on registration of dark primary color and B spline

Publications (2)

Publication Number Publication Date
CN109993708A CN109993708A (en) 2019-07-09
CN109993708B true CN109993708B (en) 2021-03-30

Family

ID=67129475

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201910163082.4A Active CN109993708B (en) 2019-03-05 2019-03-05 Underwater distorted image recovery method based on registration of dark primary color and B spline

Country Status (1)

Country Link
CN (1) CN109993708B (en)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111445505B (en) * 2020-03-25 2023-05-30 哈尔滨工程大学 Water-to-air imaging distortion correction algorithm based on secondary registration
CN111524083B (en) * 2020-05-07 2023-05-05 桂林电子科技大学 Active-passive combined underwater empty imaging image recovery method based on structured light

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106529533A (en) * 2016-11-07 2017-03-22 湖南源信光电科技有限公司 Complex weather license plate positioning method based on multi-scale analysis and matched sequencing
CN107203977A (en) * 2017-05-17 2017-09-26 河海大学 A kind of underwater image restoration method based on dark primary priori and rarefaction representation

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20140105515A1 (en) * 2012-04-04 2014-04-17 The Regents Of The University Of California Stabilizing and Deblurring Atmospheric Turbulence
US9305242B2 (en) * 2014-08-15 2016-04-05 National Taipei University Of Technology Method and image processing apparatus for image visibility restoration using fisher's linear discriminant based dual dark channel prior

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106529533A (en) * 2016-11-07 2017-03-22 湖南源信光电科技有限公司 Complex weather license plate positioning method based on multi-scale analysis and matched sequencing
CN107203977A (en) * 2017-05-17 2017-09-26 河海大学 A kind of underwater image restoration method based on dark primary priori and rarefaction representation

Non-Patent Citations (5)

* Cited by examiner, † Cited by third party
Title
Underwater image enhancement with an adaptive dehazing framework;Chunmei Qing等;《2015 IEEE International Conference on Digital Signal Processing (DSP)》;20150910;第339-342页 *
Underwater image restoration using single color channel prior;Samarth Borkar等;《 2016 International Conference on Signal and Information Processing (IConSIP)》;20170216;第1-4页 *
基于B样条迭代配准的水下扭曲图像恢复算法;马春波等;《计算机工程》;20170531;第43卷(第5期);第224-228页 *
基于暗原色先验原理的水下图像增强;朱振杰等;《图像与视频处理》;20180615;第453-462页 *
基于暗原色原理和B样条小波的雾霾车牌定位的研究;鞠丽丽;《中国优秀硕士学位论文全文数据库信息科技辑》;20170315;第2017年卷(第03期);I138-5295 *

Also Published As

Publication number Publication date
CN109993708A (en) 2019-07-09

Similar Documents

Publication Publication Date Title
CN111915484B (en) Reference image guiding super-resolution method based on dense matching and self-adaptive fusion
CN107578392B (en) Convolution neural network demosaicing method based on residual interpolation
WO2021147095A1 (en) Deep residual network for color filter array image denoising
WO2020015330A1 (en) Enhanced neural network-based image restoration method, storage medium, and system
CN111161360B (en) Image defogging method of end-to-end network based on Retinex theory
CN109993708B (en) Underwater distorted image recovery method based on registration of dark primary color and B spline
CN109118446B (en) Underwater image restoration and denoising method
Dharejo et al. A color enhancement scene estimation approach for single image haze removal
CN110047091B (en) Image stabilization method based on camera track estimation and feature block matching
CN113298810A (en) Trace detection method combining image enhancement and depth convolution neural network
CN111932452B (en) Infrared image convolution neural network super-resolution method based on visible image enhancement
CN114511480A (en) Underwater image enhancement method based on fractional order convolution neural network
CN111626943B (en) Total variation image denoising method based on first-order forward and backward algorithm
CN111340741B (en) Particle swarm optimization gray image enhancement method based on quaternion and L1 norm
CN116433525A (en) Underwater image defogging method based on edge detection function variation model
CN108257157B (en) Improved moving target detection method based on Lab color space and ViBe
WO2022111090A1 (en) Image de-scattering method based on atmospheric transmission matrix iterative optimization
CN115937021A (en) Polarization defogging method based on frequency domain feature separation and iterative optimization of atmospheric light
Shibata et al. Reflection removal using RGB-D images
CN109801238B (en) Weak illumination image enhancement method based on simplified atmospheric scattering model
CN111986123B (en) Blurred image restoration method based on KL divergence and L0 norm constraint
CN107424173B (en) Target tracking method based on extended local invariant feature description
CN116188333B (en) Multiplicative additive mixed noise removal method based on structured integral least square
CN114820387B (en) Image recovery method and terminal based on probability induction nuclear norm minimization
CN114066764B (en) Sand and dust degradation image enhancement method and device based on distance weighted color cast estimation

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant