CN114418902B - Image restoration method and computer-readable storage medium - Google Patents
Image restoration method and computer-readable storage medium Download PDFInfo
- Publication number
- CN114418902B CN114418902B CN202210327415.4A CN202210327415A CN114418902B CN 114418902 B CN114418902 B CN 114418902B CN 202210327415 A CN202210327415 A CN 202210327415A CN 114418902 B CN114418902 B CN 114418902B
- Authority
- CN
- China
- Prior art keywords
- filled
- image
- pixel
- value
- pixel points
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 93
- 238000003860 storage Methods 0.000 title claims abstract description 11
- 238000012545 processing Methods 0.000 claims abstract description 156
- 238000001914 filtration Methods 0.000 claims abstract description 71
- 230000009467 reduction Effects 0.000 claims abstract description 50
- 230000003044 adaptive effect Effects 0.000 claims abstract description 40
- 238000005070 sampling Methods 0.000 claims abstract description 8
- 230000008569 process Effects 0.000 claims description 29
- 238000009792 diffusion process Methods 0.000 claims description 11
- 238000000605 extraction Methods 0.000 claims description 10
- 239000011159 matrix material Substances 0.000 claims description 8
- 230000003247 decreasing effect Effects 0.000 claims description 5
- 238000004422 calculation algorithm Methods 0.000 claims description 2
- 230000006870 function Effects 0.000 description 22
- 238000004364 calculation method Methods 0.000 description 9
- 238000004590 computer program Methods 0.000 description 9
- 238000005259 measurement Methods 0.000 description 5
- 230000003287 optical effect Effects 0.000 description 5
- 238000013135 deep learning Methods 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 3
- NAWXUBYGYWOOIX-SFHVURJKSA-N (2s)-2-[[4-[2-(2,4-diaminoquinazolin-6-yl)ethyl]benzoyl]amino]-4-methylidenepentanedioic acid Chemical compound C1=CC2=NC(N)=NC(N)=C2C=C1CCC1=CC=C(C(=O)N[C@@H](CC(=C)C(O)=O)C(O)=O)C=C1 NAWXUBYGYWOOIX-SFHVURJKSA-N 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 238000009776 industrial production Methods 0.000 description 2
- 238000000691 measurement method Methods 0.000 description 2
- 238000012549 training Methods 0.000 description 2
- 230000009466 transformation Effects 0.000 description 2
- 102100029469 WD repeat and HMG-box DNA-binding protein 1 Human genes 0.000 description 1
- 101710097421 WD repeat and HMG-box DNA-binding protein 1 Proteins 0.000 description 1
- 230000002159 abnormal effect Effects 0.000 description 1
- 230000003321 amplification Effects 0.000 description 1
- 230000002146 bilateral effect Effects 0.000 description 1
- 230000000903 blocking effect Effects 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 238000007796 conventional method Methods 0.000 description 1
- 230000007797 corrosion Effects 0.000 description 1
- 238000005260 corrosion Methods 0.000 description 1
- 230000008878 coupling Effects 0.000 description 1
- 238000010168 coupling process Methods 0.000 description 1
- 238000005859 coupling reaction Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000009795 derivation Methods 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 239000000428 dust Substances 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 238000009499 grossing Methods 0.000 description 1
- 238000005286 illumination Methods 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 239000000463 material Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000003199 nucleic acid amplification method Methods 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 238000010422 painting Methods 0.000 description 1
- 238000007670 refining Methods 0.000 description 1
- 230000008439 repair process Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000001131 transforming effect Effects 0.000 description 1
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/77—Retouching; Inpainting; Scratch removal
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20004—Adaptive image processing
Landscapes
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Image Processing (AREA)
Abstract
An image restoration method and a computer-readable storage medium perform reduction and filtering processing on an image to be restored according to an adaptive reduction parameter and a filtering parameter to obtain a filtered image to be restored, acquire pixel points to be filled in the filtered image to be restored, acquire a better initial value of the pixel points to be filled, perform first iteration processing on the values of the pixel points to be filled based on a constraint condition that the sum of the gradients of all the pixel points to be filled is minimum and the initial values of the pixel points to be filled to obtain coarse processing values of the pixel points to be filled, perform second iteration processing on the values of the pixel points to be filled again as the initial values of the pixel points to be filled according to an energy equation and the initial values of the pixel points to be filled to obtain fine processing values of the pixel points to be filled, and use the fine processing values of the pixel points to be filled as target values of the pixel points to be filled, and the restored image is obtained by up-sampling, so that the image processing efficiency is improved.
Description
Technical Field
The present application relates to the field of image processing technologies, and in particular, to an image restoration method and a computer-readable storage medium.
Background
In daily life and industrial production, some images need to be repaired, and the repairing process can be as follows: eliminating date and watermark on the image; removing unnecessary content in the image, and filling the removed vacant area with reasonable content; repairing and filling cracks, scratches, flaws, etc. in the old photograph; the black and white photograph is restored to facilitate conversion into a color image or the like.
The main image restoration method at present is to perform image restoration based on deep learning, and the local part (to-be-filled area) of the generated image utilizes texture information of a known part of the image. The deep learning method is mainly based on an Encoder-Decoder (Encoder + Decoder) or a generated countermeasure Network (GAN) to build a model, so as to repair an image input into the model and output the repaired image.
However, the training cost and the calculation cost of the way of performing image inpainting based on deep learning are large.
Disclosure of Invention
The method mainly solves the technical problem that training cost and calculation cost of an image restoration method based on deep learning are high.
According to a first aspect, an embodiment provides an image inpainting method, comprising:
acquiring a self-adaptive reduction parameter and a self-adaptive filtering parameter;
reducing and filtering the image to be repaired according to the self-adaptive reduction parameter and the self-adaptive filtering parameter to obtain a filtered image to be repaired;
acquiring pixel points to be filled in the filtered image to be repaired and initial values of the pixel points to be filled;
performing first iteration processing on the value of the pixel point to be filled based on a constraint condition that the sum of the divergence of the gradient values of all the pixel points to be filled is minimum and the initial value of the pixel point to be filled to obtain a rough processing value of the pixel point to be filled;
taking the coarse processing value of the pixel point to be filled as the initial value of the pixel point to be filled again, carrying out second iterative processing on the value of the pixel point to be filled according to an energy equation and the initial value of the pixel point to be filled to obtain a fine processing value of the pixel point to be filled, and taking the fine processing value of the pixel point to be filled as the target value of the pixel point to be filled;
obtaining a restored reduced image according to the target value of the pixel to be filled and the values of other pixels except the pixel to be filled in the filtered image to be restored;
and performing up-sampling on the repaired reduced image to obtain a repaired image, wherein the size of the repaired image is the same as that of the image to be repaired.
According to a second aspect, an embodiment provides an image inpainting method, including:
acquiring a self-adaptive reduction parameter and a self-adaptive filtering parameter;
reducing and filtering the image to be repaired according to the self-adaptive reduction parameter and the self-adaptive filtering parameter to obtain a filtered image to be repaired;
acquiring pixels to be filled in the filtered image to be repaired and initial values of the pixels to be filled;
performing coarse processing or fine processing on the pixel points to be filled in the filtered image to be repaired to obtain target values of the pixel points to be filled;
the rough treatment comprises the following steps: based on a constraint condition of minimizing the sum of the divergence of the gradient values of all the pixels to be filled, performing first iteration processing according to the initial value of the pixels to be filled to obtain a rough processing value of the pixels to be filled, and taking the rough processing value of the pixels to be filled as a target value of the pixels to be filled;
the refining treatment comprises the following steps: performing second iterative processing on the value of the pixel point to be filled according to an energy equation and the initial value of the pixel point to be filled to obtain a fine processing value of the pixel point to be filled, and taking the fine processing value of the pixel point to be filled as a target value of the pixel point to be filled;
obtaining a restored reduced image according to the target value of the pixel to be filled and the values of other pixels except the pixel to be filled in the filtered image to be restored;
and performing up-sampling on the restored reduced image to obtain a restored image, wherein the size of the restored image is the same as that of the image to be restored.
According to a third aspect, an embodiment provides a computer readable storage medium having a program stored thereon, the program being executable by a processor to implement the method of the first or second aspect as described above.
According to a fourth aspect, there is provided in an embodiment an electronic device comprising: one or more processors; a memory; and one or more computer programs; wherein the one or more computer programs are stored in the memory; the one or more processors, when executing the one or more computer programs, cause the electronic device to implement the method of the first or second aspect as described above.
According to the image restoration method and the computer-readable storage medium of the embodiment, the adaptive reduction parameters and the adaptive filtering parameters are obtained through self-adaptation, so that the corresponding reduction parameters and the corresponding filtering parameters can be automatically matched according to the size of the area formed by the invalid pixel points in the image to be restored, the image to be restored is reduced and filtered based on the adaptive reduction parameters and the corresponding filtering parameters, the filtered image to be restored is obtained, the pixel points to be filled in the filtered image to be restored are obtained, the initial values of the better pixel points to be filled are obtained, the processing accuracy is higher, the image reduction and filtering can reduce the calculation amount of subsequent image processing under the condition of ensuring the accuracy, the calculation amount in the subsequent processing process is reduced, the image restoration processing efficiency is improved, and the constraint condition that the sum of the gradient values of all the pixel points to be filled is minimum and the initial value of the pixel points to be filled are based on the initial constraint condition that the sum of the gradient values of all the pixel points to be filled is minimum and the initial pixel points to be filled are obtained Performing first iterative processing on the value of the pixel point to be filled to obtain a rough processing value of the pixel point to be filled, thereby rapidly obtaining the rough processing value of the pixel point to be filled in an iterative mode, improving the efficiency of image restoration processing, taking the rough processing value of the pixel point to be filled as the initial value of the pixel point to be filled again, according to the energy equation and the initial value of the pixel point to be filled, performing second iterative processing on the value of the pixel point to be filled to obtain a fine processing value of the pixel point to be filled, taking the fine processing value of the pixel point to be filled as a target value of the pixel point to be filled, the energy equation in physics is introduced into the image processing field to carry out iterative solution, thereby improving the efficiency of image processing, the edge continuity of the repaired image is better, and the repaired image is more natural and more suitable for the application scene of real-time processing.
Drawings
Fig. 1 is a schematic flowchart of an image restoration method according to an embodiment of the present disclosure;
fig. 2 is a schematic flowchart of another image restoration method according to an embodiment of the present disclosure;
fig. 3 is a schematic flowchart of another image restoration method according to an embodiment of the present application;
fig. 4 is an image to be repaired according to an embodiment of the present disclosure;
fig. 5 is a filtered image to be repaired obtained from the image to be repaired shown in fig. 4 according to an embodiment of the present application;
fig. 6 is a rough processed image obtained from the filtered image to be repaired shown in fig. 5 according to an embodiment of the present disclosure;
FIG. 7 is a repaired image obtained from the rough processed image shown in FIG. 6 according to an embodiment of the present disclosure;
fig. 8 is an image after performing a refinement process on the coarse processed image shown in fig. 6 according to an embodiment of the present application.
Detailed Description
The present application will be described in further detail below with reference to the accompanying drawings by way of specific embodiments. Wherein like elements in different embodiments are numbered with like associated elements. In the following description, numerous details are set forth in order to provide a better understanding of the present application. However, those skilled in the art will readily recognize that some of the features may be omitted or replaced with other elements, materials, methods in different instances. In some instances, certain operations related to the present application have not been shown or described in detail in order to avoid obscuring the core of the present application from excessive description, and it is not necessary for those skilled in the art to describe these operations in detail, so that they may be fully understood from the description in the specification and the general knowledge in the art.
Furthermore, the features, operations, or characteristics described in the specification may be combined in any suitable manner to form various embodiments. Also, the various steps or actions in the method descriptions may be transposed or transposed in order, as will be apparent to one of ordinary skill in the art. Thus, the various sequences in the specification and drawings are for the purpose of describing certain embodiments only and are not intended to imply a required sequence unless otherwise indicated where such sequence must be followed.
The numbering of the components as such, e.g., "first", "second", etc., is used herein only to distinguish the objects as described, and does not have any sequential or technical meaning. The term "connected" and "coupled" when used in this application, unless otherwise indicated, includes both direct and indirect connections (couplings).
In daily life or industrial production, two-dimensional images or three-dimensional images are often used, and the images are often damaged by noises, such as dust or water drops on lenses, scratches of old photos, artificial paintings (such as mosaics) on the images, or damaged images, and the damaged images can be restored to the original state as far as possible by an image restoration method.
The two-dimensional image can be obtained by a linear array camera and an area-array camera. The two-dimensional image is usually repaired by repairing the pixel points with large difference with the surrounding texture information.
The three-dimensional image may be obtained by a three-dimensional vision sensor. The three-dimensional reconstruction is a process of scanning a measured object by using a certain means and method to obtain a surface space coordinate position of the measured object. Conventional methods typically extract the three-dimensional contour of an object using a three-dimensional coordinate machine. The method has long measuring period and is not suitable for detecting objects with soft surfaces due to the contact measuring characteristic. The optical three-dimensional measurement method is a non-contact three-dimensional measurement method, can measure an object in a non-contact manner, and has high measurement efficiency and high measurement speed. With the development of the fields of industry, computer vision, and the like, the performance requirements for the three-dimensional measurement technology are continuously improved, and the measurement of the fast dynamic object is gradually becoming an important direction for the development of the optical three-dimensional measurement technology.
In optical three-dimensional reconstruction, due to the influence of environmental factors such as illumination, shielding and the like, a three-dimensional image obtained frequently contains more holes and abnormal pixel points.
The method provided by the embodiment of the application can be applied to the field of two-dimensional image restoration and can also be applied to the field of three-dimensional image restoration. The pixel value of the pixel point in the two-dimensional image may be a gray value of the pixel point, and the pixel value of the pixel point in the three-dimensional image may be a depth value of the pixel point. In this application, the pixel value of a pixel is also referred to as a value of a pixel. In the application, a plane rectangular coordinate system can be established for the image no matter the two-dimensional image or the three-dimensional imageXYAnd each pixel point in the image has a corresponding coordinate, and the value of each pixel point is the gray value or the depth value of the corresponding coordinate of the rectangular coordinate system.
According to the image restoration method provided by the embodiment of the invention, the image to be restored is reduced and filtered according to the adaptive reduction parameter and the adaptive filtering parameter which are obtained by the region self-adaptively formed by the invalid pixel points of the image to be restored, the accuracy of subsequent image restoration is ensured, the calculated amount of image restoration is reduced, the image to be restored is processed based on the reduced and filtered image to be restored, the invalid pixel points (namely the pixel points to be filled) and the values of the valid pixel points around the invalid pixel points are processed and filled, the restored image with better imaging quality is obtained, and the image restoration process is high in speed and high in efficiency. The repaired image can be used in various scenes such as defect detection, attitude estimation and the like, various image application scenes with high real-time requirements are met, and the application accuracy in various scenes is improved.
The following describes the technical solutions of the present application and how to solve the above technical problems with specific embodiments.
Example one
Referring to fig. 1, fig. 1 is a schematic flowchart of an image repairing method provided in an embodiment of the present application, where the method provided in the embodiment is executed by an electronic device, and the electronic device may be a server, a computer, a smart phone, or a tablet device, and the present application is not limited thereto. The image restoration method provided by the embodiment comprises the following steps 11-19:
step 11: and acquiring an adaptive reduction parameter and an adaptive filtering parameter.
The adaptive reduction parameter is used for reducing the image to be repaired, and the adaptive reduction parameter is obtained according to the size of an invalid region formed by invalid pixel points in the image to be repaired. The adaptive downscaling parameter may be a downscaling scale.
The adaptive filtering parameter is used for filtering the image to be restored, and the adaptive filtering parameter is obtained according to the size of an invalid region formed by invalid pixel points in the image to be restored. The adaptive filter parameter may be a size of the filter kernel.
Step 12: and reducing and filtering the image to be repaired according to the self-adaptive reduction parameter and the self-adaptive filtering parameter to obtain the filtered image to be repaired.
And respectively carrying out reduction processing and filtering processing on the image to be repaired through the self-adaptive reduction parameter and the self-adaptive filtering parameter so as to obtain the filtered image to be repaired. It can be understood that the filtered image to be repaired is an image which is filtered to remove a part of noise and has a smaller size than the image to be repaired.
Step 13: and acquiring the pixel points to be filled in the filtered image to be repaired and the initial values of the pixel points to be filled.
The pixel points to be filled in the filtered image to be repaired can also be called invalid pixel points, and the pixel points to be filled refer to the invalid pixel points in the current filtered image to be repaired. The pixel points to be filled in the two-dimensional image are pixel points with larger difference of texture information of areas formed by the pixel points around the pixel points; the pixel points to be filled in the three-dimensional image refer to the pixel points beyond the field of view of the hardware.
After the reduction and filtering processes, it is necessary to obtain the to-be-filled pixel points of the currently filtered to-be-repaired image, that is, the invalid pixel points in the currently filtered to-be-repaired image. And determining pixel points to be filled in the filtered image to be repaired according to the filtered image to be repaired, and further determining initial values of the pixel points to be filled.
Step 14: and performing first iteration processing on the value of the pixel point to be filled based on the constraint condition that the sum of the divergence degrees of the gradient values of all the pixel points to be filled is minimum and the initial value of the pixel point to be filled to obtain a rough processing value of the pixel point to be filled.
This application refers to a process like that in step 14 as a rough process.
The target of image restoration is the sum of the divergence of the gradient values of all the pixels to be filled in the region where the pixels to be filled are located, so that the sum can be used as a constraint condition of image restoration. Because the constraint condition is expressed as a continuous partial differential equation, the constraint condition needs to be discretized when the constraint condition is applied to the image field, so that the value of the pixel point to be filled is continuously optimized in a first iteration processing mode until iteration is stopped, and the rough processing value of the pixel point to be filled is obtained.
Step 15: and taking the coarse processing value of the pixel point to be filled as the initial value of the pixel point to be filled again.
Step 16: and performing second iterative processing on the value of the pixel point to be filled according to the energy equation and the initial value of the pixel point to be filled to obtain a fine processing value of the pixel point to be filled.
This application refers to a process similar to that in step 16 as a refinement process.
An energy equation in physics can reflect the process of temperature diffusing to the surroundings along with time, and the energy equation is introduced into an image restoration method, wherein the value of a pixel point to be filled corresponds to the temperature in the energy equation. And performing second iteration processing on the energy equation applied to the image restoration field to obtain a fine processing value of the pixel point to be filled.
And step 17: and taking the fine processing value of the pixel point to be filled as the target value of the pixel point to be filled.
The target value of the pixel point to be filled is a result obtained after the pixel point to be filled is repaired.
Step 18: and obtaining the restored reduced image according to the target value of the pixel point to be filled and the values of other points except the pixel point to be filled in the filtered image to be restored.
The above steps are all processing on the pixel values of the pixels to be filled, and the values of other pixels (that is, effective pixels in the filtered image to be repaired) in the filtered image to be repaired except the pixels to be filled are not changed, so that the target value of the pixels to be filled is filled into the filtered image to be repaired to obtain a restored reduced image, and it can be understood that the size of the restored reduced image is the same as the size of the filtered image to be repaired.
Step 19: and performing up-sampling on the restored reduced image to obtain a restored image.
And the size of the repaired image is the same as that of the image to be repaired.
Because the size of the restored reduced image is smaller than that of the image to be restored, the restored reduced image can be restored to the image with the same size as that of the image to be restored in an up-sampling mode, namely the restored image.
Further, the above up-sampling manner may include, but is not limited to: bilinear interpolation, bicubic interpolation, or nearest neighbor interpolation.
In this embodiment, adaptive reduction parameters and adaptive filtering parameters are obtained, so that corresponding reduction parameters and filtering parameters can be automatically matched according to the size of an area formed by invalid pixels in an image to be restored, the image to be restored is reduced and filtered based on the adaptive reduction parameters and the filtering parameters, a filtered image to be restored is obtained, pixels to be filled in the filtered image to be restored are obtained, a better initial value of the pixels to be filled is obtained, the processing accuracy is higher, and the calculation amount of subsequent image processing can be reduced by reducing and filtering the image under the condition of ensuring the accuracy, so that the calculation amount in the subsequent processing process is reduced, the efficiency of image restoration processing is improved, and based on a constraint condition that the sum of the gradients of all the pixels to be filled is minimum and the initial value of the pixels to be filled, the method comprises the steps of carrying out first iteration processing on values of pixels to be filled to obtain rough processing values of the pixels to be filled, thus quickly obtaining the rough processing values of the pixels to be filled in an iteration mode, improving the efficiency of image restoration processing, taking the rough processing values of the pixels to be filled as initial values of the pixels to be filled again, carrying out second iteration processing on the values of the pixels to be filled according to an energy equation and the initial values of the pixels to be filled to obtain fine processing values of the pixels to be filled, taking the fine processing values of the pixels to be filled as target values of the pixels to be filled, introducing the energy equation in physics into the image processing field, carrying out iteration solution, improving the efficiency of image processing, enabling the continuity of restored image edges to be good, enabling the restored image to be natural, and being more suitable for application scenes of real-time processing.
On the basis of the embodiment shown in fig. 1, further, step 11 may include the following steps 111 to 113.
Step 111: and acquiring preset extraction parameters.
The extraction parameters are used for indicating the region formed by the invalid pixel points of the image to be repaired to extract the parameters.
Optionally, the extraction parameter may be set accordingly through a received input instruction input by the user, where the input instruction may be received by the electronic device through the input device.
Optionally, the extraction parameter may include an extraction value and an extraction direction, and the extraction direction may includeXDirection (b),YDirection (b),XYAnd (4) direction.
Step 112: the adaptive filtering parameters are obtained according to the following formula (1):
Wherein,Mis the parameter of the filtering that is,Sis to extract parameters, symbols
Which represents a rounding-up operation on the upper part,max() Indicating a maximum operation.
Step 113: the adaptive reduction parameter is obtained according to the following formula (2):
Wherein,Ris to scale down the parameters of the process,Mis the parameter of the filtering that is,Sis to extract parameters, symbols
Representing an operation of rounding up, sign
Indicating a rounding down operation.
Wherein a 1 is usedRRepresenting a reduced scale of the image to be restored byRThe method and the device represent the magnification ratio of the image to be repaired, and in the embodiment of the application, the image to be repaired is subjected to reduction processing.
According to the embodiment, the adaptive reduction parameters and the adaptive filtering parameters are obtained in a self-adaptive mode according to the preset extraction parameters, so that the corresponding reduction parameters and the corresponding filtering parameters can be automatically matched according to the size of the area formed by the invalid pixel points in the image to be restored, when the image is processed based on the adaptive reduction parameters and the filtering parameters, the processing accuracy is higher, the calculated amount of subsequent image processing can be reduced by reducing and filtering the image under the condition that the accuracy is ensured, the image processing efficiency is improved, and the method is more suitable for the application scene of real-time processing.
On the basis of the above embodiment, further, the reduction and filtering processing performed in step 12 may be implemented in various manners, and the present application is not limited to which manner is specifically used, and one of the manners of reduction and filtering is specifically described below.
step 121: and transforming coordinates of the pixel points of the image to be restored according to the self-adaptive reduction parameters, and performing nearest neighbor, bilinear or bicubic interpolation processing on the pixel points after the coordinates are transformed to obtain the reduced image to be restored.
The coordinate transformation can be performed on the pixel point of the image to be restored according to the following formula (3):
Wherein (A), (B), (C) and Cx i ,y i ) Are the coordinates of the image to be restored,
is a coordinate (x i ,y i ) The coordinates of the object after the transformation are obtained,sis a shrink parameter.
Step 122: and carrying out filtering processing on the reduced image to be repaired according to the self-adaptive filtering parameters to obtain the filtered image to be repaired.
The filtered image to be repaired can be obtained according to the following formula (4):
Wherein,O(i,j) Is the filtered image to be restored,Kis a filter and may also be referred to as a Kernel.
On the basis of the above embodiment, further, step 13 may include the following steps 131 and 132:
step 131: and obtaining pixel points to be filled in the filtered image to be repaired.
In some scenes, pixel points with large difference with surrounding texture information are generally required to be repaired when a two-dimensional image is repaired, pixel points exceeding a hardware view range are required to be repaired when a three-dimensional image is repaired, and different image repairing modes are required to be adopted for image repairing of different dimensions. The image to be restored in this embodiment may be a two-dimensional image or a three-dimensional image, and correspondingly, the filtered image to be restored may also be a two-dimensional image or a three-dimensional image. Therefore, the image restoration processing can be carried out on the images to be restored with different dimensions by using a uniform method.
Further, under the condition that the image to be restored is a three-dimensional image, the filtered image to be restored is a three-dimensional image, and pixel points of which the values of the pixel points in the filtered image to be restored exceed the view range of the hardware can be obtained and used as pixel points to be filled in the filtered image to be restored.
Optionally, binarization processing may be performed on the filtered image to be restored to obtain a binarized image, where a pixel in the filtered image to be restored, whose value exceeds the hardware view range, is obtained as a first preset value, the effective pixel is set as a second preset value, and a pixel corresponding to the second preset value, that is, the effective pixel, is obtained. The region formed by the effective pixel is subjected to expansion processing (the size of the structural element subjected to expansion processing may be 3 × 3, 5 × 5, or other sizes), so as to obtain an expanded region. And performing difference between the expanded region and the binary image to obtain a region to be filled, wherein pixel points in the region to be filled are the pixel points to be filled.
Further, under the condition that the image to be restored is a two-dimensional image, the filtered image to be restored is a two-dimensional image, and the pixel points to be filled can be obtained by processing the filtered image as follows:
and amplifying the filtered image to be repaired to obtain an amplified image with the same size as the image to be repaired.
And acquiring the absolute value of the difference between each pixel point of the image to be restored and the corresponding pixel point in the amplified image.
And acquiring the pixel points of which the absolute value is greater than or equal to the preset threshold as invalid pixel points in the amplified image.
The method comprises the steps that reduction filtering processing is carried out on an image to be processed, invalid pixel points in the image to be processed are processed in the process of obtaining an amplified image after amplification processing, the values of the valid pixel points in the amplified image and the image to be processed are not changed greatly, and therefore the pixel points are indicated as invalid pixel points when the absolute value of the difference is larger than or equal to a preset threshold value.
The preset threshold is a preset value larger than 0.
Wherein the absolute value of the difference being greater than or equal to the preset threshold may include the following: the absolute value of the difference is greater than a preset threshold, the absolute value of the difference is equal to the preset threshold, or the absolute value of the difference is greater than or equal to the preset threshold.
And reducing the area of the invalid pixel point in the amplified image according to the self-adaptive reduction parameter to obtain the pixel point to be filled.
In some scenes, for a two-dimensional image, the pixel to be filled obtained through the processing includes some interference pixels, and the method provided by this embodiment can perform corrosion processing and expansion processing on the region formed by the pixel to be filled in sequence after the pixel to be filled is obtained, so as to obtain a final pixel to be filled.
The images with different dimensions are respectively processed to obtain the pixel points to be filled in the filtered image to be repaired, so that the image repairing method provided by the application can be adopted for image repairing aiming at the images with different dimensions, and the application range of the image repairing method provided by the application is wide.
Step 132: and acquiring an initial value of the pixel point to be filled.
In a possible implementation manner, in step 132, the initial value of the pixel point to be filled may be obtained by: and acquiring the current value of the pixel to be filled in the filtered image to be repaired as the initial value of the pixel to be filled in the filtered image to be repaired.
In another possible implementation manner, in step 132, the initial value of the pixel point to be filled may be obtained as follows: and processing the pixel points to be filled in the filtered image to be repaired based on the image filling algorithm of mean filtering to obtain a value serving as an initial value of the pixel points to be filled in the filtered image to be repaired.
Further, there are several specific implementations of mean-filling, and one way of mean-filling is described below.
The region formed by the pixel points to be filled is expanded (for example, expansion of one pixel can be performed outward) to obtain the neighborhood region. For each invalid pixel point (namely the pixel point to be filled) in the neighborhood region, the mean value of the valid pixel points in the 3 × 3 or 5 × 5 neighborhood region is obtained in an iterative mode and is used as the initial value of the pixel point to be filled.
For example, in the first iteration, for each invalid pixel point, obtaining the average value of all valid pixel points in the neighborhood as the initial value of the invalid pixel point, wherein if the invalid pixel point obtains the initial value, the invalid pixel point is used as the valid pixel point of the next iteration; and if the neighborhood of the invalid pixel point does not have the valid pixel point, the invalid pixel point enters the next iteration processing, and the iteration is stopped until all the invalid pixel points acquire the initial value.
According to the embodiment, after the image to be restored is processed through the adaptive reduction parameter and the adaptive filtering parameter, the initial value of the invalid pixel point can be obtained well, so that the calculated amount in the subsequent processing process is reduced, and the efficiency of image restoration processing is improved.
In some embodiments, in the coarse processing, since the partial differential equation is a constraint condition in a continuous case, discretization is required for the image, so that the constraint equation expressed by the constraint condition in the coarse processing can be discretized as follows:
the optimization condition can be expressed by the following formula (5):
u xx (x，y)+u yy (x，y) =0 equation (5)
Wherein,u xx (p) Indicates the pixel to be filled is inxThe second partial derivative of the direction is,u yy (p) Indicating a pixel to be filled inySecond partial derivative of direction.
Discretizing equation (5) can yield the following equation (6):
u xx (x i ，y i )=2u(x i ，y i )-u(x i -1，y i )-u(x i +1，y i )
u yy (x i ，y i )=2u(x i ，y i )-u(x i ，y i -1)-u(x i ，y i +1) formula (6)
Wherein,u(x i ，y i ) Is a coordinate ofx i ，y i ) The value of the pixel point of (c).
On the basis of the above embodiment, further, the constraint condition that the sum of the divergences of the gradient values of all the pixels to be filled in step 14 is minimum can be represented by the following formula (7):
Wherein,pis the pixel point to be filled in,Ris a set formed by pixel points to be filled, min represents the minimum value operation,u xx (p) Indicating a pixel to be filled inxThe second partial derivative of the direction is,u yy (p) Indicating a pixel to be filled inySecond partial derivative of direction.
Further, the constraint conditions of the above equation (7) are derived below, so that the equation in the specific processing is obtained to perform the iterative processing.
Expanding equation (7) yields the following equation (8):
Reducing equation (8) can yield the following equation (9):
Wherein a coordinate matrix can be set asn×nThe matrix of (a) is a matrix of (b),nfor the number of the pixel points to be filled, 4 represents the index corresponding to the coordinates of the pixel point to be filled, and-1 is the index corresponding to the neighborhood pixels of the pixel point to be filled, and in fact, because it is difficult to encode the pixel index by using the four neighborhood points of the pixel point, the above matrix is not always-1 arranged in parallel at about 4.u(x,y) Is a coordinate ofx,y) The value of the pixel point is restored to the pixel point to be filled, so that the pixel point is locatedu(x,y) Is the value of the pixel to be filled.
The formula (9) can be expressed asAX=bThe method comprises the following steps of setting an iteration equation based on a constraint condition of a formula (9), performing first iteration processing to obtain a rough processing value of a pixel point to be filled meeting the constraint condition as much as possible, performing the first iteration processing according to a following formula (10), and determining the rough processing value of the pixel point to be filled as a value obtained by the last iteration of the pixel point to be filled in the first iteration processing:
Wherein,iis the number of iterations that are to be performed,i≥0，b i+1 is the firsti+An iteration vector of 1 iteration is calculated,X i is that the pixel point to be filled is iniThe value of the sub-iteration is,λis the first iteration step;
the iteration vector can be obtained according to the following equation (11)b i+1：
Wherein initialization is carried outb=b 0 =0That is to saybInitial value of (2)b 0 Is a zero vector.
The pixel point to be filled in the second place can be obtained according to the following formula (12)iValue of the sub-iterationX i ：
Wherein,nis the number of pixels to be filled,Ais thatn×nIs a matrix of coordinates ofx,y) To be filled with pixel pointsiAnd calculating the obtained value by the secondary iteration.
Obtaining a first iteration step size according to the following formula (13)λ：
Wherein the matrix is expressed according to the following formula (14)A：
Further, while the first iteration process is performed, the iteration is stopped until after a first iteration stop condition is satisfied. The first iteration stop condition may include: the first iteration step length is smaller than or equal to a preset threshold value, or the iteration times reach first preset times.
In other embodiments, in order to make the restored image have the edge-preserving property, the energy equation of physics may be extended into the image processing field, and how the energy equation is extended into the image processing field is described in detail in the following with specific embodiments.
On the basis of the above embodiment, further, the energy equation used in step 16 can be expressed by the following formula (16):
Wherein,u t the diffusion time of the pixel point to be filled istThe value of the time-of-day,uis the value of the pixel point to be filled,tin order to be able to measure the time of diffusion,
is a gradient operator, which is a linear operator,divis a divergence operator, and is a function of the divergence,gis a weight function of the gradient, andgis a monotonically decreasing function of the gradient and,cis a constant.
It can be understood that if the function isgFor constant scalar functions, the constraint degenerates to Gaussian filtering, ifgFor scalar functions, the constraint is then degenerated into a filter weighting of equal weight for each direction, whengIn the case of a vector function, the filter weights are weighted unequally in each direction.gThe function may be a number of monotonically decreasing functions to characterize the smaller the smoothing effect at more graded locations in the image.
Exemplary parabolic equations expressed by equation (17), P-M (Perona-Malik) equation expressed by equation (18), and Weikt (Weickert) equation expressed by equation (19) are described belowgThe specific form of (1):
Wherein,Cis a constant parameter, and is,C=3.31488。
further, the following solving process can be adopted for the above equation (16):
let an input initial image beu(x,y0), represents timetFunction when =0, the weight function parameter iscThe iteration step isθThe number of iterations isN。
Calculating gradient values of the pixels to be filled, wherein the gradient values can be calculated in a differential mode, and the following exemplary shows a mode of adopting forward difference, backward difference and intermediate difference.
Wherein, the forward difference can be expressed by the following formula (20):
Wherein, the backward difference can be expressed by the following formula (21):
Wherein the intermediate difference can be expressed by the following equation (22):
The gradient value of the pixel point to be filled can be obtained by the following formula (23):
So that a weight function corresponding to the gradient can be obtainedgCan be expressed by the following equation (24):
Therefore, through the derivation process, an iterative formula can be obtained for iterative processing. Since it is considered that the diffusion of the temperature in the original energy process has no fixed direction, i.e. it can be performed in any direction around, in the field of image processing, any direction around can be simplified, and the diffusion process in four directions, i.e. up, down, left and right, of the pixel point is considered, so as to obtain an iterative formula, which will be described in detail with specific embodiments below.
On the basis of the above embodiment, further, the step 16 may include the following steps:
performing second iteration processing on the value of the pixel point to be filled according to the following formula (25) until a second iteration stop condition is met, and determining the fine processing value of the pixel point to be filled as the value obtained by the last iteration of the pixel point to be filled in the second iteration processing:
Wherein,tis the time of the diffusion of the light,t≥0，u(x,y,t) Is time of diffusion oftThe time coordinate is (x,y) The value of the pixel point of (a) is,θis a second iteration step size;
is a ladder with upward pixel points to be filledThe weight function of the degree of the received signal,
is a weight function of the downward gradient of the pixel points to be filled,
is a weight function of the gradient to the left of the pixel to be filled,
is a weight function of the gradient to the right of the pixel to be filled,gis a monotonically decreasing function;
is a coordinate ofx,y) The upward gradient of the pixel points to be filled,
is a coordinate ofx,y) The downward gradient of the pixel points to be filled,
is a coordinate ofx,y) The gradient of the pixel points to be filled to the right,
is a coordinate ofx,y) The gradient of the pixel points to be filled to the left,u(x,y) Is a coordinate ofx,y) The value of the pixel point of (a) is,cis a constant.
Further, the gradient value in the formula (25) can be obtained by forward difference, backward difference or middle difference, and the following exemplary method is described in which the gradient value is obtained by forward difference.
The upward gradient of the pixel points to be filled, the downward gradient of the pixel points to be filled, the leftward gradient of the pixel points to be filled and the rightward gradient of the pixel points to be filled are obtained according to the following formula (26):
Wherein,
is a coordinate ofx,y) The upward gradient of the pixel points to be filled,
is a coordinate ofx,y) The downward gradient of the pixel points to be filled,
is a coordinate ofx,y) The gradient of the pixel points to be filled to the right,
is a coordinate ofx,y) The gradient of the pixel points to be filled to the left,u(x,y) Is a coordinate ofx,y) The value of the pixel point of (1).
Further, when the above-described second iterative processing is performed, the iteration is stopped until a second iteration stop condition is satisfied. The second iteration stop condition may include: the iteration times reach a second preset time.
In the embodiment, the energy equation in physics is introduced into the field of image processing, and the diffusion direction is simplified into four directions, namely an upper direction, a lower direction, a left direction and a right direction, so that iterative solution is performed, the calculation process is simplified, the processing efficiency is improved, the continuity of the edge of the repaired image is better, and the repaired image is more natural.
Example two
The image restoration method provided in this embodiment can process the image to be restored through the rough processing procedure in the image restoration provided in the first embodiment. The following will explain details of the present invention by specific examples.
Referring to fig. 2, fig. 2 is a schematic flow chart of another image restoration method provided in an embodiment of the present application, and the embodiment shown in fig. 2 is based on the first embodiment, and further, the method provided in this embodiment includes the following steps 11, 12, 13, 14, 21, 18, and 19:
since the implementation principle of step 11, step 12, step 13, step 14, step 18 and step 19 and the technical solution formed by the subordinate concepts thereof are similar to those of the above embodiments, they are not described herein again.
Step 21: and taking the rough processing value of the pixel point to be filled as the target value of the pixel point to be filled.
In the embodiment, the adaptive reduction parameter and the adaptive filtering parameter are obtained in an adaptive manner, so that the corresponding reduction parameter and the corresponding filtering parameter can be automatically matched according to the size of the area formed by the invalid pixel points in the image to be restored, the image to be restored is reduced and filtered based on the adaptive reduction parameter and the corresponding filtering parameter, the filtered image to be restored is obtained, the pixel points to be filled in the filtered image to be restored are obtained, the better initial value of the pixel points to be filled is obtained, the processing accuracy is higher, and the calculation amount of subsequent image processing can be reduced by reducing and filtering the image under the condition of ensuring the accuracy, so that the calculation amount in the subsequent processing process is reduced, the efficiency of image restoration processing is improved, and based on the constraint condition that the sum of the gradient values of all the pixel points to be filled is minimum and the initial value of the pixel points to be filled, and carrying out first iteration processing on the value of the pixel point to be filled to obtain a rough processing value of the pixel point to be filled, so that the rough processing value of the pixel point to be filled is quickly obtained in an iteration mode, a repaired image is obtained, the image repairing processing efficiency is improved, and the method is more suitable for an application scene of real-time processing.
On the basis of the embodiment shown in fig. 2, further, step 21 may be implemented by:
and according to the coarse processing value of the pixel point to be filled, filtering the region formed by the pixel point to be filled to obtain a filtered coarse processing value.
And taking the filtered coarse processing value as a target value of the pixel point to be filled.
The filtering process may be gaussian filtering, bilateral filtering or other filtering methods.
EXAMPLE III
The image restoration method provided in this embodiment can process the image to be restored through the refinement processing procedure in the image restoration provided in the first embodiment. The following will explain details of the present invention by specific examples.
Referring to fig. 3, fig. 3 is a schematic flow chart of another image restoration method provided in an embodiment of the present application, and the embodiment shown in fig. 3 is based on the first embodiment, and further, the method provided in this embodiment includes the following steps 11, 12, 13, 16, 17, 18, and 19:
due to the technical solution of step 11, step 12, step 13, step 16, step 17, step 18, step 19 and their subordinate concepts, the implementation principle is similar to that of the above embodiments, and thus is not described herein again.
In the embodiment, the adaptive reduction parameters and the adaptive filtering parameters are obtained in an adaptive manner, so that the corresponding reduction parameters and the corresponding filtering parameters can be automatically matched according to the size of the area formed by the invalid pixel points in the image to be restored, the image to be restored is reduced and filtered based on the adaptive reduction parameters and the corresponding filtering parameters, the filtered image to be restored is obtained, the pixel points to be filled in the filtered image to be restored are obtained, the initial values of the better pixel points to be filled are obtained, the processing accuracy is higher, the calculated amount of subsequent image processing can be reduced by reducing and filtering the image under the condition of ensuring the accuracy, the calculated amount in the subsequent processing process is reduced, the efficiency of image restoration processing is improved, the second iteration processing is carried out on the values of the pixel points to be filled according to an energy equation and the initial values of the pixel points to be filled, the method comprises the steps of obtaining fine processing values of pixel points to be filled, further obtaining a restored image, introducing an energy equation in physics into the image processing field, and carrying out iterative solution, so that the image processing efficiency is improved, the edge continuity of the restored image is better, the restored image is more natural, and the method is more suitable for application scenes of real-time processing.
On the basis of the embodiment shown in fig. 3, further, before step 16, a processed value of the pixel to be filled may be obtained by searching for similar other image parts in the filtered image to be repaired, blocking the other image parts, and integrally transplanting the other image parts to the region formed by the pixel to be filled for processing, and the processed value of the pixel to be filled is used as an initial value of the pixel to be filled.
The following is an exemplary description of a process of obtaining a repaired image by applying the image repairing method provided by the present embodiment as shown in fig. 4 to 8.
Fig. 4 is an image to be repaired according to an embodiment of the present application, fig. 5 is a filtered image to be repaired obtained from the image to be repaired shown in fig. 4 according to an embodiment of the present application, fig. 6 is a coarsely processed image obtained from the filtered image to be repaired shown in fig. 5 according to an embodiment of the present application, fig. 7 is a repaired image obtained from the coarsely processed image shown in fig. 6 according to an embodiment of the present application, and fig. 8 is an image obtained by performing a refinement process on the coarsely processed image shown in fig. 6 according to an embodiment of the present application.
Example four
The present embodiment provides a computer-readable storage medium having stored thereon a program executable by a processor to implement the image inpainting method as in the first-real time embodiment three above.
EXAMPLE five
The present embodiment provides an electronic device, including: one or more processors; a memory; and one or more computer programs; wherein the one or more computer programs are stored in the memory; the one or more processors, when executing the one or more computer programs, cause the electronic device to implement the image inpainting method as in the first-real time example three above.
Those skilled in the art will appreciate that all or part of the functions of the various methods in the above embodiments may be implemented by hardware, or may be implemented by computer programs. When all or part of the functions of the above embodiments are implemented by a computer program, the program may be stored in a computer-readable storage medium, and the storage medium may include: a read only memory, a random access memory, a magnetic disk, an optical disk, a hard disk, etc., and the program is executed by a computer to realize the above functions. For example, the program may be stored in a memory of the device, and when the program in the memory is executed by the processor, all or part of the functions described above may be implemented. In addition, when all or part of the functions in the above embodiments are implemented by a computer program, the program may be stored in a storage medium such as a server, another computer, a magnetic disk, an optical disk, a flash disk, or a removable hard disk, and may be downloaded or copied to a memory of a local device, or may be version-updated in a system of the local device, and when the program in the memory is executed by a processor, all or part of the functions in the above embodiments may be implemented.
The present application has been described with reference to specific examples, which are provided only to aid understanding of the present application and are not intended to limit the present application. For a person skilled in the art to which the application pertains, several simple deductions, modifications or substitutions may be made according to the idea of the application.
Claims (10)
1. An image restoration method, comprising:
obtaining a self-adaptive reduction parameter and a self-adaptive filtering parameter, wherein the self-adaptive reduction parameter is obtained according to the size of an invalid region formed by invalid pixel points in an image to be restored; the self-adaptive filtering parameter is obtained according to the size of an invalid region formed by invalid pixel points in the image to be restored;
reducing and filtering the image to be repaired according to the self-adaptive reduction parameter and the self-adaptive filtering parameter to obtain a filtered image to be repaired;
acquiring pixel points to be filled in the filtered image to be repaired and initial values of the pixel points to be filled;
performing first iteration processing on the value of the pixel point to be filled based on a constraint condition that the sum of the divergence of the gradient values of all the pixel points to be filled is minimum and the initial value of the pixel point to be filled to obtain a rough processing value of the pixel point to be filled;
taking the coarse processing value of the pixel point to be filled as the initial value of the pixel point to be filled again, carrying out second iterative processing on the value of the pixel point to be filled according to an energy equation and the initial value of the pixel point to be filled to obtain a fine processing value of the pixel point to be filled, and taking the fine processing value of the pixel point to be filled as the target value of the pixel point to be filled;
obtaining a restored reduced image according to the target value of the pixel to be filled and the values of other pixels except the pixel to be filled in the filtered image to be restored;
up-sampling the restored reduced image to obtain a restored image, wherein the size of the restored image is the same as that of the image to be restored;
the obtaining of the adaptive reduction parameter and the adaptive filtering parameter includes:
acquiring preset extraction parameters;
the adaptive filtering parameters are obtained according to the following formula:
wherein,Mis the parameter of the filtering that is,Sis to extract parameters, symbols
Which represents a rounding-up operation on the upper part,max() Representing the maximum value operation;
the adaptive reduction parameter is obtained according to the following formula:
wherein,Ris to shrinkThe parameters of the number of the first and second antennas,Mis the parameter of the filtering that is,Sis to extract parameters, symbols
Indicating a rounding-up operation, sign
Indicating a rounding down operation.
2. An image restoration method, comprising:
obtaining a self-adaptive reduction parameter and a self-adaptive filtering parameter, wherein the self-adaptive reduction parameter is obtained according to the size of an invalid region formed by invalid pixel points in an image to be restored; the self-adaptive filtering parameters are obtained according to the size of an invalid region formed by invalid pixel points in the image to be restored;
reducing and filtering the image to be repaired according to the self-adaptive reduction parameter and the self-adaptive filtering parameter to obtain a filtered image to be repaired;
acquiring pixel points to be filled in the filtered image to be repaired and initial values of the pixel points to be filled;
performing coarse processing or fine processing on the pixel points to be filled in the filtered image to be repaired to obtain target values of the pixel points to be filled;
the rough treatment comprises the following steps: based on a constraint condition of minimizing the sum of the divergence of the gradient values of all the pixels to be filled, performing first iteration processing according to the initial value of the pixels to be filled to obtain a rough processing value of the pixels to be filled, and taking the rough processing value of the pixels to be filled as a target value of the pixels to be filled;
the refinement processing comprises: according to an energy equation and the initial value of the pixel point to be filled, carrying out second iterative processing on the value of the pixel point to be filled to obtain a fine processing value of the pixel point to be filled, and taking the fine processing value of the pixel point to be filled as a target value of the pixel point to be filled;
obtaining a restored reduced image according to the target value of the pixel to be filled and the values of other pixels except the pixel to be filled in the filtered image to be restored;
up-sampling the restored reduced image to obtain a restored image, wherein the size of the restored image is the same as that of the image to be restored;
the obtaining of the adaptive reduction parameter and the adaptive filtering parameter includes:
acquiring preset extraction parameters;
the adaptive filtering parameters are obtained according to the following formula:
wherein,Mis the parameter of the filtering that is,Sis to extract parameters, symbols
Which represents a rounding-up operation on the upper part,max() Representing the maximum value operation;
the adaptive reduction parameter is obtained according to the following formula:
wherein,Ris to scale down the parameters of the process,Mis the parameter of the filtering that is,Sis to extract parameters, symbols
Indicating a rounding-up operation, sign
Indicating a rounding down operation.
3. The method of claim 1 or 2, wherein the constraint that minimizes the sum of the divergence of the gradient values for all pixels to be filled comprises:
wherein,pis the pixel point to be filled in,Ris a set formed by pixel points to be filled, min represents the minimum value operation,u xx (p) Indicates the pixel to be filled is inxThe second partial derivative of the direction is,u yy (p) Indicating a pixel to be filled inySecond partial derivative of direction.
4. The method of claim 1 or 2, wherein the energy equation comprises:
wherein,u t the diffusion time of the pixel point to be filled istThe value of (c) time of day,uis the value of the pixel point to be filled,tthe time for diffusion, is the gradient operator,divis a divergence operator, and is a function of the divergence,gis a weight function of the gradient, andgis a monotonically decreasing function of the gradient and,cis a constant.
5. The method according to claim 1 or 2, wherein the obtaining of the pixel to be filled in the filtered image to be repaired and the initial value of the pixel to be filled comprises:
acquiring pixel points to be filled in the filtered image to be repaired;
obtaining a current value of the pixel to be filled in the filtered image to be repaired, or taking a value obtained by processing the pixel to be filled in the filtered image to be repaired based on an image filling algorithm of mean filtering as an initial value of the pixel to be filled in the filtered image to be repaired.
6. The method as claimed in claim 5, wherein said obtaining pixels to be filled in the filtered image to be repaired includes:
under the condition that the image to be repaired is a three-dimensional image, acquiring pixel points of which the values of the pixel points in the filtered image to be repaired exceed the view range of hardware as pixel points to be filled in the filtered image to be repaired;
under the condition that the image to be repaired is a two-dimensional image, amplifying the filtered image to be repaired to obtain an amplified image with the same size as the image to be repaired; acquiring the absolute value of the difference between each pixel point of the image to be restored and the corresponding pixel point in the amplified image; acquiring pixel points of which the absolute value of the difference is greater than or equal to a preset threshold as invalid pixel points in the amplified image; and carrying out reduction processing on the area of the invalid pixel point in the amplified image according to the self-adaptive reduction parameter to obtain the pixel point to be filled.
7. The method as claimed in claim 1 or 2, wherein the performing a first iteration process according to the initial value of the pixel to be filled based on a constraint condition that minimizes the sum of the divergences of the gradient values of all the pixel to be filled to obtain a rough processed value of the pixel to be filled comprises:
performing the first iteration processing based on the following formula, and determining that the rough processing value of the pixel point to be filled is the value obtained by the last iteration of the pixel point to be filled in the first iteration processing:
wherein,iis the number of iterations that are to be performed,i≥0，b i+1 is the firsti+An iteration vector of 1 iteration is calculated,X i is that the pixel point to be filled is iniThe value of the sub-iteration is,λis the first iteration step size;
obtaining an iteration vector according to the following formulab i+1：
Obtaining the pixel point to be filled in the second place according to the following formulaiValue of the sub-iterationX i ：
Wherein,nis the number of pixels to be filled,Ais thatn×nThe matrix of (a) is,u i (x,y) Is a coordinate ofx,y) To be filled with pixel pointsiA value obtained by the secondary iterative computation;
obtaining a first iteration step size according to the following formulaλ：
Wherein the matrix is expressed according to the following formulaA：
8. The method according to claim 1 or 2, wherein the second iteration processing is performed on the value of the pixel to be filled according to the energy equation and the gradient value of the pixel to be filled until a second iteration stop condition is satisfied to obtain a fine processing value of the pixel to be filled, and the method comprises:
performing the second iteration processing on the value of the pixel point to be filled according to the following formula until the second iteration stop condition is met, and determining the fine processing value of the pixel point to be filled as the value obtained by the last iteration of the pixel point to be filled in the second iteration processing:
wherein,tis the time of the diffusion of the light,t≥0，u(x,y,t) Is time of diffusion oftThe time coordinate is (x,y) The value of the pixel point of (a) is,θis a second iteration step size;
is a weight function of the upward gradient of the pixel points to be filled,
is a weight function of the downward gradient of the pixel points to be filled,
is a weight function of the gradient to the left of the pixel to be filled,
is a weight function of the gradient to the right of the pixel to be filled,gis a monotonically decreasing function;
is a coordinate ofx,y) The upward gradient of the pixel points to be filled,
is a coordinate ofx,y) The downward gradient of the pixel points to be filled,
is a coordinate ofx,y) The gradient of the pixel points to be filled to the right,
is a coordinate ofx,y) The gradient of the pixel points to be filled to the left,u(x,y) Is a coordinate ofx,y) The value of the pixel point of (a) is,cis a constant.
9. The method of claim 8, wherein the upward gradient of the pixel points to be filled, the downward gradient of the pixel points to be filled, the leftward gradient of the pixel points to be filled, and the rightward gradient of the pixel points to be filled are obtained according to the following formulas:
wherein,
is a coordinate ofx,y) The upward gradient of the pixel points to be filled,
is a coordinate ofx,y) The downward gradient of the pixel points to be filled,
is a coordinate ofx,y) The gradient of the pixel points to be filled to the right,
is a coordinate ofx,y) The gradient of the pixel points to be filled to the left,u(x,y) Is a coordinate ofx,y) The value of the pixel point of (1).
10. A computer-readable storage medium, characterized in that the medium has stored thereon a program which is executable by a processor to implement the method according to any one of claims 1-9.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210327415.4A CN114418902B (en) | 2022-03-31 | 2022-03-31 | Image restoration method and computer-readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210327415.4A CN114418902B (en) | 2022-03-31 | 2022-03-31 | Image restoration method and computer-readable storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114418902A CN114418902A (en) | 2022-04-29 |
| CN114418902B true CN114418902B (en) | 2022-07-08 |
Family
ID=81263431
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210327415.4A Active CN114418902B (en) | 2022-03-31 | 2022-03-31 | Image restoration method and computer-readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114418902B (en) |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101674397A (en) * | 2009-09-27 | 2010-03-17 | 上海大学 | Repairing method of scratch in video sequence |
| CN102142132A (en) * | 2011-03-31 | 2011-08-03 | 北京交通大学 | Module-based image inpainting method |
| CN103778604A (en) * | 2014-01-10 | 2014-05-07 | 浙江大学 | Image restoration method based on texture direction |
| CN108648221A (en) * | 2018-05-10 | 2018-10-12 | 重庆大学 | A kind of depth map cavity restorative procedure based on mixed filtering |
| CN109242791A (en) * | 2018-08-22 | 2019-01-18 | 东北农业大学 | A kind of batch restorative procedure for damaged plant leaf blade |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20080310695A1 (en) * | 2003-09-04 | 2008-12-18 | Garnier Stephen J | Locally Adaptive Nonlinear Noise Reduction |
| CN110097590A (en) * | 2019-04-24 | 2019-08-06 | 成都理工大学 | Color depth image repair method based on depth adaptive filtering |
-
2022
- 2022-03-31 CN CN202210327415.4A patent/CN114418902B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101674397A (en) * | 2009-09-27 | 2010-03-17 | 上海大学 | Repairing method of scratch in video sequence |
| CN102142132A (en) * | 2011-03-31 | 2011-08-03 | 北京交通大学 | Module-based image inpainting method |
| CN103778604A (en) * | 2014-01-10 | 2014-05-07 | 浙江大学 | Image restoration method based on texture direction |
| CN108648221A (en) * | 2018-05-10 | 2018-10-12 | 重庆大学 | A kind of depth map cavity restorative procedure based on mixed filtering |
| CN109242791A (en) * | 2018-08-22 | 2019-01-18 | 东北农业大学 | A kind of batch restorative procedure for damaged plant leaf blade |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114418902A (en) | 2022-04-29 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Zhuo et al. | Defocus map estimation from a single image | |
| CN106027851B (en) | Method and system for processing images | |
| Mahmood et al. | Nonlinear approach for enhancement of image focus volume in shape from focus | |
| CN107025660B (en) | Method and device for determining image parallax of binocular dynamic vision sensor | |
| Lee et al. | Robust exemplar-based inpainting algorithm using region segmentation | |
| CN110866924A (en) | Line structured light center line extraction method and storage medium | |
| Fang et al. | Single image dehazing and denoising: a fast variational approach | |
| Perra et al. | An analysis of 3D point cloud reconstruction from light field images | |
| Duan et al. | Fast algorithm for color texture image inpainting using the non-local CTV model | |
| Agrafiotis et al. | Underwater photogrammetry in very shallow waters: main challenges and caustics effect removal | |
| CN109509163B (en) | FGF-based multi-focus image fusion method and system | |
| CN109035170A (en) | Adaptive wide-angle image correction method and device based on single grid chart subsection compression | |
| JP5812705B2 (en) | Crack detection method | |
| Agrafiotis et al. | The effect of underwater imagery radiometry on 3D reconstruction and orthoimagery | |
| CN110415193A (en) | The restored method of coal mine low-light (level) blurred picture | |
| CN113971669A (en) | Three-dimensional detection system applied to pipeline damage identification | |
| CN111260675A (en) | High-precision extraction method and system for image real boundary | |
| Seo | Image denoising and refinement based on an iteratively reweighted least squares filter | |
| Saleem et al. | A non-reference evaluation of underwater image enhancement methods using a new underwater image dataset | |
| Ghita et al. | Texture Enhanced Histogram Equalization Using TV-${\rm L}^{1} $ Image Decomposition | |
| Lao et al. | Corresnerf: Image correspondence priors for neural radiance fields | |
| Keaomanee et al. | Implementation of four kriging models for depth inpainting | |
| Chang et al. | A self-adaptive single underwater image restoration algorithm for improving graphic quality | |
| CN114418902B (en) | Image restoration method and computer-readable storage medium | |
| Tung et al. | Multiple depth layers and all-in-focus image generations by blurring and deblurring operations |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |