CN113989135A - Image restoration method and device, electronic equipment and storage medium - Google Patents
Image restoration method and device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN113989135A CN113989135A CN202111180704.8A CN202111180704A CN113989135A CN 113989135 A CN113989135 A CN 113989135A CN 202111180704 A CN202111180704 A CN 202111180704A CN 113989135 A CN113989135 A CN 113989135A
- Authority
- CN
- China
- Prior art keywords
- repaired
- determining
- area
- picture
- preset
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 47
- 230000008439 repair process Effects 0.000 claims description 39
- 238000004891 communication Methods 0.000 claims description 3
- 238000004364 calculation method Methods 0.000 abstract description 4
- 230000002950 deficient Effects 0.000 description 14
- 238000012545 processing Methods 0.000 description 10
- 239000003086 colorant Substances 0.000 description 9
- 238000010586 diagram Methods 0.000 description 9
- 238000005516 engineering process Methods 0.000 description 7
- 230000003287 optical effect Effects 0.000 description 6
- 230000007547 defect Effects 0.000 description 4
- 230000008569 process Effects 0.000 description 4
- 238000004590 computer program Methods 0.000 description 3
- 230000006870 function Effects 0.000 description 3
- 238000003062 neural network model Methods 0.000 description 3
- 230000009286 beneficial effect Effects 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 230000004927 fusion Effects 0.000 description 2
- 239000013307 optical fiber Substances 0.000 description 2
- 230000002093 peripheral effect Effects 0.000 description 2
- 230000000644 propagated effect Effects 0.000 description 2
- 201000004569 Blindness Diseases 0.000 description 1
- 238000003491 array Methods 0.000 description 1
- 230000015572 biosynthetic process Effects 0.000 description 1
- 238000004422 calculation algorithm Methods 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000002349 favourable effect Effects 0.000 description 1
- 230000008707 rearrangement Effects 0.000 description 1
- 230000010076 replication Effects 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
- 238000012216 screening Methods 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000003786 synthesis reaction Methods 0.000 description 1
- 238000012549 training Methods 0.000 description 1
- 238000012795 verification Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/77—Retouching; Inpainting; Scratch removal
Landscapes
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Image Processing (AREA)
Abstract
The embodiment of the invention discloses an image restoration method and device, electronic equipment and a storage medium. The method comprises the steps of obtaining at least two boundary points to be repaired on a picture to be repaired, and determining a target boundary point according to pixel information of the boundary points to be repaired and a preset boundary point determining condition; determining a region to be repaired of the target boundary point according to a preset region range to be repaired; determining a target area on the picture to be repaired according to the size of the area to be repaired and a preset target area determination range; and updating the area to be repaired into the target area, judging whether the boundary point to be repaired exists on the picture to be repaired, and if not, determining that the picture to be repaired is repaired. The technical scheme of the embodiment of the invention is suitable for being applied to large-area image restoration work, and all pixel points in the image do not need to be traversed for restoration, so that the calculation amount is saved, the restoration difficulty is reduced, and the image restoration efficiency is improved.
Description
Technical Field
The embodiment of the invention relates to the technical field of image processing, in particular to an image restoration method and device, electronic equipment and a storage medium.
Background
With the development of computer technology, image processing technology based on computer vision has been regarded as important because of high quality and efficiency of processed pictures.
At present, technologies such as texture synthesis, search sampling and deep learning are mainly used for repairing a missing part of an image, the technologies repair the missing image to different degrees, but the technologies have very large computation amount without exception, the problem of image blurring exists, the image repairing precision and repairing efficiency are low, and the technologies are not suitable for processing large-area images, so that the technologies are difficult to transplant to a mobile client.
Disclosure of Invention
The embodiment of the invention provides an image restoration method and device, electronic equipment and a storage medium, which are used for realizing restoration of a missing part of an image.
In a first aspect, an embodiment of the present invention provides an image repairing method, including:
acquiring at least two boundary points to be repaired on a picture to be repaired, and determining a target boundary point according to pixel information of the boundary points to be repaired and a preset boundary point determining condition;
determining a region to be repaired of the target boundary point according to a preset region range to be repaired;
determining a target area on the picture to be repaired according to the size of the area to be repaired and a preset target area determination range;
and updating the area to be repaired into the target area, judging whether the boundary point to be repaired exists on the picture to be repaired, and if not, determining that the picture to be repaired is repaired.
In a second aspect, an embodiment of the present invention further provides an apparatus for image restoration, including:
the target boundary point determining module is used for acquiring at least two boundary points to be repaired on the picture to be repaired, and determining the target boundary points according to the pixel information of the boundary points to be repaired and preset boundary point determining conditions;
the to-be-repaired area determining module is used for determining the to-be-repaired area of the target boundary point according to a preset to-be-repaired area range;
the target area determining module is used for determining a target area on the picture to be repaired according to the size of the area to be repaired and a preset target area determining range;
and the restoration judging module is used for updating the area to be restored into the target area, judging whether the boundary point to be restored exists on the picture to be restored, and if the boundary point to be restored does not exist, determining that the restoration of the picture to be restored is completed.
In a third aspect, an embodiment of the present invention further provides an electronic device, including a memory, a processor, a bus, and a communication interface; the memory is used for storing computer execution instructions, and the processor is connected with the memory through the bus;
when the electronic device runs, the processor executes the computer-executable instructions stored in the memory, so that the electronic device executes the method for image restoration according to any one of the embodiments of the present invention.
In a fourth aspect, the embodiment of the present invention further provides a computer-readable storage medium, where instructions are stored in the computer-readable storage medium, and when the instructions are executed by a computer, the computer is caused to execute the method for image restoration according to any one of the embodiments of the present invention.
According to the technical scheme of the embodiment of the invention, the target boundary point and the area to be repaired are determined on the boundary line of the incomplete area and the non-incomplete area of the picture to be repaired, the target area is determined according to the area to be repaired, and the area to be repaired is updated by the target area, so that the incomplete area is repaired. The method improves the precision and efficiency of repairing the picture by determining the region to be repaired near the target boundary point instead of randomly determining the region to be repaired, solves the defects that the image repairing method in the prior art is overlarge in calculated amount and is not suitable for repairing large-area images, repairs the whole picture on the basis of the boundary line of the incomplete region and the non-incomplete region, is suitable for being applied to large-area image repairing work, does not need to traverse all pixel points in the picture for repairing, saves the calculated amount, reduces the repairing difficulty and improves the efficiency of repairing the picture. By determining the target boundary point, the image area which needs to be repaired firstly in the image is repaired, so that the condition of image repairing blur is effectively reduced, repairing traces are reduced, and the pixel connection of the repaired image area and the image of the non-defective area is smooth.
Drawings
Fig. 1A is a flowchart of a method for repairing an image according to an embodiment of the present invention;
fig. 1B is a schematic diagram of a target boundary point in a picture to be repaired according to an embodiment of the present invention;
FIG. 2 is a flowchart of a method for repairing an image according to a second embodiment of the present invention;
fig. 3A is a flowchart of a method for repairing an image according to a third embodiment of the present invention;
fig. 3B is a schematic diagram of a candidate pixel point in a to-be-repaired picture according to a third embodiment of the present invention;
fig. 4 is a structural diagram of an image restoration apparatus according to a fourth embodiment of the present invention;
fig. 5 is a structural diagram of an electronic device according to a fifth embodiment of the present invention.
Detailed Description
The present invention will be described in further detail with reference to the accompanying drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the invention and are not limiting of the invention. It should be further noted that, for the convenience of description, only some of the structures related to the present invention are shown in the drawings, not all of the structures.
Example one
Fig. 1A is a flowchart of an image repairing method according to an embodiment of the present invention, where the embodiment of the present invention is applicable to a case of repairing an image defective portion, and the method may be executed by an image repairing apparatus, which may be implemented by software and/or hardware and is specifically configured in an electronic device.
Referring to fig. 1A, the method specifically includes the following steps:
s110, obtaining at least two boundary points to be repaired on the picture to be repaired, and determining conditions according to pixel information of the boundary points to be repaired and preset boundary points to determine target boundary points.
The boundary point to be repaired is a pixel point on a boundary line adjacent to a incomplete area and a non-incomplete area which need to be repaired on the picture to be repaired. The pixel information is information describing characteristics of the pixel itself, and may include, but is not limited to, RGB (Red Green Blue, Red Green Blue three color channel) values, color depth, saturation, and the like.
The preset boundary point determining condition is used for screening pixel points which can be used for image restoration from the boundary points to be restored, wherein the pixel points are target boundary points, namely pixel points which are used for positioning restoration areas and starting to restore the images in the image restoration process.
The preset boundary point determining condition may be set artificially, for example, it may be set on a boundary line adjacent to a defective region and a non-defective region of the picture to be repaired, and at least three colors formed by pixels around a certain pixel point (where the defective region of the picture to be repaired may be a blank or gray image, and thus may be regarded as having a color in the image processing process), that is, it is determined as a target boundary point; or a neural network model based on image processing can be obtained in advance through a large number of experimental trainings, the picture to be repaired is input into the model, and the model outputs the target boundary point.
In a specific example, as shown in fig. 1B, fig. 1B is a schematic diagram of a target boundary point in a picture to be repaired according to a first embodiment of the present invention. In the picture to be repaired, the curve 10 is a boundary line between the incomplete region and the non-incomplete region, the region 12 is the incomplete region, the regions 13 and 14 are the non-incomplete regions, the regions 13 and 14 have different colors, and the straight line 11 is used as a boundary line between the two colors, at this time, three different colors exist around the point a on the boundary line 10, so the point a is used as a target boundary point.
And S120, determining the area to be repaired of the target boundary point according to a preset area range to be repaired.
The preset area range to be repaired refers to a picture area with a preset size and taking the target boundary point as a center, and the optional position range of the area range to be repaired can be determined according to the adjacent boundary line of the incomplete area and the non-incomplete area of the picture to be repaired. For example, the range within the preset distance on both sides of the boundary line may be determined as the selectable position range of the area to be repaired, and the area to be repaired may be determined within the selectable position range.
The size of the area to be repaired may be set manually, or may be adaptively determined by the image repairing apparatus according to the size of the actual image and the size of the area to be repaired. For example, 99 × 99 pixels may be artificially set as the range of the region to be repaired, that is, the size of the region to be repaired is 99 × 99 pixels in each step of image repairing process.
After the position of the range of the area to be repaired and the size of the area to be repaired are determined, the position of the range of the area to be repaired can be determined according to the position of the target boundary point determined in the previous step, and the shape of the area to be repaired can be square, circular and the like. Optionally, the target boundary point may be used as a geometric center of the region to be repaired. Therefore, the area to be repaired may not be completely the incomplete area of the picture to be repaired, and the area to be repaired may include both the incomplete area and the non-incomplete area of the picture to be repaired. As shown in fig. 1B, the area to be repaired 15 includes both the defective area 12 and a part of the non-defective areas 13 and 14.
S130, determining a target area on the picture to be repaired according to the size of the area to be repaired and a preset target area determining range.
The target area refers to an image area used for repairing the area to be repaired in the non-defective area of the picture to be repaired. The target area determination range is an image range in which the target area can be selected in the non-defective area. The preset target region determination range may be determined in the vicinity of a boundary line near the culled region in the non-culled region, the range all belonging to the non-culled region. And according to the area to be repaired and the predetermined target area range, searching an area which is in accordance with the target area range and has the same size as the area to be repaired on the picture to be repaired to serve as the target area.
As shown in fig. 1B, since the colors of the non-defective regions 13 and 14 included in the region to be repaired 15 are different, the region to be repaired 15 includes both the colors and is located on the two-color boundary line 11, and a region having the same size as the region to be repaired 15 can be found in the vicinity of the boundary line 11 as a target region, so that the region 16 can be selected as the target region. The selection of the target region in the above-mentioned pictures is merely used as an explanatory reference and should not be construed as a limitation of the target region.
S140, updating the area to be repaired into the target area, judging whether the boundary point to be repaired exists on the picture to be repaired, and if not, determining that the picture to be repaired is repaired.
Copying and pasting the target area determined in the previous step to the area to be repaired is an updating operation, checking whether the boundary points to be repaired exist in the picture after the updating is finished, and if no boundary line adjacent to the incomplete area and the non-incomplete area exists, determining that the current picture is repaired successfully.
According to the technical scheme of the embodiment of the invention, the target boundary point and the area to be repaired are determined on the boundary line of the incomplete area and the non-incomplete area of the picture to be repaired, the target area is determined according to the area to be repaired, and the area to be repaired is updated by the target area, so that the incomplete area is repaired. The method improves the precision and efficiency of repairing the picture by determining the region to be repaired near the target boundary point instead of randomly determining the region to be repaired, solves the defects that the image repairing method in the prior art is overlarge in calculated amount and is not suitable for repairing the image of the large region, repairs the whole picture on the basis of the boundary line of the incomplete region and the non-incomplete region, is suitable for being applied to the large-area image repairing work, does not need to traverse all pixel points in the picture for repairing, saves the calculated amount, reduces the repairing difficulty and improves the efficiency of repairing the picture. By determining the target boundary point, the image area which needs to be repaired firstly in the repaired image is determined, the condition of image repairing blur is effectively reduced, repairing traces are reduced, and the pixel connection of the repaired image area and the image of the non-defective area is smooth.
In an optional implementation manner, after determining whether a boundary point to be repaired exists on the picture to be repaired, the method may further include: and if the boundary points to be repaired exist on the picture to be repaired, executing the step of acquiring at least two boundary points to be repaired on the picture to be repaired, and determining a target boundary point according to the pixel information of the boundary points to be repaired and a preset boundary point determination condition.
Specifically, if the boundary line between the incomplete area and the non-incomplete area still exists in the picture to be repaired, that is, the boundary point to be repaired inevitably exists, then S110 is executed. When the picture to be repaired is repaired, each repair is to repair an area where the target boundary point is located, but not to repair a single target boundary point, so that the repaired boundary line and the repaired boundary point can be changed. Determining a new target boundary point on the boundary line, determining a region to be repaired and a target region according to the target boundary point, repairing a defective region near the target boundary point until no boundary point exists, and completing the repair of the picture; and if the boundary point still exists, continuing to repair the current picture to be repaired. The method has the advantages that all the incomplete areas of the picture are guaranteed to be effectively repaired, the target boundary points are determined in each repairing process, and the precision of the picture repairing is improved.
Example two
Fig. 2 is a flowchart of an image repairing method according to a second embodiment of the present invention. The embodiment of the invention optimizes the determination operation of the target boundary point on the basis of the technical schemes of the embodiment so as to determine the target boundary point according to the repair priority and the repair rule.
Referring to fig. 2, the method for repairing an image specifically includes the following steps:
s210, determining the repair priority of the boundary point to be repaired according to the pixel information of the boundary point to be repaired; wherein the pixel information comprises pixel point coordinates.
The repairing priority is to sort the repairing sequence of all the boundary points to be repaired, and the boundary points to be repaired with high priority are repaired before other boundary points to be repaired. Optionally, the priority determination method may be set artificially, for example, the confidence of the boundary point to be repaired is multiplied by the pixel information to be used as a priority parameter for comparison; the priority can also be calculated by a pre-trained neural network model.
S220, determining a target boundary point from the boundary points to be repaired according to the repair priority and a preset priority repair rule.
The priority repairing rule is a rule for determining a repairing order of boundary points to be repaired based on priority, for example, images around the boundary points to be repaired with high repairing priority can be repaired first, so that the boundary points to be repaired with the highest current repairing priority can be used as target boundary points. It can be understood that, after the image around the previous target boundary point is repaired, the boundary line between the incomplete region and the non-incomplete region changes, and the boundary point to be repaired determined according to the pixel characteristics near the boundary line changes, so that the original repair priority ranking may have the situation that the boundary line changes, the repair priority may be determined again according to the repaired boundary line, the next target boundary point is selected according to the repair priority, and the steps are repeated until the image is completely repaired.
And S230, determining the area to be repaired of the target boundary point according to a preset area range to be repaired.
S240, determining a target area on the picture to be repaired according to the size of the area to be repaired and a preset target area determining range.
And S250, updating the area to be repaired into the target area, judging whether the boundary point to be repaired exists on the picture to be repaired, and if not, determining that the picture to be repaired is repaired.
In an optional implementation manner, the determining, according to the pixel information of the boundary point to be repaired, the repair priority of the boundary point to be repaired may include: determining the coordinate value of the boundary point to be repaired according to the pixel point coordinate of the boundary point to be repaired; and sorting the coordinate values of the boundary points to be repaired, and determining the repair priority of the boundary points to be repaired according to a preset priority sorting rule.
Specifically, a two-dimensional coordinate system can be established for the pixels in the picture based on the number of the pixels in the whole picture, and the coordinate value is used for each pixel crown, and the determination of the priority can be influenced by the size of the coordinate value. For example, a two-dimensional coordinate system is established with the geometric center of the image as an origin, and the coordinate values of the pixel points are calculated according to the horizontal and vertical coordinates of the pixel points, and it can be preset that the larger the coordinate values are, the higher the repair priority is. For another example, in the case where the abscissa of the boundary point to be repaired is the same, the larger the absolute value of the ordinate, the higher the priority thereof, and the positional order is provided as a reference for determining the repair order of the boundary points to be repaired. .
Therefore, according to the technical scheme of the embodiment, the coordinate values are given to the pixel points, the sequence of searching the positions of the target repair points is determined according to the sizes of the coordinate values, the calculated amount is saved, meanwhile, an effective basis is provided for determining the priority of the boundary points to be repaired, and the efficiency of determining the target repair points is improved. The target repairing point is determined through the priority, the protruding part with the large absolute value of the coordinates in the incomplete area can be repaired preferentially, the non-incomplete area is repaired from outside to inside, and the repairing efficiency of the picture is improved.
According to the technical scheme of the embodiment of the invention, the position coordinate of the boundary point to be repaired is used as one of the conditions for evaluating the repair priority, so that the position sequence is provided for selecting the target boundary point from the boundary points to be repaired, the defects of blindness caused by randomly selecting the target repair point, large calculation amount caused by randomly comparing the priority and the like are overcome, the efficiency of selecting the target repair point is improved, and a foundation is provided for repairing the whole picture.
EXAMPLE III
Fig. 3A is a flowchart of an image repairing method according to a third embodiment of the present invention. The embodiment of the invention optimizes the determination operation of the target area on the basis of the technical scheme of the embodiment so as to realize the selection of the target area.
Referring to fig. 3A, the method for repairing an image specifically includes the following steps:
s310, obtaining at least two boundary points to be repaired on the picture to be repaired, and determining conditions according to pixel information of the boundary points to be repaired and preset boundary points to determine target boundary points.
S320, determining the area to be repaired of the target boundary point according to a preset area range to be repaired.
S330, determining candidate pixel points in a preset target area determination range.
The preset target area determination range is determined in the vicinity of the non-incomplete area of the picture to be repaired close to the boundary line, and may be automatically determined by a pre-trained neural network model, for example, when the picture is input into the model, the model automatically determines the target area determination range. And then selecting the candidate pixel points in the range, wherein the selected rule can be that similar pixel points are searched for as the candidate pixel points according to the characteristics of the pixel information around the target boundary point.
As shown in fig. 3B, fig. 3B is a schematic diagram of a candidate pixel point in a picture to be repaired according to a third embodiment of the present invention. The geometric center point a of the region to be repaired 35 on the boundary line 30 is a target boundary point, the straight line 31 divides the non-incomplete region into two color parts 33 and 34, the point a is on the straight line 31, and thus it can be seen that there are at least two colors of pixels around the point a, so that a candidate pixel point is selected on the straight line 31, and it can be ensured that the colors, the number of colors, and the positions of pixels around the candidate pixel point are the same as those around the point a, and therefore, the point B can be selected as the candidate pixel point. And the candidate pixel points are selected close to the boundary line, so that the pixel characteristics of the pixels near the candidate pixel points are similar to the pixel characteristics of the pixels near the target boundary point, the image restoration precision is improved, the boundary blurring is avoided, and the image restoration fluency is improved.
And S340, determining a candidate region with the size consistent with that of the region to be repaired by taking the candidate pixel point as a center.
And taking the candidate pixel points determined in the step S330 as geometric centers, and determining a candidate region having a size consistent with that of the region to be repaired, that is, the number of the horizontal pixel points in the region is equal to that of the horizontal pixel points in the region to be repaired, and the number of the vertical pixel points in the region is equal to that of the vertical pixel points in the region to be repaired. For example, if the current region to be repaired contains 99 × 99 pixels, the candidate pixels are taken as the geometric center, and the image region of 99 × 99 pixels is selected as the candidate region.
As shown in fig. 3B, after the candidate pixel point B is determined, the target area 36 is determined according to the size of the area to be repaired 35.
S350, determining the square error of the pixel values of the area to be repaired and the candidate area, and judging whether the square error of the pixel values meets a preset square error condition.
The pixel square error can be calculated by the geometric distance of RGB, the preset square error condition can be an error range set by human, and the pixel value of the candidate region and the pixel value of the region to be repaired can be subjected to square error calculation. For example, the euclidean distance between the candidate region and the pixel of the region to be repaired may be calculated as the square error. Specifically, the square error of the pixel values of the image in the determined region to be repaired and the candidate region is used to check whether the error can meet a preset error range.
And S360, if so, determining the candidate area as a target area.
And if the candidate area meets the preset square error condition, determining the candidate area as the target area. If not, searching the candidate pixel points and the candidate area around the area to be repaired again, continuing to calculate the square error of the candidate area and the area to be repaired until finding the candidate area meeting the preset square error condition, and taking the candidate area as the target area.
And S370, updating the area to be repaired into the target area, judging whether the boundary point to be repaired exists on the picture to be repaired, and if not, determining that the picture to be repaired is repaired.
In an optional implementation manner, the determining a target area on the to-be-repaired picture according to the size of the to-be-repaired area and a preset target area determination range may further include: determining at least two candidate areas on the picture to be repaired according to a preset target area determination range; the size of the candidate region is consistent with that of the region to be repaired; determining the square error of the pixel values of the area to be repaired and at least two candidate areas, and judging whether the square error of the pixel values corresponding to the candidate areas meets a preset square error condition or not; and if so, determining the candidate area as a target area.
When selecting the candidate region, at least two candidate pixel points can be selected in a preset target region determining range, and at least two candidate regions are determined according to the at least two candidate pixel points. And a plurality of candidate regions can be directly selected from the preset target region determining range, so that the candidate regions determined by the candidate pixel points are prevented from exceeding the preset target region determining range. The target region determination range may be an image range in the vicinity of the boundary line. Calculating the square error of the pixel values of the candidate areas relative to the target area, and if one area meeting a preset square error condition exists in the candidate areas, taking the candidate area meeting the preset square error condition as the target area; and if a plurality of candidate areas meeting the preset square error condition exist, setting all the candidate areas as the areas to be superposed. For example, the preset square error condition may be that the euclidean distance between the pixel points of the candidate region and the pixel points of the target region is less than 100 pixel unit distances. The beneficial effect who sets up like this lies in, has improved the efficiency of picture restoration.
In an optional implementation manner, after determining whether there is a square error of a pixel value corresponding to the candidate region that satisfies a preset square error condition, the method may further include: and if the square error of the pixel values of at least two candidate areas meets the preset square error condition, overlapping the at least two candidate areas meeting the preset square error condition to obtain the target area.
Specifically, if at least two candidate regions meet a preset error range, image pixels of the at least two candidate regions meeting the error range are overlapped, that is, the regions to be overlapped are overlapped. Because the sizes of all the candidate regions are equal to the to-be-repaired region, the size of the new candidate region after being overlapped is also equal to the to-be-repaired region, and the new candidate region after being overlapped is used as a target region to repair the to-be-repaired region.
Through the fusion of a plurality of candidate regions, the robustness of the image restoration algorithm is improved. The problem of complex texture replication errors is solved to some extent. In this embodiment, the number of candidate regions to be fused may be limited, and loss of texture details during fusion may be avoided. Meanwhile, the pixel information of the new candidate area can be updated, so that the pixel characteristics of the new candidate area are more favorable for repairing the area to be repaired, the repairing quality is improved, and the image repairing effect and efficiency are improved.
According to the technical scheme of the embodiment of the invention, appropriate candidate pixel points are selected around the area to be repaired, the candidate area is determined, and the target area is determined according to the square error value, so that the target area close to the pixel characteristics near the area to be repaired can be accurately found, the picture is repaired by the target area, the precision and the effect of picture repair are improved, the target area does not need to be searched by traversing pixels, a large amount of calculation can be saved, the repair process is simplified, and the repair efficiency is improved.
Example four
Fig. 4 is a structural diagram of an image repairing apparatus according to a fourth embodiment of the present invention, where the embodiment of the present invention is applicable to a situation of repairing an image defective portion, the apparatus may be implemented by software and/or hardware, and may be configured in an electronic device. As shown in fig. 4, the apparatus may include:
a target boundary point determining module 410, configured to obtain at least two boundary points to be repaired on a picture to be repaired, and determine a target boundary point according to pixel information of the boundary points to be repaired and a preset boundary point determining condition;
a to-be-repaired area determining module 420, configured to determine, according to a preset to-be-repaired area range, an to-be-repaired area of the target boundary point;
a target area determining module 430, configured to determine a target area on the picture to be repaired according to the size of the area to be repaired and a preset target area determining range;
the repair determining module 440 is configured to update the to-be-repaired area to the target area, determine whether a to-be-repaired boundary point exists on the to-be-repaired picture, and if not, determine that the to-be-repaired picture is repaired.
According to the technical scheme of the embodiment of the invention, the target boundary point and the area to be repaired are determined on the boundary line of the incomplete area and the non-incomplete area of the picture to be repaired, the target area is determined according to the area to be repaired, and the area to be repaired is updated by the target area, so that the incomplete area is repaired. The method improves the precision and efficiency of repairing the picture by determining the region to be repaired near the target boundary point instead of randomly determining the region to be repaired, solves the defects that the image repairing method in the prior art is overlarge in calculated amount and is not suitable for repairing the image of the large region, repairs the whole picture on the basis of the boundary line of the incomplete region and the non-incomplete region, is suitable for being applied to the large-area image repairing work, does not need to traverse all pixel points in the picture for repairing, saves the calculated amount, reduces the repairing difficulty and improves the efficiency of repairing the picture. By determining the target boundary point, the image area which needs to be repaired firstly in the image is repaired, so that the condition of image repairing blur is effectively reduced, repairing traces are reduced, and the pixel connection of the repaired image area and the image of the non-defective area is smooth.
In an alternative embodiment, the target boundary point determining module 410 may include:
the priority determining unit is used for determining the repairing priority of the boundary point to be repaired according to the pixel information of the boundary point to be repaired; wherein the pixel information comprises pixel point coordinates;
and the target boundary point determining unit is used for determining a target boundary point from the boundary points to be repaired according to the repair priority and a preset priority repair rule.
In an optional embodiment, the priority determining unit may include:
a coordinate value determining subunit, configured to determine, according to the pixel point coordinate of the boundary point to be repaired, a coordinate value of the boundary point to be repaired;
and the priority sorting subunit is used for sorting the coordinate values of the boundary points to be repaired and determining the repair priority of the boundary points to be repaired according to a preset priority sorting rule.
In an alternative embodiment, the target area determination module 430 may include:
the candidate pixel point determining unit is used for determining candidate pixel points in a preset target area determining range;
a candidate region determining unit, configured to determine, with the candidate pixel point as a center, a candidate region having a size that is consistent with the size of the region to be repaired;
the condition judgment unit is used for determining the square error of the pixel value of the area to be repaired and the candidate area and judging whether the square error of the pixel value meets a preset square error condition or not;
and the target determining unit is used for determining the candidate area as the target area if the candidate area is the target area.
In an alternative embodiment, the target area determination module 430 may include:
the candidate area selection unit is used for determining at least two candidate areas on the picture to be repaired according to a preset target area determination range; the size of the candidate region is consistent with that of the region to be repaired;
the condition verification unit is used for determining the square errors of the pixel values of the area to be repaired and at least two candidate areas and judging whether the square errors of the pixel values corresponding to the candidate areas meet a preset square error condition or not;
and the area determining unit is used for determining the candidate area as the target area if the candidate area is the target area.
In an optional implementation, the target area determination module 430 may further include:
and the target area obtaining unit is used for superposing the at least two candidate areas meeting the preset square error condition to obtain the target area if the square errors of the pixel values of the at least two candidate areas meet the preset square error condition after judging whether the square errors of the pixel values corresponding to the candidate areas meet the preset square error condition.
In an alternative embodiment, the apparatus for image restoration may further include:
and the to-be-repaired boundary point judging module is used for acquiring at least two to-be-repaired boundary points on the to-be-repaired picture if the to-be-repaired boundary points exist on the to-be-repaired picture after judging whether the to-be-repaired boundary points exist on the to-be-repaired picture, and determining a target boundary point according to pixel information of the to-be-repaired boundary points and a preset boundary point determining condition.
The image restoration device provided by the embodiment of the invention can execute the image restoration method provided by any embodiment of the invention, and has the corresponding functional modules and beneficial effects of the image restoration methods.
EXAMPLE five
Fig. 5 is a structural diagram of an electronic device according to a fifth embodiment of the present invention. FIG. 5 illustrates a block diagram of an exemplary electronic device 512 that may be suitable for use in implementing embodiments of the present invention. The electronic device 512 shown in fig. 5 is only an example and should not bring any limitations to the function and scope of use of the embodiments of the present invention.
As shown in fig. 5, electronic device 512 is in the form of a general purpose computing device. Components of the electronic device 512 may include, but are not limited to: one or more processors or processing units 516, a system memory 528, and a bus 518 that couples the various system components including the system memory 528 and the processing unit 516.
The system memory 528 may include computer system readable media in the form of volatile memory, such as Random Access Memory (RAM)530 and/or cache memory 532. The electronic device 512 may further include other removable/non-removable, volatile/nonvolatile computer system storage media. By way of example only, storage system 534 may be used to read from and write to non-removable, nonvolatile magnetic media (not shown in FIG. 5, and commonly referred to as a "hard drive"). Although not shown in FIG. 5, a magnetic disk drive for reading from and writing to a removable, nonvolatile magnetic disk (e.g., a "floppy disk") and an optical disk drive for reading from or writing to a removable, nonvolatile optical disk (e.g., a CD-ROM, DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to bus 518 through one or more data media interfaces. Memory 528 may include at least one program product having a set (e.g., at least one) of program modules that are configured to carry out the functions of embodiments of the invention.
A program/utility 540 having a set (at least one) of program modules 542, including but not limited to an operating system, one or more application programs, other program modules, and program data, may be stored in, for example, the memory 528, each of which examples or some combination may include an implementation of a network environment. The program modules 542 generally perform the functions and/or methods of the described embodiments of the invention.
The electronic device 512 may also communicate with one or more external devices 514 (e.g., keyboard, pointing device, display 524, etc.), with one or more devices that enable a user to interact with the electronic device 512, and/or with any devices (e.g., network card, modem, etc.) that enable the electronic device 512 to communicate with one or more other computing devices. Such communication may occur via input/output (I/O) interfaces 522. Also, the electronic device 512 may communicate with one or more networks (e.g., a Local Area Network (LAN), a Wide Area Network (WAN), and/or a public network, such as the internet) via the network adapter 520. As shown, the network adapter 520 communicates with the other modules of the electronic device 512 via the bus 518. It should be appreciated that although not shown, other hardware and/or software modules may be used in conjunction with the electronic device 512, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, among others.
The processing unit 516 executes various functional applications and data processing, such as implementing the method of image restoration provided by the embodiments of the present invention, by running at least one of the other programs stored in the system memory 528.
EXAMPLE six
The sixth embodiment of the present invention further provides a computer-readable storage medium, on which a computer program (or referred to as computer-executable instructions) is stored, where the computer program is used for executing, when executed by a processor, an image inpainting method provided by the sixth embodiment of the present invention: acquiring at least two boundary points to be repaired on a picture to be repaired, and determining a target boundary point according to pixel information of the boundary points to be repaired and a preset boundary point determining condition; determining a region to be repaired of the target boundary point according to a preset region range to be repaired; determining a target area on the picture to be repaired according to the size of the area to be repaired and a preset target area determination range; and updating the area to be repaired into the target area, judging whether the boundary point to be repaired exists on the picture to be repaired, and if not, determining that the picture to be repaired is repaired.
Computer storage media for embodiments of the invention may employ any combination of one or more computer-readable media. The computer readable medium may be a computer readable signal medium or a computer readable storage medium. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples (a non-exhaustive list) of the computer readable storage medium would include the following: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this document, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.
A computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.
Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.
Computer program code for carrying out operations for embodiments of the present invention may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C + +, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet service provider).
It is to be noted that the foregoing is only illustrative of the preferred embodiments of the present invention and the technical principles employed. It will be understood by those skilled in the art that the present invention is not limited to the particular embodiments described herein, but is capable of various obvious changes, rearrangements and substitutions as will now become apparent to those skilled in the art without departing from the scope of the invention. Therefore, although the present invention has been described in greater detail by the above embodiments, the present invention is not limited to the above embodiments, and may include other equivalent embodiments without departing from the spirit of the present invention, and the scope of the present invention is determined by the scope of the appended claims.
Claims (10)
1. A method of image inpainting, comprising:
acquiring at least two boundary points to be repaired on a picture to be repaired, and determining a target boundary point according to pixel information of the boundary points to be repaired and a preset boundary point determining condition;
determining a region to be repaired of the target boundary point according to a preset region range to be repaired;
determining a target area on the picture to be repaired according to the size of the area to be repaired and a preset target area determination range;
and updating the area to be repaired into the target area, judging whether the boundary point to be repaired exists on the picture to be repaired, and if not, determining that the picture to be repaired is repaired.
2. The method according to claim 1, wherein determining a target boundary point according to the pixel information of the boundary point to be repaired and a preset boundary point determination condition comprises:
determining the repair priority of the boundary point to be repaired according to the pixel information of the boundary point to be repaired; wherein the pixel information comprises pixel point coordinates;
and determining a target boundary point from the boundary points to be repaired according to the repair priority and a preset priority repair rule.
3. The method according to claim 2, wherein determining the repair priority of the boundary point to be repaired according to the pixel information of the boundary point to be repaired comprises:
determining the coordinate value of the boundary point to be repaired according to the pixel point coordinate of the boundary point to be repaired;
and sorting the coordinate values of the boundary points to be repaired, and determining the repair priority of the boundary points to be repaired according to a preset priority sorting rule.
4. The method according to claim 1, wherein determining a target area on the picture to be repaired according to the size of the area to be repaired and a preset target area determination range comprises:
determining candidate pixel points within a preset target area determination range;
determining a candidate area with the size consistent with that of the area to be repaired by taking the candidate pixel point as a center;
determining the square error of the pixel values of the area to be repaired and the candidate area, and judging whether the square error of the pixel values meets a preset square error condition or not;
and if so, determining the candidate area as a target area.
5. The method according to claim 1, wherein determining a target area on the picture to be repaired according to the size of the area to be repaired and a preset target area determination range comprises:
determining at least two candidate areas on the picture to be repaired according to a preset target area determination range; the size of the candidate region is consistent with that of the region to be repaired;
determining the square error of the pixel values of the area to be repaired and at least two candidate areas, and judging whether the square error of the pixel values corresponding to the candidate areas meets a preset square error condition or not;
and if so, determining the candidate area as a target area.
6. The method according to claim 5, after determining whether there is a square error of pixel values corresponding to the candidate region that satisfies a preset square error condition, further comprising:
and if the square error of the pixel values of at least two candidate areas meets the preset square error condition, overlapping the at least two candidate areas meeting the preset square error condition to obtain the target area.
7. The method according to claim 1, after determining whether there is a boundary point to be repaired on the picture to be repaired, further comprising:
and if the boundary points to be repaired exist on the picture to be repaired, executing the step of acquiring at least two boundary points to be repaired on the picture to be repaired, and determining a target boundary point according to the pixel information of the boundary points to be repaired and a preset boundary point determination condition.
8. An apparatus for image restoration, comprising:
the target boundary point determining module is used for acquiring at least two boundary points to be repaired on the picture to be repaired, and determining the target boundary points according to the pixel information of the boundary points to be repaired and preset boundary point determining conditions;
the to-be-repaired area determining module is used for determining the to-be-repaired area of the target boundary point according to a preset to-be-repaired area range;
the target area determining module is used for determining a target area on the picture to be repaired according to the size of the area to be repaired and a preset target area determining range;
and the restoration judging module is used for updating the area to be restored into the target area, judging whether the boundary point to be restored exists on the picture to be restored, and if the boundary point to be restored does not exist, determining that the restoration of the picture to be restored is completed.
9. An electronic device comprising a memory, a processor, a bus, and a communication interface; the memory is used for storing computer execution instructions, and the processor is connected with the memory through the bus;
when the electronic device is running, a processor executes the computer-executable instructions stored by the memory to cause the electronic device to perform the method of image inpainting according to any one of claims 1-7.
10. A computer-readable storage medium having stored therein instructions, which when executed by a computer, cause the computer to perform the method of image inpainting as claimed in any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111180704.8A CN113989135A (en) | 2021-10-11 | 2021-10-11 | Image restoration method and device, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111180704.8A CN113989135A (en) | 2021-10-11 | 2021-10-11 | Image restoration method and device, electronic equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113989135A true CN113989135A (en) | 2022-01-28 |
Family
ID=79738092
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111180704.8A Pending CN113989135A (en) | 2021-10-11 | 2021-10-11 | Image restoration method and device, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113989135A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2024179333A1 (en) * | 2023-02-28 | 2024-09-06 | 北京字跳网络技术有限公司 | Image processing method and apparatus, device, and storage medium |
-
2021
- 2021-10-11 CN CN202111180704.8A patent/CN113989135A/en active Pending
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2024179333A1 (en) * | 2023-02-28 | 2024-09-06 | 北京字跳网络技术有限公司 | Image processing method and apparatus, device, and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112348815B (en) | Image processing method, image processing apparatus, and non-transitory storage medium | |
| US11328401B2 (en) | Stationary object detecting method, apparatus and electronic device | |
| US11676257B2 (en) | Method and device for detecting defect of meal box, server, and storage medium | |
| CN109753971B (en) | Correction method and device for distorted text lines, character recognition method and device | |
| US20210295546A1 (en) | Satellite image processing method, network training method, related devices and electronic device | |
| CN110503619B (en) | Image processing method, device and readable storage medium | |
| WO2017219875A1 (en) | Hand detecting and tracking method and device | |
| WO2021169740A1 (en) | Image restoration method and apparatus, computer device, and storage medium | |
| CN112102201B (en) | Image shadow reflection eliminating method and device, computer equipment and storage medium | |
| CN105513083A (en) | PTAM camera tracking method and device | |
| CN111127376A (en) | Method and device for repairing digital video file | |
| CN109543683A (en) | Image labeling modification method, device, equipment and medium | |
| CN112559341A (en) | Picture testing method, device, equipment and storage medium | |
| CN115884480A (en) | Multi-optical-axis tripod head lamp control method and device based on image processing and storage medium | |
| CN113989135A (en) | Image restoration method and device, electronic equipment and storage medium | |
| CN114820679A (en) | Image annotation method and device, electronic equipment and storage medium | |
| CN103837135B (en) | Workpiece inspection method and system thereof | |
| CN113516697B (en) | Image registration method, device, electronic equipment and computer readable storage medium | |
| KR102366032B1 (en) | Method for determining defect of circuit board by learning auto-augmented circuit board image based on machine learning and computing device using the same | |
| CN118071719A (en) | Defect detection method, defect detection device, computer equipment and computer readable storage medium | |
| CN111815748A (en) | Animation processing method and device, storage medium and electronic equipment | |
| US10679049B2 (en) | Identifying hand drawn tables | |
| CN118265998A (en) | Dead pixel detection model training method, dead pixel detection method and dead pixel restoration method | |
| JP2023535662A (en) | Method, apparatus, apparatus, medium and computer program for restoring video | |
| CN114862642A (en) | Method for removing short video visible watermark and computer readable storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |