CN117132503A - Method, system, equipment and storage medium for repairing local highlight region of image - Google Patents
Method, system, equipment and storage medium for repairing local highlight region of image Download PDFInfo
- Publication number
- CN117132503A CN117132503A CN202311254350.6A CN202311254350A CN117132503A CN 117132503 A CN117132503 A CN 117132503A CN 202311254350 A CN202311254350 A CN 202311254350A CN 117132503 A CN117132503 A CN 117132503A
- Authority
- CN
- China
- Prior art keywords
- image
- highlight
- pixel
- repairing
- aperture
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 70
- 229920006395 saturated elastomer Polymers 0.000 claims abstract description 29
- 230000008439 repair process Effects 0.000 claims abstract description 18
- 230000000007 visual effect Effects 0.000 claims abstract description 11
- 238000004590 computer program Methods 0.000 claims description 10
- 230000003044 adaptive effect Effects 0.000 claims description 4
- 239000011159 matrix material Substances 0.000 claims description 3
- 230000009466 transformation Effects 0.000 claims description 3
- 238000004891 communication Methods 0.000 description 6
- 238000010586 diagram Methods 0.000 description 5
- 230000000694 effects Effects 0.000 description 5
- 238000009792 diffusion process Methods 0.000 description 3
- 238000003384 imaging method Methods 0.000 description 3
- 238000012986 modification Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 238000007500 overflow downdraw method Methods 0.000 description 3
- 230000008859 change Effects 0.000 description 2
- 238000000354 decomposition reaction Methods 0.000 description 2
- 230000007547 defect Effects 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 230000004075 alteration Effects 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 238000007499 fusion processing Methods 0.000 description 1
- 125000001475 halogen functional group Chemical group 0.000 description 1
- 238000003709 image segmentation Methods 0.000 description 1
- 230000005764 inhibitory process Effects 0.000 description 1
- 238000000926 separation method Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/762—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using clustering, e.g. of similar faces in social networks
- G06V10/763—Non-hierarchical techniques, e.g. based on statistics of modelling distributions
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
- G06V10/761—Proximity, similarity or dissimilarity measures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/80—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Software Systems (AREA)
- Physics & Mathematics (AREA)
- Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- Computing Systems (AREA)
- Databases & Information Systems (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Medical Informatics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Probability & Statistics with Applications (AREA)
- Image Processing (AREA)
Abstract
The application provides a method, a system, equipment and a storage medium for repairing a local highlight region of an image, which comprise the following steps: s1, obtaining an optimal sub-aperture image by fusing sub-aperture images with different visual angles; s2, classifying the highlight pixel points into a saturated highlight pixel point and an unsaturated highlight pixel point by taking the acquired optimal sub-aperture image as input and utilizing the multi-view characteristic of the light field and combining an x-means mean value clustering algorithm; s3, repairing the unsaturated highlight pixel by using a highlight pixel repairing method based on a bicolor reflection model for the extracted unsaturated highlight pixel; and S4, using the highlight image after the repair of the unsaturated pixel points as an input image, and repairing the saturated highlight pixels by using a priority-based self-adaptive direction method. The method effectively removes the local mirror highlight region contained in the image, restores the texture information of the image surface, solves the limitation of the traditional highlight removal algorithm, and improves the robustness of the highlight removal algorithm.
Description
Technical Field
The application relates to the technical field of light field imaging, in particular to a method, a system, equipment and a storage medium for repairing a local highlight region of an image.
Background
It is well known that objects typically have diffuse and specular reflection under natural light. However, the presence of specular highlights can hide useful information of the object image in artifact detection, object recognition, and three-dimensional reconstruction, destroying the original texture features of the object. Therefore, how to remove the high light in the natural light environment and obtain a clear and complete image is always a concern in the field of computer vision. Traditional camera imaging cannot obtain three-dimensional information of the surface of a measured object. Meanwhile, the existing highlight removing algorithm is limited by the measured object and the shooting scene, and the high reflection removal of a single image is very unstable depending on the information decomposition inside the image. The multi-image method requires multiple shots, which increases acquisition time and computational complexity. The specular highlight image restoration based on the light field multi-view characteristic is not influenced by the measured object, only single shooting is needed, and the space complexity of image acquisition is reduced.
Aiming at the high reflection removal of the image surface, in the prior art, the method based on a plurality of images has strict requirements on shooting conditions, ambient light or angle, and severely limits the applicability of the performance. The complexity of image acquisition based on the single image method is low, the image is dependent on the intrinsic decomposition of the image, and the processing effect is unstable and the robustness is low to fail. The method based on the additional hardware has the defects of higher complexity and poor timeliness of image acquisition due to the limitation of a system structure.
Therefore, the method is used for solving the problem of utilizing the multi-view characteristic of the light field camera, effectively removing small area highlights in an image, simultaneously repairing unsaturated highlights by combining a method based on a bicolor reflection model, repairing saturated highlights by using a priority-based self-adaptive direction method, reducing the space complexity of image acquisition, improving the accuracy of depth estimation by excellent highlight inhibition, and having practical significance and good application prospect.
Disclosure of Invention
Aiming at the technical problems in the prior art, the application provides a method, a system, equipment and a storage medium for repairing a local highlight region of an image, which solve the limitations of the traditional highlight removal algorithm and improve the robustness of the highlight removal algorithm.
According to a first aspect of the present application, the present application provides a method for repairing a locally highlighted area of an image, comprising the steps of:
step 1, obtaining an optimal sub-aperture image by fusing sub-aperture images with different visual angles;
step 2, classifying the highlight pixels into saturated highlight pixels and unsaturated highlight pixels by taking the acquired optimal sub-aperture image as input and utilizing the multi-view characteristic of the light field and combining an x-means mean value clustering algorithm;
step 3, repairing the unsaturated highlight pixel by using a highlight pixel repairing method based on a bicolor reflection model for the extracted unsaturated highlight pixel;
step 4, using the highlight image after the unsaturated pixel point repair is completed as an input image, and repairing the saturated highlight pixel point by using a priority-based self-adaptive direction method.
On the basis of the technical scheme, the application can also make the following improvements.
Optionally, the obtaining the optimal sub-aperture image by fusing the sub-aperture images of different viewing angles includes:
and acquiring images, decoding and extracting sub-aperture images through a light field toolbox, and fusing and eliminating small-area highlight pixel points to extract an optimal sub-aperture image.
Optionally, the fusing the sub-aperture images with different viewing angles includes the following steps:
step 1, selecting a central sub-aperture image I with higher image quality from a plurality of sub-aperture images c As a target image, separating out a highlight region { M } of the sub-aperture image by using an x-means mean value clustering algorithm;
step 2, aligning images of different viewpoints to a sub-aperture image Ic through a rigid body transformation matrix according to an ICP algorithm;
step 3, obtaining a fused image I by fusing the multi-view images f From the fused image I according to the highlight region f Extracting a target area;
step 4, from the fused image I f Extracting pixels of the target area, and splicing the pixels to the missing area in the center view image to obtain an optimal image I o 。
Preferably, the classifying the highlight pixel points by utilizing the multi-view characteristic of the light field and combining an x-means mean value clustering algorithm simultaneously comprises the following steps: and differentiating the obtained optimal sub-aperture image, and differentiating each pixel of the optimal sub-aperture image.
Preferably, the differencing each pixel of the optimal sub-aperture image specifically includes: and extracting specular highlight pixels in the optimal sub-aperture image by using an x-means mean clustering algorithm, refocusing the image, and calculating variance through the multi-viewpoint light field image.
Preferably, the repairing of the unsaturated highlight pixel by using the highlight pixel repairing method based on the bicolor reflection model specifically comprises the following steps:
according to the bicolor reflection model, the original image Iua Is composed of specular reflection component Is and diffuse reflection component I b Composition;
clustering the mirror Gao Guangxiang pixels and the diffuse reflection pixels by adopting an x-means mean value clustering algorithm, and calculating a threshold value of a diffuse reflection area;
evaluating the local highlight region, and repairing the diffusion component I of the unsaturated highlight pixel by adopting a highlight pixel repairing method b 。
Preferably, the repairing saturated highlight pixels by using the priority-based adaptive direction method specifically includes: the following formula is substituted:
wherein Ib' represents the restored image, ψ represents the image stitching of the light field, filling the best selected 3×3 pixel block into the region to be restored; (u, v) represents direction information of the light ray, (x ', y') represents a start point coordinate of the best selection of the 3 x 3 pixel block, and (x, y) represents a start point of the region to be repaired.
According to a second aspect of the present application, there is provided an image local highlight region restoration system comprising:
the image acquisition module is used for fusing the sub-aperture images with different visual angles to obtain an optimal sub-aperture image;
the image classification module is used for classifying the highlight pixel points into saturated highlight pixel points and unsaturated highlight pixel points by taking the acquired optimal sub-aperture image as input and utilizing the multi-view characteristic of the light field and combining an x-means mean value clustering algorithm;
the first repair module is used for repairing the unsaturated highlight pixel by using a highlight pixel repair method based on a bicolor reflection model for the extracted unsaturated highlight pixel;
and the second restoration module is used for taking the highlight image after the unsaturated pixel point restoration is completed as an input image, and restoring the saturated highlight pixels by using a priority-based self-adaptive direction method.
The application also provides an electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor realizes the method for repairing the local highlight region of the image when executing the program.
The present application also provides a non-transitory computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements an image local highlight region restoration method as described above.
The application has the technical effects and advantages that:
the application provides a method, a system, equipment and a storage medium for repairing a local highlight region of an image, which effectively remove the local highlight region of a mirror surface in the image and recover the texture information of the surface of the image; according to the method, the highlight pixel points are classified into saturated highlight pixel points and unsaturated highlight pixel points by combining the multi-view characteristic of the light field, so that the limitation of the traditional highlight removal algorithm is solved, and the robustness of the highlight removal algorithm is improved.
Additional features and advantages of the application will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by practice of the application. The objectives and other advantages of the application may be realized and attained by the structure particularly pointed out in the written description and claims hereof as well as the appended drawings.
Drawings
FIG. 1 is a flow chart of a method for repairing a local highlight region based on an image according to an embodiment of the present application;
FIG. 2 is a schematic diagram of distinguishing different types of highly reflective pixels according to an embodiment of the present application;
FIG. 3 is a graph showing the effect of removing high reflection of an image according to an embodiment of the present application;
FIG. 4 is a schematic diagram of optimal sub-aperture image acquisition and highlight restoration according to an embodiment of the present application;
FIG. 5 is a graph showing the comparison of the effects of different methods provided by embodiments of the present application;
fig. 6 is a schematic hardware structure of a possible electronic device according to the present application.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the present application more apparent, the technical solutions of the present application will be clearly and completely described below with reference to the accompanying drawings, and it is apparent that the described embodiments are some embodiments of the present application, not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the application without making any inventive effort, are intended to be within the scope of the application.
It can be appreciated that, based on the defects in the background technology, the embodiment of the application provides a method for repairing a local highlight region of an image, and referring to fig. 1, the repairing method specifically comprises the following steps:
step 1, obtaining an optimal sub-aperture image by fusing sub-aperture images with different visual angles;
it should be noted that, in the embodiment of the present application, a Lytro Illum light field camera is used as an imaging device, and the object to be measured is placed in the field of view of the camera.
The acquired images are decoded and extracted through a Matlab light field toolbox, and sub-aperture images with different visual angles are fused to eliminate highlight pixel points with small areas to extract an optimal sub-aperture image.
And in a further step, decoding the light field original image through a Matlab light field toolbox to obtain a full-view sub-aperture image, fusing different sub-aperture images by adopting a multi-view image fusion method, and removing local small-scale mirror surface highlight to obtain an optimal sub-aperture image.
Fig. 2 is a schematic diagram of distinguishing different types of high-reflection pixels according to an embodiment of the present application, and as shown in fig. 2, a fusion process of different sub-aperture images by using a multi-view image fusion method is as follows:
step 1, identifying a missing region; taking the edge distortion and halo effect of a lens into consideration, selecting a center sub-aperture image Ic with higher image quality in a plurality of sub-aperture images as a target image, and separating out a highlight region { M } of the sub-aperture images by using an x-means mean value clustering algorithm;
step 2, aligning the images; because the distribution of the highlight areas changes along with the change of the observation angle, images of different visual angles have certain complementarity, and the multi-visual angle images are fused under one visual angle; images of different viewpoints are aligned to the sub-aperture image Ic by a rigid body transformation matrix according to the ICP algorithm.
Step 3, extracting a target area; obtaining a fused image I by fusing multi-view images f From the fused image I according to the highlight region f Extracting a target area;
step 4, splicing the missing areas; from the fused image I f And (3) extracting pixels of the target area, and splicing the pixels to the missing area in the center view image to obtain an optimal image Io.
Step 2, classifying the highlight pixels into saturated highlight pixels and unsaturated highlight pixels by taking the acquired optimal sub-aperture image as input and utilizing the multi-view characteristic of the light field and combining an x-means mean value clustering algorithm;
the multi-view characteristic of the light field is characterized in that the gray value of the high-light pixel changes along with the observation angle; the x-means mean clustering algorithm is an unsupervised clustering algorithm similar to the Euclidean distance-based clustering algorithm, and is defined as the closer the distance between two targets is, the greater the similarity is. The method can be used for determining the optimal K value in the K-means algorithm, and is often used in the field of image segmentation.
The classifying the highlight pixel points by utilizing the multi-view characteristic of the light field and combining an x-means mean value clustering algorithm specifically comprises the following steps: and differentiating the obtained optimal sub-aperture image, and differentiating (subtracting) each pixel of the optimal sub-aperture image.
The step of differencing each pixel of the optimal sub-aperture image specifically comprises the following steps: and extracting specular highlight pixels in the optimal sub-aperture image by using an x-means mean clustering algorithm, refocusing the image, and calculating variance through the multi-viewpoint light field image.
Step 3, repairing the unsaturated highlight pixel by using a highlight region pixel repairing method based on a bicolor reflection model for the extracted unsaturated highlight pixel;
in this embodiment, the non-saturated highlight pixels are repaired using a highlight pixel repair method based on a bicolor reflection model, according to which the original image I ua From specular reflection component I s And diffuse reflection component I b Two parts.
I ua ＝I b +I s (1)
Clustering the mirror Gao Guangxiang pixels and the diffuse reflection pixels by adopting an x-means mean value clustering algorithm, and calculating a threshold value of a diffuse reflection area:
τ＝I ua(min) -ασ I (2)
I ua(min) mirror surface highlight pixel with lowest gray value and sigma I Alpha (here 0.8) represents the standard deviation sigma of the image, which is the standard deviation of the gray values of the image I Is a coefficient of (a). Local areaThe highlight region highlight components were evaluated as follows:
V ps ＝I ua -τ ( V ps ≥0) (3)
V ps is a pseudo specular component reflection component. Since the diffuse reflection image is smoothly continuous, the diffuse reflection pixel fluctuation in the same area is small. The pixel extraction (3 pixels in this example) along the boundary of the locally highlighted region is considered to be a valid surrounding region. The diffuse reflection components of the two regions are set equal.
V ps '＝I ua' -τ (V ps '≥0). (4)
I uh' Is the pixel around the highlight region, V ps' Is the pseudo specular component of the pixel near the highly reflective region-meaning that the average is taken and k is the coefficient of uncertainty solved by equation (6). Finally, the diffusion component I of each unsaturated highlight pixel repaired by the highlight pixel repairing method b Expressed as:
the diffuse reflection component is expressed as:
as shown in fig. 3, the number of diffuse reflection components included in different saturated pixels is different, rectangular blocks with protruding middle areas indicate saturated highlight pixels, and surrounding rectangular blocks indicate diffuse reflection pixels. Therefore, it is necessary to calculate the priority of each pixel and to repair more diffusion components around the saturated pixel first. Further consider that the contributions of neighboring pixels in different directions are different. Fig. 3 (a) shows saturated pixels p1 and p2 at different positions. We calculate the contribution values of neighboring pixels in different directions by calculating the gradient. As shown in fig. 3 (b), the points p1 and p2 have 8 adjacent pixel points in 8 directions. Where the p1 point has three directions of adjacent pixels, which are also saturated high-light pixels (circular arrows), which are considered as invalid directions. Generally, the diffuse reflection image is smoothly continuous, so we should choose the direction (arrow) with the smallest gradient change among the remaining 5 effective directions as the best restoration direction.
Step 4, using the highlight image after the unsaturated pixel point repair is completed as an input image, and repairing the saturated highlight pixels by using a priority-based adaptive direction method.
In order to further improve the repair quality of saturated pixel points, the embodiment of the application provides a priority-based self-adaptive direction method for solving the optimal repair direction:
wherein W is p Is a 3 x 3 rectangular block of pixels centered on specular highlight p, |W p I represents W p The total number of pixels included in the display. V (V) p Represents W p Is included in the display panel. T (T) p Indicating priority of highlight pixel repair, D n Gradient information representing different repair directions, D q The best repair direction is represented to find the best repair direction in that direction and the most appropriate candidate block in that direction.
W p A 3 x 3 rectangular frame is selected, with the specular highlight p as the center. W (W) q Is a 3 x 3 rectangular box selected in the optimal selection direction, wq' is returned W P The rectangle box with highest similarity. Then the local highlight region image I to be repaired b’ Can be expressed as:
ψ represents the concatenation, filling the best selected 3 x 3 pixel block into the area to be repaired. (u, v) represents direction information of the light, (x ', y') represents a best selected start point coordinate of the 3×3 pixel block, (x, y) represents a start point of the region to be repaired, I b’ Representing the restored image. And filling the rectangular frame into the area to be repaired.
In summary, the method according to the embodiment of the present application classifies the highlight pixels into the "saturated highlight pixels" and the "unsaturated highlight pixels" by combining the multi-view characteristics of the light field, thereby solving the limitation of the conventional highlight removal algorithm, effectively removing the local highlight region of the mirror surface in the image, and recovering the texture information of the image surface.
FIG. 4 is a schematic diagram of optimal sub-aperture image acquisition and highlight restoration according to an embodiment of the present application; as shown in fig. 4, obtaining an optimal sub-aperture image and performing highlight restoration includes the following steps:
step 1, obtaining an optimal sub-aperture image; decoding an original light field image Ir through a light field tool box to obtain an all-view sub-aperture image Iarray, fusing different sub-aperture images by adopting a multi-view image fusion method, removing local small-scale specular highlights, and obtaining an optimal sub-aperture image I o ；
Step 2, mirror pixel separation; extracting optimal sub-aperture image I by using x-means mean value clustering algorithm o Specular highlight pixels in (a). Refocusing the image, computing variance from the multi-viewpoint light field image, whereby the highlight pixels are divided into an unsaturated component Iuah and a saturated component Iah;
step 3, recovering unsaturated pixels; recovering unsaturated pixels by a mirror surface highlight pixel repairing method based on a dichromatic reflection model, wherein the repaired pixels are represented by Ib';
step 4, recovering saturated pixels; and repairing the saturated high-brightness pixels by adopting a priority-based self-adaptive direction method, wherein the repaired pixels are represented by Ib.
Finally, in order to more intuitively compare the difference between the method proposed by the present application and other methods, we selected three existing advanced highlight removal algorithms for quantitative comparison, as shown in fig. 5, and in order to ensure the effectiveness of the comparison, the subject herein included three sets of data sets (the light field data set of the university of stenford). The image local highlight region restoration method of the present application shows excellent results by comparing the enlarged highlight region in the rectangular block, regardless of the angle removed from the high reflection and the angle of highlight restoration.
The image local highlighting region restoration system provided by the embodiment of the application is described below, and the image local highlighting region restoration system described below and the image local highlighting region restoration method described above can be referred to correspondingly.
In addition, the embodiment of the application also provides a system for repairing the local highlight region of the image, which comprises the following steps:
the image acquisition module is used for fusing the sub-aperture images with different visual angles to obtain an optimal sub-aperture image;
the image classification module is used for classifying the highlight pixel points into saturated highlight pixel points and unsaturated highlight pixel points by taking the acquired optimal sub-aperture image as input and utilizing the multi-view characteristic of the light field and combining an x-means mean value clustering algorithm;
the first repair module is used for repairing the unsaturated highlight pixel by using a highlight pixel repair method based on a bicolor reflection model for the extracted unsaturated highlight pixel;
and the second restoration module is used for taking the highlight image after the unsaturated pixel point restoration is completed as an input image, and restoring the saturated highlight pixels by using a priority-based self-adaptive direction method.
It can be understood that the image local highlight region restoration system provided by the present application corresponds to an image local highlight region restoration method provided by the foregoing embodiments, and relevant technical features of an image local highlight region restoration system may refer to relevant technical features of an image local highlight region restoration method, which are not described herein.
Fig. 6 illustrates a physical schematic diagram of an electronic device, as shown in fig. 6, which may include: processor 810, communication interface (Communications Interface) 820, memory 830, and communication bus 840, wherein processor 810, communication interface 820, memory 830 accomplish communication with each other through communication bus 840. Processor 810 may invoke logic instructions in memory 830 to perform the steps of a method for repairing locally highlighted areas of an image.
Further, the logic instructions in the memory 830 described above may be implemented in the form of software functional units and may be stored in a computer-readable storage medium when sold or used as a stand-alone product. Based on this understanding, the technical solution of the present application may be embodied essentially or in a part contributing to the prior art or in a part of the technical solution, in the form of a software product stored in a storage medium, comprising several instructions for causing a computer device (which may be a personal computer, a server, a network device, etc.) to perform all or part of the steps of the method according to the embodiments of the present application. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a random access Memory (RAM, random Access Memory), a magnetic disk, or an optical disk, or other various media capable of storing program codes.
In another aspect, the present application also provides a computer program product comprising a computer program, the computer program being storable on a non-transitory computer readable storage medium, the computer program being capable of performing the steps of an image local highlight region restoration method as described above when being executed by a processor.
In yet another aspect, the present application also provides a non-transitory computer readable storage medium having stored thereon a computer program which, when executed by a processor, is implemented to perform the steps of an image local highlight region restoration method as described above.
The apparatus embodiments described above are merely illustrative, wherein the elements illustrated as separate elements may or may not be physically separate, and the elements shown as elements may or may not be physical elements, may be located in one place, or may be distributed over a plurality of network elements. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art will understand and implement the present application without undue burden.
From the above description of the embodiments, it will be apparent to those skilled in the art that the embodiments may be implemented by means of software plus necessary general hardware platforms, or of course may be implemented by means of hardware. Based on this understanding, the foregoing technical solution may be embodied essentially or in a part contributing to the prior art in the form of a software product, which may be stored in a computer readable storage medium, such as ROM/RAM, a magnetic disk, an optical disk, etc., including several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the method described in the respective embodiments or some parts of the embodiments.
While preferred embodiments of the present application have been described, additional variations and modifications in those embodiments may occur to those skilled in the art once they learn of the basic inventive concepts. It is therefore intended that the following claims be interpreted as including the preferred embodiments and all such alterations and modifications as fall within the scope of the application.
Finally, it should be noted that: the above embodiments are only for illustrating the technical solution of the present application, and are not limiting; although the application has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present application.
Claims (10)
1. A restoration method of local highlight region of image, which is used for removing and restoring the highlight of local mirror surface of the image collected by micro lens array light field camera; the method is characterized by comprising the following specific steps:
step 1, obtaining an optimal sub-aperture image by fusing sub-aperture images with different visual angles;
step 2, classifying the highlight pixels into saturated highlight pixels and unsaturated highlight pixels by taking the acquired optimal sub-aperture image as input and utilizing the multi-view characteristic of the light field and combining an x-means mean value clustering algorithm;
step 3, repairing the unsaturated highlight pixel by using a highlight pixel repairing method based on a bicolor reflection model for the extracted unsaturated highlight pixel;
step 4, using the highlight image after the unsaturated pixel point repair is completed as an input image, and repairing the saturated highlight pixels by using a priority-based adaptive direction method.
2. The method of claim 1, wherein obtaining the optimal sub-aperture image by fusing sub-aperture images of different viewing angles comprises:
and acquiring images, decoding and extracting sub-aperture images through a light field toolbox, fusing the sub-aperture images with different visual angles to eliminate highlight pixel points with small areas, and extracting an optimal sub-aperture image.
3. The method for repairing a locally highlighted area of an image according to claim 2, wherein said fusing the sub-aperture images of different viewing angles comprises the steps of:
step 1, selecting a central sub-aperture image I with higher image quality from a plurality of sub-aperture images c As a target image, separating out a highlight region { M } of the sub-aperture image by using an x-means mean value clustering algorithm;
step 2, according to ICP algorithm, aligning the images of different viewpoints to the sub-aperture image I through rigid body transformation matrix c ；
Step 3, by fusing the multi-view imagesObtaining a fused image I f From the fused image I according to the highlight region f Extracting a target area;
step 4, from the fused image I f Extracting pixels of the target area, and splicing the pixels to the missing area in the center view image to obtain an optimal image I o 。
4. The method for repairing a locally highlighted area of an image according to claim 1, wherein classifying the highlight pixels by using the multi-view characteristic of the light field and combining an x-means mean clustering algorithm simultaneously comprises: and differentiating the obtained optimal sub-aperture image, and differentiating each pixel of the optimal sub-aperture image.
5. The method for repairing a locally highlighted area of an image of claim 4, wherein said differencing each pixel of the optimal sub-aperture image specifically comprises: and extracting specular highlight pixels in the optimal sub-aperture image by using an x-means mean clustering algorithm, refocusing the image, and calculating variance through the multi-viewpoint light field image.
6. The method for repairing a locally highlight region of an image according to claim 1, wherein the repairing unsaturated highlight pixels by using a highlight pixel repairing method based on a bicolor reflection model specifically comprises:
original image I according to the bicolor reflection model ua From specular reflection component I s And diffuse reflection component I b Composition;
clustering the mirror Gao Guangxiang pixels and the diffuse reflection pixels by adopting an x-means mean value clustering algorithm, and calculating a threshold value of a diffuse reflection area;
evaluating the local highlight region, and repairing the diffuse reflection component I of the unsaturated highlight pixel by adopting a highlight pixel repairing method b 。
7. The method of claim 1, wherein the repairing saturated highlight pixels using a priority-based adaptive direction method comprises: the following formula is substituted:
wherein I is b’ Representing the restored image, ψ representing the image stitching of the light field, filling the best selected 3×3 pixel block into the region to be restored; (u, v) represents direction information of the light ray, (x ', y') represents a start point coordinate of the best selection of the 3 x 3 pixel block, and (x, y) represents a start point of the region to be repaired.
8. An image local highlight region restoration system, comprising:
the image acquisition module is used for fusing the sub-aperture images with different visual angles to obtain an optimal sub-aperture image;
the image classification module is used for classifying the highlight pixel points into saturated highlight pixel points and unsaturated highlight pixel points by taking the acquired optimal sub-aperture image as input and utilizing the multi-view characteristic of the light field and combining an x-means mean value clustering algorithm;
the first repair module is used for repairing the unsaturated highlight pixel by using a highlight pixel repair method based on a bicolor reflection model for the extracted unsaturated highlight pixel;
and the second restoration module is used for taking the highlight image after the unsaturated pixel point restoration is completed as an input image, and restoring the saturated highlight pixels by using a priority-based self-adaptive direction method.
9. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements a method for repairing locally highlight areas of an image as claimed in any one of claims 1 to 7 when executing the program.
10. A non-transitory computer readable storage medium having stored thereon a computer program, wherein the computer program when executed by a processor implements a method of image local highlight restoration as claimed in any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311254350.6A CN117132503A (en) | 2023-09-26 | 2023-09-26 | Method, system, equipment and storage medium for repairing local highlight region of image |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311254350.6A CN117132503A (en) | 2023-09-26 | 2023-09-26 | Method, system, equipment and storage medium for repairing local highlight region of image |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117132503A true CN117132503A (en) | 2023-11-28 |
Family
ID=88850960
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311254350.6A Pending CN117132503A (en) | 2023-09-26 | 2023-09-26 | Method, system, equipment and storage medium for repairing local highlight region of image |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117132503A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117474921A (en) * | 2023-12-27 | 2024-01-30 | 中国科学院长春光学精密机械与物理研究所 | Anti-noise light field depth measurement method, system and medium based on specular highlight removal |
-
2023
- 2023-09-26 CN CN202311254350.6A patent/CN117132503A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117474921A (en) * | 2023-12-27 | 2024-01-30 | 中国科学院长春光学精密机械与物理研究所 | Anti-noise light field depth measurement method, system and medium based on specular highlight removal |
| CN117474921B (en) * | 2023-12-27 | 2024-05-07 | 中国科学院长春光学精密机械与物理研究所 | Anti-noise light field depth measurement method, system and medium based on specular highlight removal |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Le et al. | Deep homography estimation for dynamic scenes | |
| WO2019148739A1 (en) | Comprehensive processing method and system for blurred image | |
| CN111652085B (en) | Object identification method based on combination of 2D and 3D features | |
| US8385630B2 (en) | System and method of processing stereo images | |
| CN103578116B (en) | For tracking the apparatus and method of object | |
| CN107025660B (en) | Method and device for determining image parallax of binocular dynamic vision sensor | |
| CN111126412B (en) | Image key point detection method based on characteristic pyramid network | |
| CN109242959B (en) | Three-dimensional scene reconstruction method and system | |
| CN111241924B (en) | Face detection and alignment method, device and storage medium based on scale estimation | |
| Chen et al. | Multi-scale adaptive dehazing network | |
| CN113657528B (en) | Image feature point extraction method and device, computer terminal and storage medium | |
| CN117132503A (en) | Method, system, equipment and storage medium for repairing local highlight region of image | |
| Huang et al. | Removing reflection from a single image with ghosting effect | |
| CN113807237A (en) | Training of in vivo detection model, in vivo detection method, computer device, and medium | |
| Wang et al. | Single-image dehazing using color attenuation prior based on haze-lines | |
| CN112348762A (en) | Single image rain removing method for generating confrontation network based on multi-scale fusion | |
| CN108961322B (en) | Mismatching elimination method suitable for landing sequence images | |
| CN113643290B (en) | Straw counting method and device based on image processing and storage medium | |
| CN112991419B (en) | Parallax data generation method, parallax data generation device, computer equipment and storage medium | |
| CN109360174B (en) | Three-dimensional scene reconstruction method and system based on camera pose | |
| Arvanitidou et al. | Short-term motion-based object segmentation | |
| Huang et al. | HRN: Haze-Relevant Network Using Multi-Object Constraints for Single Image Dehazing | |
| Su et al. | Exposing digital video logo-removal forgery by inconsistency of blur | |
| Yuan et al. | Single Image Defogging Method based on Deep Learning | |
| CN114821128B (en) | Scale-adaptive template matching method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |